# Distributed Differential Graphical Game for Control of Double-Integrator
Multi-Agent Systems with Input Delay

Hossein B. Jond The author is with the Department of Cybernetics, Faculty of
Electrical Engineering, Czech Technical University in Prague, Czech Republic
(e-mail: hossein.jond@fel.cvut.cz).

###### Abstract

This paper studies cooperative control of noncooperative double-integrator
multi-agent systems (MASs) with input delay on connected directed graphs in
the context of a differential graphical game (DGG). In the distributed DGG,
each agent seeks a distributed information control policy by optimizing an
individual local performance index (PI) of distributed information from its
graph neighbors. The local PI, which quadratically penalizes the agent’s
deviations from cooperative behavior (e.g., the consensus here), is
constructed through the use of the graph Laplacian matrix. For DGGs for
double-integrator MASs, the existing body of literature lacks the explicit
characterization of Nash equilibrium actions and their associated state
trajectories with distributed information. To address this issue, we first
convert the $N$-player DGG with $m$ communication links into $m$ coupled
optimal control problems (OCPs), which, in turn, convert to the two-point
boundary-value problem (TPBVP). We derive the explicit solutions for the TPBV
that constitute the explicit distributed information expressions for Nash
equilibrium actions and the state trajectories associated with them for the
DGG. An illustrative example verifies the explicit solutions of local
information to achieve fully distributed consensus.

Consensus, differential graphical game (DGG), distributed information, input
delay, multi-agent system (MAS), Nash equilibrium.

##  1 Introduction

Cooperation is fundamental in distributed biological multi-agent systems
(MASs) for certain eco-evolutionary advantages [1]. Cooperative human-
engineered distributed MASs are a research trend that draws inspiration from
biological systems like flocks of birds or bacterial colonies. In the past
decades, cooperative control of MASs on graphs has received increasing
attention due to its extensive applications such as consensus [2],
synchronization [3], flocking [4], formation [5], rendezvous [6], and so on.
Classical control designs for such systems are centralized and require global
knowledge of the system or a complete communication graph, which is excessive
and conflicts with the communication infrastructure of MAS[7, 8]. Cooperative
distributed control of MAS in terms of optimality is of prime interest.
However, finding a locally distributed control that is optimal in some sense,
e.g., when each agent is optimizing its own local performance index (PI), is
particularly demanding [9]. For noncooperative MASs on graphs, this can be
achieved by a differential graphical game (DGG) [10], where the notion of
optimality is Nash equilibrium.

DGGs are a significant class of differential games with applications to the
distributed control of networked systems on graphs. In recent years, there has
been an increased interest in studying various cooperative control problems
using differential (or dynamic) games (see, e.g.,consensus [11, 12], flocking
[13], synchronization [14, 15], formation [7, 16], and pursuit-evasion [17,
18]). The challenges of applying DGG-based control are twofold. First,
differential games, in general, are difficult to solve [19, 20, 10]. Second,
even if a solution is found, it requires global information. However, there
have been attempts to overcome these challenges, for instance, by estimation
[21], constructing local solutions to global Nash [22], iterative control laws
that converge to the Nash equilibrium [23], neural network-based integrated
heuristic dynamic programming [24], reinforcement learning for Nash [25],
value function approximation [10]. Solving a DGG also depends on whether it is
discrete-time [12] or Stackelberg type [26]. This paper focuses on a
continuous-time DGG of the Nash type.

Due to its analytical tractability, the framework of linear-quadratic (LQ)
differential games is popular for analyzing MASs. The double-integrator MASs
on graphs can be modeled as LQ DGGs with quadratic performance indices (PIs)
[28, 27, 7, 14]. The emerging noncooperative DGG problem in this study is in
fact a continuous-time LQ Nash differential game. The open-loop Nash
equilibrium solution as well as its uniqueness and existence for this game are
given in [29, 30]. Nevertheless, the given Nash equilibrium solution requires
global state information [27, 7]. The main challenge in noncooperative
differential games is finding a distributed information Nash equilibrium
solution that can be executed locally [31, 7]. Recently, a distributed
adaptive Nash equilibrium solution for DGGs with an infinite planning horizon
was proposed [14]. In infinite-horizon differential games, there is no
specified terminal time nor a terminal state constraint in the PIs associated
with players, and the solution to the HJB equation for nonlinear systems or
the Riccati equation for linear systems is time-invariant. In the case of a
finite horizon, a terminal state constraint is usually included in the PIs,
and the solution to either the HJB equation or the Riccati equation is time-
dependent. Finding a distributed solution with a finite planning horizon is a
great challenge. A distributed estimation of Nash equilibrium for DDGs with a
finite planning horizon has been constituted in [7]. On the other hand,
explicit expressions have not been found except for differential games with
simplified single-integrator dynamics in 1-dimensional coordinates (e.g., see
[32, 33, 34]). The input delay considered in this paper even makes finding
such a solution more difficult.

Commonly, a time delay occurs in the control input when the control action
depends on the relative state information transmitted over the wireless
communication network. Input-delayed cooperative MASs have been extensively
studied by the control community. However, only a few research works have been
done in the game theoretical control of MASs with input delay (e.g., [35]).

In this paper, cooperative control of double-integrator MASs on graphs is
formulated as an LQ DGG. The Nash equilibrium of the DGG is a mutually
beneficial control strategy that nobody will deviate from. Therefore, it can
be exploited as a self-enforcing MAS control strategy. The main contribution
of the paper is deriving a distributed explicit expression of time, delay, and
the initial state for the LQ DGG with input delay. Our results differs from
similar works in [27, 7, 30, 28, 32, 33, 35] as follows:

  1. 1.

The DGG in this paper has both a terminal PI of state and a running PI of
state and control with input delay. The DGG in [27, 7] has a terminal PI and a
running PI of only control without delay. The DGG in [32, 33] has only running
PI of state and control without delay and without terminal PI.

  2. 2.

[30, 28, 35] do not offer an explicit distributed solution for DGGs. While
[30, 28] presented a numerical non-distributed solution, [35] proposed a
distributed Nash equilibrium-seeking algorithm with a distributed observer for
MASs with input delay that converges asymptotically.

The rest of this paper is organized as follows. Section 2 provides
preliminaries. The DGG for cooperative control of double-integrator multi-
agent systems is introduced in Section 3. In Section 4, the introduced DGG is
converted to a set of coupled optimal control problems (OCPs) where the
explicit solution for the latter is derived. Section 5 illustrates the
simulation results for the cooperative consensus control of double-integrator
multi-agent systems with input delay. The conclusion is given in Section 6.

##  2 Preliminaries

###  2.1 Kronecker Product

Let $X\in\mathit{\mathbb{R}}^{m\times n},Y\in\mathit{\mathbb{R}}^{p\times q}$
be real-valued (or complex-valued) matrices. The Kronecker product is defined
as

$X\otimes Y=\begin{bmatrix}x_{11}Y&\cdots&x_{1n}Y\\\ \vdots&\ddots&\vdots\\\
x_{m1}Y&\cdots&x_{mn}Y\end{bmatrix}\in\mathit{\mathbb{R}}^{mp\times nq}.$

The Kronecker product has the following properties [36]

$\displaystyle(X\otimes Y)^{\top}=X^{\top}\otimes Y^{\top},\quad(X\otimes
Y)^{-1}=X^{-1}\otimes Y^{-1},$ $\displaystyle(X\otimes Y)(U\otimes
V)=XU\otimes YV,\quad\mathrm{e}^{X\otimes
Y}=\mathrm{e}^{Y}\otimes\mathrm{e}^{X},$ $\displaystyle\mathrm{det}(X\otimes
Y)=(\mathrm{det}X)^{m}(\mathrm{det}Y)^{n},X\in\mathit{\mathbb{R}}^{n\times
n},Y\in\mathit{\mathbb{R}}^{m\times m}.$ (1)

###  2.2 Graph Theory

A directed graph is a pair $\mathcal{G}(\mathcal{V},\mathcal{E})$ where
$\mathcal{V}$ is a finite set of vertices/nodes and
$\mathcal{E}\subseteq\\{(i,j):i,j\in\mathcal{V}\\}$ is a set of directed
edges/arcs. Each edge $(i,j)\in\mathcal{E}$ represents an information flow
from node $i$ to node $j$ and is assigned a positive weight $\mu_{ij}>0$. The
set of neighbors of vertex $i$ is defined by
$\mathcal{N}_{i}=\\{j\in\mathcal{V}:(i,j)~{}\text{or}~{}(j,i)\in\mathcal{E},j\neq
i\\}$. Graph $\mathcal{G}$ is connected if for every pair of vertices
$(i,j)\in\mathcal{V}\times\mathcal{V}$, from $i$ to $j$ for all
$j\in\mathcal{V},j\neq i$, there exists a path of (undirected) edges from
$\mathcal{E}$.

Matrix $D\in\mathit{\mathbb{R}}^{\mathcal{|V|}\times\mathcal{|E|}}$ is the
incidence matrix of $\mathcal{G}$ where $D$’s $uv$th element is 1 if the node
$u$ is the head of the edge $v$, $-1$ if the node $u$ is the tail, and $0$,
otherwise. The distributed Laplacian matrix for each node $i$ that depends
only on information about that node and its neighbors in the graph is defined
as

$L_{i}=DW_{i}D^{\top}\in\mathbb{R}^{\mathcal{|V|}\times\mathcal{|V|}}$

where $W_{i}=\mathrm{diag}(\cdots,\mu_{ij},\cdots)~{}\forall
j\in\mathcal{N}_{i}\in\mathit{\mathbb{R}}^{\mathcal{|E|}\times\mathcal{|E|}}$.

The Kronecker product is used to extend the dimension of a matrix. The
extended dimension Laplacian matrix $\hat{L}_{i}$ is defined as

$\hat{L}_{i}=\hat{D}\hat{W}_{i}\hat{D}^{\top}\in\mathbb{R}^{m\mathcal{|V|}\times
m\mathcal{|V|}}$

where $\hat{D}=D\otimes I_{q\times q}$, $\hat{W}_{i}=W_{i}\otimes I_{q\times
q}$, and $I_{q\times q}$ is the identity matrix of dimension
$q\in\mathit{\mathbb{N}}$. The $m$-dimensional graph Laplacian $\hat{L}_{i}$
is symmetric $(\hat{L}_{i}=\hat{L}_{i}^{\top})$, positive semidefinite
$(\hat{L}_{i}\geq 0)$, and satisfies the (local) sum-of-squares property [4]

$\sum_{\forall
j\in\mathcal{N}_{i}}\mu_{ij}\|x_{i}-x_{j}\|^{2}=x^{\top}\hat{L}_{i}x$ (2)

where $x=[x_{1},\cdots,x_{\mathcal{|V|}}]^{\top}$ is a nonzero vector of all
nodes states $x_{i}\in\mathbb{R}^{q\times 1}$ and $\lVert.\rVert$ is the
Euclidean norm.

##  3 Differential Graphical Game

Consider a MAS $\mathcal{V}=\\{0,1,\cdots,N\\}$ with $N+1$ agents, where 0
corresponds to an externally controlled agent and the rest are agents with
their control input to be designed. The inter-agent communications are
restricted by a connected directed graph
$\mathcal{G}(\mathcal{V},\mathcal{E})$. Each agent $i\in\\{1,\cdots,N\\}$’s
nonlinear dynamics is reduced to a double-integrator model as follows using
the feedback linearization technique (e.g., see unmanned aerial vehicle (UAV)
linearization in [27, 37] and distributed generators (DGs) linearization in
[14])

$\ddot{p}_{i}(t)=u_{i}(t-\tau)$ (3)

where $p_{i}\in\mathbb{R}^{q\times 1}$ and $u_{i}\in\mathbb{R}^{q\times 1}$
are the $q$-dimensional information and control input vectors, respectively,
and $\tau>0$ is a constant time delay ($u_{i}(t)=0$ when $t<\tau$). The
dynamics (3) is equivalent to the following linear time-invariant agent
dynamics with input delay

$\begin{bmatrix}\dot{p}_{i}(t)\\\
\ddot{p}_{i}(t)\end{bmatrix}=(\begin{bmatrix}0&1\\\ 0&0\end{bmatrix}\otimes
I_{q\times q})\begin{bmatrix}p_{i}(t)\\\
\dot{p}_{i}(t)\end{bmatrix}+(\begin{bmatrix}0\\\ 1\end{bmatrix}\otimes
I_{q\times q})u_{i}(t-\tau).$ (4)

Let
$x(t)=[p_{0}^{\top}(t),\cdots,p_{N}^{\top}(t),\dot{p}_{0}^{\top}(t),\cdots,\dot{p}_{N}^{\top}(t)]^{\top}\in\mathbb{R}^{2(N+1)q\times
1}$ be the state vector for $\mathcal{V}$. The state dynamics of the system
with a given initial state can then be described by

$\dot{x}(t)=A_{0}x(t)+\sum_{i=0}^{N}B_{i}u_{i}(t-\tau),\quad x(0)=x_{0}$ (5)

where $A_{0}=A\otimes I_{q\times q}\in\mathbb{R}^{2(N+1)q\times 2(N+1)q}$,
$A=\begin{bmatrix}0&1\\\ 0&0\end{bmatrix}\otimes
I_{(N+1)\times(N+1)}\in\mathbb{R}^{2(N+1)\times 2(N+1)}$,
$B_{i}=\begin{bmatrix}0_{(N+1)\times 1}\\\ b_{i}\end{bmatrix}\otimes
I_{q\times q}\in\mathbb{R}^{2(N+1)q\times q}$,
$b_{i}=[0,\cdots,1,\cdots,0]^{\top}\in\mathbb{R}^{(N+1)\times 1}$, and
$0_{(N+1)\times 1}$ is the zero vector of dimension $N+1$.

In the context of a DGG, each agent $i\in\\{1,\cdots,N\\}$ is so-called a
player of the DGG, $\\{1,\cdots,N\\}$ corresponds to the set of players, and
the system dynamics (5) represents the state dynamics of the DGG. In the DGG
approach to cooperative control of the noncooperative MAS (5), each player
seeks to optimize a local PI that solely reflects its interests by finding a
suitable control policy. Define the finite horizon PI

$J_{i}=C_{i}(x(T),T)+\int_{0}^{T}L_{i}(x(t),u_{i}(t-\tau),t)~{}\mathrm{dt}$
(6)

for each player $i\in\\{1,\cdots,N\\}$ where $C_{i}(x(T),T)$ is the terminal
PI at horizon time $T$ and $L_{i}(x(t),u_{i}(t-\tau),t)$ is the running PI
over the entire horizon. Note that $C_{i}$ and $L_{i}$ are associated with
distributed information available only to agent $i$ locally through the graph
$\mathcal{G}(\mathcal{V},\mathcal{E})$. Each player $i\in\\{1,\cdots,N\\}$
runs an optimization to find an admissible control policy
$u_{i}(t-\tau)\in\mathcal{U}_{i}$ that optimizes its unified PI $J_{i}$
subject to the state dynamics (5).

Various cooperative control problems can be depicted by a distributed PI (6)
that quadratically penalizes the local behavior of an agent. In this paper, we
define $C_{i}(x(T),T)$ and $L_{i}(x(t),u_{i}(t-\tau),t)$ mathematically for
consensus-seeking agents. Thereby, the vectors $p_{i}(t)$, $\dot{p}_{i}(t)$,
and $u_{i}(t)$ represent the position, velocity, and acceleration input for
agent $i$ in an $q$-dimensional space. Under the DGG framework, each player’s
aim is to achieve consensus while minimizing their PI $J_{i}$.

The MAS in $\mathcal{V}$ governed by the state dynamics (5) and the graph
$\mathcal{G}(\mathcal{V},\mathcal{E})$ is said to achieve consensus if for any
initial position $p_{i}(0)$ and initial velocity $\dot{p}_{i}(0)$, $\forall
i\in\\{1,\cdots,N\\}$,

$\sum_{j\in\mathcal{N}_{i}}\mu_{ij}\left(\|p_{i}(t)-p_{j}(t)\|^{2}+\|\dot{p}_{i}(t)-\dot{p}_{j}(t)\|^{2}\right)\rightarrow
0$ (7)

as $t\rightarrow T$ and $T>\tau$ [38]. In the DGG context, a weighted sum of
local consensus errors and local velocity errors at the horizon can be defined
for each player $i\in\\{1,\cdots,N\\}$ as

$\displaystyle C_{i}$ $\displaystyle(x(T),T)=$
$\displaystyle\sum_{j\in\mathcal{N}_{i}}\omega_{ij}\left(\|p_{i}(T)-p_{j}(T)\|^{2}+\|\dot{p}_{i}(T)-\dot{p}_{j}(T)\|^{2}\right)$
(8)

where $\omega_{ij}>0$ is a scalar. In addition to the least (7) and (3), each
player at the same time naturally seeks to expend the least control effort
over the entire horizon. Therefore, a reasonable running PI for player
$i\in\\{1,\cdots,N\\}$ is

$\displaystyle L_{i}$ $\displaystyle(x(t),u_{i}(t-\tau),t)=r_{i}\lVert
u_{i}(t-\tau)\rVert^{2}+$
$\displaystyle\sum_{j\in\mathcal{N}_{i}}\mu_{ij}\left(\|p_{i}(t)-p_{j}(t)\|^{2}+\|\dot{p}_{i}(t)-\dot{p}_{j}(t)\|^{2}\right)$
(9)

where $r_{i}>0$ is a scalar penalizing the control effort term.

The PIs (3) and (3) can be depicted in compact form by using the sum-of-
squares property (2) as follows

$\displaystyle C_{i}(x(T),T)$ $\displaystyle=x^{\top}(T)\big{(}I_{2\times
2}\otimes\hat{L}_{iT}\big{)}x(T),$ (10) $\displaystyle
L_{i}(x(t),u_{i}(t-\tau),t)$ $\displaystyle=x^{\top}(t)\big{(}I_{2\times
2}\otimes\hat{L}_{i}\big{)}x(t)+$ $\displaystyle
u_{i}^{\top}(t-\tau)R_{i}u_{i}(t-\tau)$ (11)

where $\hat{L}_{iT}=\hat{D}\hat{W}_{iT}\hat{D}^{\top}$, $\hat{D}=D\otimes
I_{q\times q}$, $\hat{W}_{iT}=W_{iT}\otimes I_{q\times q}$,
$W_{iT}=\mathrm{diag}(\cdots,\omega_{ij},\cdots)~{}\forall
j\in\mathcal{N}_{i}\in\mathit{\mathbb{R}}^{m\times m}$ (where
$m=|\mathcal{E}|$), and $R_{i}=r_{i}\otimes I_{q\times q}$. Therefore, the
minimization problem for each player $i\in\\{1,\cdots,N\\}$ consists of the
unified PI (6) with components in compact form (10)-(11) subject to state
dynamics (5).

The requirement for consensus (7) can be easily generalized to most
cooperative control problems, such as formation control. Accordingly, similar
expressions to (3) and (3) and thereby the compact forms (10) and (11) can be
acquired. For formation control, the compact form of PIs is given in [39].

Each player’s optimal strategy is the solution to its optimization problem.
Under an open-loop information structure in the DGG (5) and (6), all players
simultaneously determine their strategies at the beginning of the game and use
this open-loop strategy for the entire horizon. The players must then adjust
their control inputs based on the delayed information. In a noncooperative
DGG, the players of the game cannot make binding agreements. Therefore, the
solution ought to be self-enforcing, such that, once agreed upon, no one has
the incentive to deviate from it [40]. A Nash equilibrium possesses the
characteristic that any individual player’s decision to deviate unilaterally
does not result in a lower cost for that player. It is a strategy combination
of all players in a noncooperative game where no one can achieve a lower cost
by unilaterally deviating from it. The open-loop Nash equilibrium is defined
as a set of admissible actions ($u_{1}^{*},\cdots,u_{N}^{*}$) if for all
admissible ($u_{1},\cdots,u_{N}$) the following inequalities

$\displaystyle J_{i}(u_{1}^{*},\cdots,u_{i-1}^{*},u_{i}^{*},u_{i+1}^{*}$
$\displaystyle,\cdots,u_{N}^{*})\leq$ $\displaystyle
J_{i}(u_{1}^{*},\cdots,u_{i-1}^{*},u_{i},u_{i+1}^{*},\cdots,u_{N}^{*})$

hold for $i\in\\{1,\cdots,N\\}$ where $u_{i}\in\Gamma_{i}$ and
$\Gamma_{i}=\\{u_{i}(t,x_{0})|t\in[0,T]\\}$ is the admissible strategy set for
player $i$. We assume that $u_{i}$ ($\forall i\in\\{1,\cdots,N\\}$) consists
of the set of measurable functions from $[0,T]$ into $\Gamma_{i}$ for which
the differential equation (5) has a unique solution and the PI (6) exists.

The noncooperative DGG problem in (5) and (6) for $\tau=0$ admits a unique
Nash equilibrium solution for every initial state $x_{0}$ iff there exists a
solution set to a set of the coupled (asymmetric) Riccati differential
equations (see Theorem 7.2 in [30]) or, equivalently, iff a specific matrix is
invertible (see Theorem 7.1 in [30]). Moreover, Nash equilibrium requires
global knowledge of the initial state vector $x(0)$. Therefore, the existence
of a unique open-loop Nash equilibrium and whether or not the equilibrium and
its associated state trajectories are distributed, as well as the input delay,
are the main issues that need to be addressed for the DGG problem (5) and (6).

##  4 Main Results

In this section, we present an explicit solution of distributed Nash actions
and their associated state trajectories to the input delayed DGG problem in
(5) and (6). To the best of our knowledge, such solutions have not yet been
reported for the LQ differential game. The presented solution results from
applying the proposed systematic approach in Fig. 1. The first step in this
approach is converting the $N$-player DGG problem to $m$ OCPs. In terms of the
MAS on the graph $\mathcal{G}(\mathcal{V},\mathcal{E})$, this is as converting
the $(N+1)(=|\mathcal{V}|)$-node MAS to the $m(=|\mathcal{E}|)$-edge system.

Figure 1: Block diagram representation of the proposed systematic approach to
solving the DGG.

Toward this, we define the following new state vector and new control inputs

$\displaystyle z(t)=(I_{2\times
2}\otimes\hat{D}^{\top})x(t)\in\mathbb{R}^{2Mq\times 1},$ (12)
$\displaystyle\xi_{i}(t)=\begin{bmatrix}0&\cdots&I_{q\times
q}&\cdots&0\end{bmatrix}\hat{D}^{\top}\begin{bmatrix}u_{0}(t)\\\ \vdots\\\
u_{N}(t)\end{bmatrix}\in\mathbb{R}^{q\times 1}.$ (13)

The state dynamics (5) in terms of the state vector $z(t)$ and control inputs
$\xi_{i}(t)$ is given by

$\dot{z}(t)=\hat{A}_{0}z(t)+\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(t-\tau),\quad
z(0)=z_{0}$ (14)

where $\hat{A}_{0}=\hat{A}\otimes I_{q\times q}\in\mathbb{R}^{2mq\times 2mq}$,
$\hat{A}=\begin{bmatrix}0&1\\\ 0&0\end{bmatrix}\otimes I_{m\times
m}\in\mathbb{R}^{2m\times 2m}$, $\hat{B}_{i}=\begin{bmatrix}0\\\
b_{i}\end{bmatrix}\otimes I_{q\times q}\in\mathbb{R}^{2mq\times q}$,
$\hat{b}_{i}=[0,\cdots,1,\cdots,0]^{\top}\in\mathbb{R}^{m\times 1}$, and
$z_{0}=(I_{2\times 2}\otimes\hat{D}^{\top})x_{0}$.

The PIs (10) and (11) are rewritten as follows

$\displaystyle C_{i}(x(T),T)=x^{\top}(T)\big{(}I_{2\times
2}\otimes\hat{D}\hat{W}_{iT}\hat{D}^{\top}\big{)}x(T)$
$\displaystyle=x^{\top}(T)\big{(}I_{2\times
2}\otimes\hat{D}\big{)}\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}\big{(}I_{2\times
2}\otimes\hat{D}^{\top}\big{)}x(T)$ $\displaystyle=\Big{(}\big{(}I_{2\times
2}\otimes\hat{D}^{\top}\big{)}x(T)\Big{)}^{\top}\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}\big{(}I_{2\times
2}\otimes\hat{D}^{\top}\big{)}x(T)$
$\displaystyle=z^{\top}(T)\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}z(T)\triangleq\mathcal{C}_{i}(z(T),T),$ (15)
$\displaystyle\mathcal{L}_{i}(z(t),\xi_{i}(t-\tau),t)\triangleq$
$\displaystyle z^{\top}(t)\big{(}I_{2\times
2}\otimes\hat{W}_{i}\big{)}z(t)+\xi_{i}^{\top}(t-\tau)\hat{R}_{i}\xi_{i}(t-\tau),$
(16)

where $\hat{R}_{i}=\hat{r}_{i}\otimes I_{q\times q}$. Without loss of
generality, we assume $r_{i}=r_{j}\forall(i,j)\in\mathcal{E}$, i.e.,
$\hat{r}_{k}=r_{i}=r_{j}$ for the $k$th edge $(i,j)\in\mathcal{E}$ in the
$k$th column of $D$. As a result, the following PI emerges

$\mathcal{J}_{i}=\mathcal{C}_{i}(z(T),T)+\int_{0}^{T}\mathcal{L}_{i}(z(t),\xi_{i}(t-\tau),t)~{}\mathrm{dt}$
(17)

for $i\in\\{1,\cdots,m\\}$. The $N$-player DGG problem with input delay in (5)
and (6) is reduced to $m$ OCPs with input delay in (14) and (17). In other
words, the transition from the $(N+1)$-node MAS to the $m$-edge system, as
shown in Fig. 1, is complete. The latter is analytically more tractable to
investigate for an explicit solution.

The existence and uniqueness of the optimal control policies $\xi_{i}(t)$s and
their associated state trajectory $z(t)$ for the $m$ OCPs (14) and (17) can be
investigated by applying the necessary conditions for optimality using
Pontryagin’s principle. As a result, these problems are converted into the
two-point boundary-value problem (TPBVP). In the context of the DGG (5) and
(6), the Laplacian matrices $\hat{L}_{i}$ and $\hat{L}_{iT}$ that occur in the
performance index (6) are symmetric and positive semidefinite, and $R_{i}$ are
positive definite. According to [41], then for the $N$-player differential
game, the PIs $J_{i}$ ($i\in\\{1,\cdots,N\\}$) are strictly convex function of
$u_{i}$ for all admissible control functions $u_{j}$, $j\neq i$, and for all
$x_{0}$. This implies that the conditions obtained from Pontryagin’s principle
are both necessary and sufficient for consensus control of the given MAS.
Similarly, the matrices $\hat{W}_{i}$ and $\hat{W}_{iT}$ in the performance
index (17) are also symmetric and positive semidefinite, and $\hat{R}_{i}$ are
positive definite. Therefore, also for the $m$ OCPs, the conditions obtained
from Pontryagin’s principle are both necessary and sufficient.

Before proceeding with applying the necessary conditions to the $m$ OCPs in
order to obtain the TPBVP, one should address the input delay. A common
approach in dealing with the input delayed linear dynamical system (14) is to
apply an integral transformation to obtain a delay-free linear dynamical
system [42]. Below, we convert the input delayed OCPs (14) and (17) to the
delay-free OCPs.

The PI (17) can be decomposed by

$\displaystyle\mathcal{J}_{i}=$ $\displaystyle\mathcal{C}_{i}(z(T),T)+$
$\displaystyle\int_{0}^{\tau}\mathcal{L}_{i}(z(t),t)~{}\mathrm{dt}+\int_{0}^{T-\tau}\mathcal{L}_{i}(z(t+\tau),\xi_{i}(t),t)~{}\mathrm{dt}$

where the middle integral term on the right-hand side is a constant since the
control does not take place for $t\in[0,\tau[$. Therefore, the minimization of
$\mathcal{J}_{i}$ is equivalent to the minimization of

$\displaystyle\hat{\mathcal{J}}_{i}=\mathcal{C}_{i}(z(T),T)+\int_{0}^{T-\tau}\mathcal{L}_{i}(z(t+\tau),\xi_{i}(t),t)~{}\mathrm{dt}.$
(18)

The solution of (14) at $t+\tau$ is given by

$\displaystyle z(t+\tau)$ $\displaystyle=\mathrm{e}^{(t+\tau)\hat{A}_{0}}z(0)$
$\displaystyle+\int_{0}^{t+\tau}\mathrm{e}^{(t+\tau-s)\hat{A}_{0}}\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(s-\tau)~{}\mathrm{ds}$
$\displaystyle=\mathrm{e}^{\tau\hat{A}_{0}}\Big{(}\mathrm{e}^{t\hat{A}_{0}}z(0)+\int_{0}^{t}\mathrm{e}^{(t-s)\hat{A}_{0}}\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(s-\tau)~{}\mathrm{ds}$
$\displaystyle+\int_{t}^{t+\tau}\mathrm{e}^{(t-s)\hat{A}_{0}}\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(s-\tau)~{}\mathrm{ds}\Big{)}.$
(19)

The first and second terms inside the outer bracket pair on the right-hand
side denote the solution of (14) at $t$. Thereby,

$\displaystyle z(t+\tau)$
$\displaystyle=\mathrm{e}^{\tau\hat{A}_{0}}\left(z(t)+\int_{t}^{t+\tau}\mathrm{e}^{(t-s)\hat{A}_{0}}\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(s-\tau)~{}\mathrm{ds}\right)$
$\displaystyle=\mathrm{e}^{\tau\hat{A}_{0}}\left(z(t)+\int_{t-\tau}^{t}\mathrm{e}^{(t-s-\tau)\hat{A}_{0}}\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(s)~{}\mathrm{ds}\right)$
$\displaystyle\triangleq\mathrm{e}^{\tau\hat{A}_{0}}y(t).$ (20)

By letting $t=0$ and $t=T-\tau$, we obtain
$\mathrm{e}^{-\tau\hat{A}_{0}}z(\tau)=y(0)$ and
$z(T)=\mathrm{e}^{\tau\hat{A}_{0}}y(T-\tau)$, respectively. By substituting
(4) into (18), we have

$\displaystyle\hat{\mathcal{J}}_{i}=\hat{\mathcal{C}}_{i}(y(T-\tau),T-\tau)+\int_{0}^{T-\tau}\hat{\mathcal{L}}_{i}(y(t),\xi_{i}(t),t)~{}\mathrm{dt}$
(21)

where

$\displaystyle\mathcal{C}(z(T$ $\displaystyle),T)=z^{\top}(T)\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}z(T)$
$\displaystyle=\left(\mathrm{e}^{\tau\hat{A}_{0}}y(T-\tau)\right)^{\top}\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}\left(\mathrm{e}^{\tau\hat{A}_{0}}y(T-\tau)\right)$
$\displaystyle=y^{\top}(T-\tau)\left(\mathrm{e}^{\tau\hat{A}_{0}^{\top}}\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}\mathrm{e}^{\tau\hat{A}_{0}}\right)y(T-\tau)$
$\displaystyle\triangleq\hat{\mathcal{C}}_{i}(y(T-\tau),T-\tau)$

and

$\displaystyle\mathcal{L}_{i}(z($ $\displaystyle t+\tau),\xi_{i}(t),t)$
$\displaystyle=z^{\top}(t+\tau)\big{(}I_{2\times
2}\otimes\hat{W}_{i}\big{)}z(t+\tau)+\xi_{i}^{\top}(t)\hat{R}_{i}\xi_{i}(t)$
$\displaystyle=y^{\top}(t)\left(\mathrm{e}^{\tau\hat{A}_{0}^{\top}}\big{(}I_{2\times
2}\otimes\hat{W}_{i}\big{)}\mathrm{e}^{\tau\hat{A}_{0}}\right)y(t)+\xi_{i}^{\top}(t)\hat{R}_{i}\xi_{i}(t)$
$\displaystyle\triangleq\hat{\mathcal{L}}_{i}(y(t),\xi_{i}(t),t).$

We notice that $\hat{A}^{2}=0$ and
$\hat{A}_{0}^{2}=(\hat{A}_{0}^{\top})^{2}=0$. The matrix exponential terms can
then be rewritten as

$\displaystyle\mathrm{e}^{\tau\hat{A}_{0}^{\top}}$
$\displaystyle\big{(}I_{2\times
2}\otimes\hat{W}_{iT}\big{)}\mathrm{e}^{\tau\hat{A}_{0}}=$
$\displaystyle(I+\tau\hat{A}_{0}^{\top})(I_{2\times
2}\otimes\hat{W}_{iT})(I+\tau\hat{A}_{0})\triangleq\hat{Q}_{iT},$
$\displaystyle\mathrm{e}^{\tau\hat{A}_{0}^{\top}}$
$\displaystyle\big{(}I_{2\times
2}\otimes\hat{W}_{i}\big{)}\mathrm{e}^{\tau\hat{A}_{0}}=$
$\displaystyle(I+\tau\hat{A}_{0}^{\top})(I_{2\times
2}\otimes\hat{W}_{i})(I+\tau\hat{A}_{0})\triangleq\hat{Q}_{i}.$

Therefore, the PIs $\hat{\mathcal{C}}_{i}(y(T-\tau),T-\tau)$ and
$\hat{\mathcal{L}}_{i}(y(t),\xi_{i}(t),t)$ in (21) are simplified as

$\displaystyle\hat{\mathcal{C}}_{i}(y(T-\tau),T-\tau)=y^{\top}(T-\tau)\hat{Q}_{iT}y(T-\tau),$
(22)
$\displaystyle\hat{\mathcal{L}}_{i}((t),\xi_{i}(t),t)=y^{\top}(t)\hat{Q}_{i}y(t)+\xi_{i}^{\top}(t)\hat{R}_{i}\xi_{i}(t).$
(23)

The time derivative of (4) is given by

$\displaystyle\mathrm{e}^{\tau\hat{A}_{0}}\dot{y}(t)=\dot{z}(t+\tau)=\hat{A}_{0}z(t+\tau)+\sum_{i=1}^{m}\hat{B}_{i}\xi_{i}(t)$
(24)

or equivalently,

$\displaystyle\dot{y}(t)=\mathrm{e}^{-\tau\hat{A}_{0}}\hat{A}_{0}z(t+\tau)+\sum_{i=1}^{m}\hat{B}_{i0}\xi_{i}(t)$
(25)

where $\hat{B}_{i0}=\mathrm{e}^{-\tau\hat{A}_{0}}\hat{B}_{i}$. We also notice

$\displaystyle\mathrm{e}^{\pm\tau\hat{A}_{0}}\hat{A}_{0}=(\hat{A}_{0}\pm\tau\hat{A}_{0}^{2})=\hat{A}_{0}\mathrm{e}^{\pm\tau\hat{A}_{0}}=\hat{A}_{0}.$
(26)

Using the simplification above, we get

$\displaystyle\dot{y}(t)$
$\displaystyle=\hat{A}_{0}z(t+\tau)+\sum_{i=1}^{m}\hat{B}_{i0}\xi_{i}(t).$
(27)

Substituting $z(t+\tau)=\mathrm{e}^{\tau\hat{A}_{0}}y(t)$ into (27) yields

$\displaystyle\dot{y}(t)$
$\displaystyle=\hat{A}_{0}\mathrm{e}^{\tau\hat{A}_{0}}y(t)+\sum_{i=1}^{m}\hat{B}_{i0}\xi_{i}(t)$
(28)

and re-substituting (26), finally the delay-free state dynamics is given by

$\displaystyle\dot{y}(t)$
$\displaystyle=\hat{A}_{0}y(t)+\sum_{i=1}^{m}\hat{B}_{i0}\xi_{i}(t),y(0)=(I-\tau\hat{A}_{0})z(\tau).$
(29)

Now, we apply the conditions obtained from Pontryagin’s principle to the
delay-free $m$ OCPs in (29) and (21). According to Pontryagin’s principle, the
Hamiltonian

$\displaystyle H_{i}$
$\displaystyle=\hat{\mathcal{L}}_{i}(y(t),\xi_{i}(t),t)+\psi_{i}^{\top}(t)\dot{y}(t)$
(30)

is minimized with respect to $\xi_{i}(t)$ (for $i=1,\cdots,m$). This gives the
necessary conditions

$\xi_{i}(t)=-\hat{R}_{i}^{-1}B_{i0}^{\top}\psi_{i}(t)$ (31)

where the vectors $\psi_{i}(t)$ satisfy

$\dot{\psi}_{i}(t)=-\hat{Q}_{i}y(t)-\hat{A}_{0}^{\top}\psi_{i}(t),\quad\psi_{i}(T-\tau)=\hat{Q}_{iT}y(T-\tau).$
(32)

Substituting (31) into (29) yields

$\dot{y}(t)=\hat{A}_{0}y(t)-\sum_{i=1}^{m}\hat{S}_{i}\psi_{i}(t),\quad
y(0)=y_{0}$ (33)

where $\hat{S}_{i}=B_{i0}\hat{R}_{i}^{-1}B_{i0}^{\top}$.

The delay-free $m$ OCPs in (29) and (21) have a solution iff the set of
differential equations (32) and (33) with the given boundary conditions above
has a solution. Reference [30] (see Proof of Theorem 7.1) shows that these
differential equations with their boundary conditions convert to a TPBVP for a
two-player differential game. This result can be generalized straightforwardly
to the $m$ OCPs in (29) and (21) as envisaged in Fig. 1. The optimal control
policy $\xi_{i}(t)$ as well as the associated state trajectory $y(t)$ can be
calculated from the TPBVP.

For the set of differential equations in (32) and (33) for every initial state
$y_{0}$, the associated state trajectory $y(t)$ with the optimal control
policies is given by (see the proof of Theorem 7.4 in [30])

$y(t)=\hat{H}(T-\tau-t)\hat{H}^{-1}(T-\tau)y_{0}$ (34)

where $\hat{M}=\begin{bmatrix}-\hat{A}_{0}&\hat{S}_{1}&\cdots&\hat{S}_{m}\\\
\hat{Q}_{1}&\hat{A}_{0}^{\top}&0&0\\\ \vdots&0&\ddots&0\\\
\hat{Q}_{m}&0&0&\hat{A}_{0}^{\top}\end{bmatrix}$ and
$\hat{H}(T-\tau-t)=\begin{bmatrix}I_{2mq\times
2mq}&0&\cdots&0\end{bmatrix}\mathrm{e}^{(T-\tau-t)\hat{M}}\begin{bmatrix}I\\\
\hat{Q}_{1T}\\\ \vdots\\\ \hat{Q}_{mT}\end{bmatrix}$.

Using the Kronecker product property (2.1), we obtain

$\displaystyle\hat{Q}_{iT}$ $\displaystyle=(I+\tau\hat{A}^{\top}\otimes
I_{q\times q})(I_{2\times 2}\otimes W_{iT}\otimes I_{q\times
q}\big{)}(I+\tau\hat{A}\otimes I_{q\times q})$
$\displaystyle=\left((I+\tau\hat{A}^{\top})(I_{2\times 2}\otimes
W_{iT}\big{)}(I+\tau\hat{A})\right)\otimes I_{q\times q}$
$\displaystyle\triangleq Q_{iT}\otimes I_{q\times q},$ (35)
$\displaystyle\hat{Q}_{i}=$
$\displaystyle\left((I+\tau\hat{A}^{\top})(I_{2\times 2}\otimes
W_{i}\big{)}(I+\tau\hat{A})\right)\otimes I_{q\times q}\triangleq Q_{i}\otimes
I_{q\times q},$ (36) $\displaystyle\hat{S}_{i}=\Big{(}\begin{bmatrix}0\\\
\hat{b}_{i}\end{bmatrix}\otimes I_{q\times
q}\Big{)}\Big{(}\frac{1}{\hat{r}_{i}}\otimes I_{q\times
q}\Big{)}\Big{(}\begin{bmatrix}0&\hat{b}_{i}^{\top}\end{bmatrix}\otimes
I_{q\times q}\Big{)}=$ $\displaystyle\Big{(}\begin{bmatrix}0\\\
\hat{b}_{i}\end{bmatrix}\frac{1}{\hat{r}_{i}}\begin{bmatrix}0&\hat{b}_{i}^{\top}\end{bmatrix}\Big{)}\otimes
I_{q\times q}=\begin{bmatrix}0&0\\\
0&\frac{1}{\hat{r}_{i}}\hat{b}_{i}\hat{b}_{i}^{\top}\end{bmatrix}\otimes
I_{q\times q}=$ $\displaystyle\begin{bmatrix}0&0\\\
0&1\end{bmatrix}\otimes\frac{1}{\hat{r}_{i}}\hat{b}_{i}\hat{b}_{i}^{\top}\otimes
I_{q\times q}\triangleq S_{i}\otimes I_{q\times q}.$ (37)

Using (35)-(37), the matrices $\hat{M}$ and $\hat{H}(T-\tau)$ can be expanded
as $\hat{M}=M\otimes I_{q\times q}$ and $\hat{H}(T-\tau-t)=H(T-\tau-t)\otimes
I_{q\times q}$, respectively, where
$M=\begin{bmatrix}-\hat{A}&S_{1}&\cdots&S_{m}\\\ Q_{1}&\hat{A}^{\top}&0&0\\\
\vdots&0&\ddots&0\\\ Q_{m}&0&0&\hat{A}^{\top}\end{bmatrix}$ and
$H(T-\tau-t)=\begin{bmatrix}I_{2m\times
2m}&0&\cdots&0\end{bmatrix}\mathrm{e}^{(T-\tau-t)M}\begin{bmatrix}I\\\
Q_{1T}\\\ \vdots\\\ Q_{mT}\end{bmatrix}$.

Therefore, (34) is rewritten as

$\displaystyle y(t)$ $\displaystyle=\big{(}H(T-\tau-t)\otimes I_{q\times
q}\big{)}\big{(}H^{-1}(T-\tau)\otimes I_{q\times q}\big{)}y_{0}$
$\displaystyle=\big{(}H(T-\tau-t)H^{-1}(T-\tau)\otimes I_{q\times
q}\big{)}y_{0}.$ (38)

From (4), it is obvious that the existence of a unique solution to the TPBVP
depends on whether or not $H(T-\tau)$ is invertible. If invertible, there
exists a unique open-loop Nash equilibrium for the DGG problem. Besides, if
the individual trajectories $y_{i}(t)$ in (4) do not require knowledge of
$y_{j}(0),j\neq i$ from $y_{0}$, one can conclude that the equilibrium and its
associated state trajectories for the DGG problem are distributed. In the
following, we prove the invertibility of $H(T-\tau)$ i.e., the existence of
unique optimal control policies and present individual state trajectories
$y_{i}(t)$ in the form of explicit expressions of time, delay, and the initial
state $y_{i}(0)$. Before, the following definitions were given.

Let $\lambda_{i}$ denote the $i$th eigenvalue of $M$. Define
$f(\lambda_{i})\triangleq f_{1}(\lambda_{i})+f_{2}(\lambda_{i})$ where
$f_{1}(\lambda_{i})\triangleq(\frac{\omega_{i}}{\hat{r}_{i}}+\frac{\mu_{i}}{\hat{r}_{i}}\tau)\frac{1}{\lambda_{i}^{2}}-\lambda_{i}$
and
$f_{2}(\lambda_{i})\triangleq(\frac{\omega_{i}}{\hat{r}_{i}}\tau+\frac{\mu_{i}}{\hat{r}_{i}}(\tau^{2}+1))\frac{1}{\lambda_{i}}$.
Also, $g(\lambda_{i})\triangleq g_{1}(\lambda_{i})+g_{2}(\lambda_{i})$ where
$g_{1}(\lambda_{i})\triangleq\frac{\omega_{i}}{\hat{r}_{i}}\tau\frac{1}{\lambda_{i}^{2}}+1$
and
$g_{2}(\lambda_{i})\triangleq\frac{\omega_{i}}{\hat{r}_{i}}(\tau^{2}+1)\frac{1}{\lambda_{i}}$.
Notice that $f_{1}(-\lambda_{i})=f_{1}(\lambda_{i})$,
$f_{2}(-\lambda_{i})=-f_{2}(\lambda_{i})$,
$g_{1}(-\lambda_{i})=g_{1}(\lambda_{i})$, and
$g_{2}(-\lambda_{i})=-g_{2}(\lambda_{i},\tau)$. Define

$\sigma_{i}(\lambda_{i})=[0,\cdots,\varpi_{i}(\lambda_{i}),\cdots,0]^{\top},\varpi_{i}^{2}(\lambda_{i})=\frac{1}{2}\frac{\lambda_{i}^{3}}{\frac{\mu_{i}}{r_{i}}-\lambda_{i}^{4}}.$
(39)

###### Theorem 1

$H(T-\tau)$ is nonsingular and

$\displaystyle y_{i}(t)$
$\displaystyle=\left(\frac{1}{\Delta_{i}(T-\tau)}\begin{bmatrix}q_{i}(t)&\hat{q}_{i}(t)\\\
\tilde{q}_{i}(t)&\bar{q}_{i}(t)\end{bmatrix}\otimes I_{q\times
q}\right)y_{i}(0)$ (40)

where

$\displaystyle\Delta_{i}(T-\tau)=\beta_{i}(T-\tau)\gamma_{i}(T-\tau)-\alpha_{i}(T-\tau)\eta_{i}(T-\tau),$
$\displaystyle
q_{i}(t)=\beta_{i}(T-\tau-t)\gamma_{i}(T-\tau)-\alpha_{i}(T-\tau-t)\eta_{i}(T-\tau),$
$\displaystyle\hat{q}_{i}(t)=\alpha_{i}(T-\tau-t)\beta_{i}(T-\tau)-\beta_{i}(T-\tau-t)\alpha_{i}(T-\tau),$
$\displaystyle\tilde{q}_{i}(t)=\eta_{i}(T-\tau-t)\gamma_{i}(T-\tau)-\gamma_{i}(T-\tau-t)\eta_{i}(T-\tau),$
$\displaystyle\bar{q}_{i}(t)=\gamma_{i}(T-\tau-t)\beta_{i}(T-\tau)-\eta_{i}(T-\tau-t)\alpha_{i}(T-\tau),$
$\displaystyle\alpha_{i}(\phi)=4\Re(\varpi_{i}^{2}(\lambda_{i})\Big{[}f_{1}(\lambda_{i})\sinh(\phi\lambda_{i})+f_{2}(\lambda_{i})\cosh(\phi\lambda_{i})\Big{]}),$
$\displaystyle\beta_{i}(\phi)=4\Re(\varpi_{i}^{2}(\lambda_{i})\Big{[}g_{1}(\lambda_{i})\sinh(\phi\lambda_{i})+g_{2}(\lambda_{i})\cosh(\phi\lambda_{i})\Big{]}),$
$\displaystyle\gamma_{i}(\phi)=-4\Re(\lambda_{i}\varpi_{i}^{2}(\lambda_{i})\Big{[}f_{1}(\lambda_{i})\sinh(\phi\lambda_{i})+f_{2}(\lambda_{i})\cosh(\phi\lambda_{i})\Big{]}),$
$\displaystyle\eta_{i}(\phi)=-4\Re(\lambda_{i}\varpi_{i}^{2}(\lambda_{i})\Big{[}g_{1}(\lambda_{i})\sinh(\phi\lambda_{i})+g_{2}(\lambda_{i})\cosh(\phi\lambda_{i})\Big{]}).$

######  Proof 4.2.

To extract the explicit expressions for $y_{i}(t)$ from (4), one has to find
the inverse of $H(\phi)$. The invertibility of $H(\phi)$ depends on $M$ and
$\phi$. We begin with the decomposition of the square matrix $M$ into the
product of matrices. Matrix $M$ is defective (i.e., non-diagonalizable, see
Appendix 10). Its Jordan decomposition is

$M=\Phi J\Psi$ (41)

where $\Phi$, $J$, and $\Psi$ are square matrices given in Appendix 7.

The matrix exponential term $\mathrm{e}^{\phi M}$ in $H(\phi)$ is expanded as

$\mathrm{e}^{\phi M}=\Phi\mathrm{e}^{\phi J}\Psi.$ (42)

Using this expansion, as shown in Appendix 12, we have

$\displaystyle
H(\phi)=\sum_{i=1}^{m}\Big{(}K(\lambda_{i})+K(\overline{\lambda}_{i})+K(-\lambda_{i})+K(-\overline{\lambda}_{i})\Big{)}$
(43)

where

$\displaystyle K(\lambda_{i})$
$\displaystyle=\mathrm{e}^{\phi\lambda_{i}}\begin{bmatrix}f(\lambda_{i})&g(\lambda_{i})\\\
-\lambda_{i}f(\lambda_{i})&-\lambda_{i}g(\lambda_{i})\end{bmatrix}\otimes\sigma_{i}(\lambda_{i})\sigma_{i}^{\top}(\lambda_{i})$
(44)

and $f(\lambda_{i})$, $g(\lambda_{i})$, and $\sigma_{i}(\lambda_{i})$ are
defined beforehand.

As $\overline{\lambda}_{i}$ are $\lambda_{i}$ the complex conjugate of each
other, we have

$\displaystyle\Re(K(\overline{\lambda}_{i}))=\Re(K(\lambda_{i})),\quad\Im(K(\overline{\lambda}_{i}))=-\Im(K(\lambda_{i}))$
(45)

where $\Re(.)$ and $\Im(.)$ denote the real and imaginary parts, respectively.
Thereby, (43) is simplified further as

$H(\phi)=2\sum_{i=1}^{m}\big{(}\Re(K(\lambda_{i}))+\Re(K(-\lambda_{i}))\big{)}.$
(46)

Therefore, $H(\phi)$ is a real-valued matrix.

From (44) we can see that $K(\lambda_{i})$ has the structure (47), shown at
the bottom of the page.

$\displaystyle K(\lambda_{i})$
$\displaystyle=\begin{bmatrix}\mathrm{diag}(0,\cdots,\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})f(\lambda_{i}),\cdots,0)&\mathrm{diag}(0,\cdots,\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})g(\lambda_{i}),\cdots,0)\\\
\mathrm{diag}(0,\cdots,-\lambda_{i}\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})f(\lambda_{i}),\cdots,0)&\mathrm{diag}(0,\cdots,-\lambda_{i}\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})g(\lambda_{i}),\cdots,0)\end{bmatrix}$
(47)

On this basis, $H(\phi)$ has the following form

$\displaystyle H(\phi)$
$\displaystyle=\begin{bmatrix}\mathrm{diag}(\cdots,\alpha_{i}(\phi),\cdots)&\mathrm{diag}(\cdots,\beta_{i}(\phi),\cdots)\\\
\mathrm{diag}(\cdots,\gamma_{i}(\phi),\cdots)&\mathrm{diag}(\cdots,\eta_{i}(\phi),\cdots)\end{bmatrix}$
(48)

where $\alpha_{i}(\phi)$, $\beta_{i}(\phi)$, $\gamma_{i}(\phi)$,
$\eta_{i}(\phi)$ are given by (49)-(52), shown at the bottom of the next page.

$\displaystyle\alpha_{i}(\phi)=$
$\displaystyle\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})f(\lambda_{i})+\mathrm{e}^{-\phi\lambda_{i}}\varpi_{i}^{2}(-\lambda_{i})f(-\lambda_{i})+\mathrm{e}^{\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(\overline{\lambda}_{i})f(\overline{\lambda}_{i})+\mathrm{e}^{-\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(-\overline{\lambda}_{i})f(-\overline{\lambda}_{i}),$
(49) $\displaystyle\beta_{i}(\phi)=$
$\displaystyle\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})g(\lambda_{i})+\mathrm{e}^{-\phi\lambda_{i}}\varpi_{i}^{2}(-\lambda_{i})g(-\lambda_{i})+\mathrm{e}^{\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(\overline{\lambda}_{i})g(\overline{\lambda}_{i},\tau)+\mathrm{e}^{-\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(-\overline{\lambda}_{i})g(-\overline{\lambda}_{i}),$
(50) $\displaystyle\gamma_{i}(\phi)=$
$\displaystyle-\lambda_{i}\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})f(\lambda_{i})-\lambda_{i}\mathrm{e}^{-\phi\lambda_{i}}\varpi_{i}^{2}(-\lambda_{i})f(-\lambda_{i})-\overline{\lambda}_{i}\mathrm{e}^{\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(\overline{\lambda}_{i})f(\overline{\lambda}_{i})-\overline{\lambda}_{i}\mathrm{e}^{-\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(-\overline{\lambda}_{i})f(-\overline{\lambda}_{i}),$
(51) $\displaystyle\eta_{i}(\phi)=$
$\displaystyle-\lambda_{i}\mathrm{e}^{\phi\lambda_{i}}\varpi_{i}^{2}(\lambda_{i})g_{i}(\lambda_{i})-\lambda_{i}\mathrm{e}^{-\phi\lambda_{i}}\varpi_{i}^{2}(-\lambda_{i})g_{i}(-\lambda_{i})-\overline{\lambda}_{i}\mathrm{e}^{\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(\overline{\lambda}_{i})g_{i}(\overline{\lambda}_{i})-\overline{\lambda}_{i}\mathrm{e}^{-\phi\overline{\lambda}_{i}}\varpi_{i}^{2}(-\overline{\lambda}_{i})g_{i}(-\overline{\lambda}_{i})$
(52)

To find its inverse, first, we show that $H(\phi)$ is nonsingular. The
eigenvalues of $H(\phi)$ are the roots of its characteristic polynomial

$\displaystyle\rho(\delta)=\det\big{(}H(\phi)-\delta
I\big{)}=\det\Big{(}\mathrm{diag}(\cdots,$
$\displaystyle\big{(}\alpha_{i}(\lambda_{i})-\delta\big{)}\big{(}\eta_{i}(\lambda_{i})-\delta\big{)}-\beta_{i}(\lambda_{i})\gamma_{i}(\lambda_{i}),\cdots)\Big{)}=\prod_{i=1}^{m}\Big{(}\delta^{2}-$
$\displaystyle\big{(}\alpha_{i}(\lambda_{i})+\eta_{i}(\lambda_{i})\big{)}\delta+\alpha_{i}(\lambda_{i})\eta_{i}(\lambda_{i})-\beta_{i}(\lambda_{i})\gamma_{i}(\lambda_{i})\Big{)}.$

None of the roots of the quadratic expression above are zero, meaning that
$H(\phi)$ has no zero eigenvalues and is nonsingular. As its blocks (which are
diagonal matrices) are commuting matrices, the inverse of $H(\phi)$ is
obtained by matrix analyses as (53), shown at the bottom of the next page.

$\displaystyle
H^{-1}(\phi)=\begin{bmatrix}\mathrm{diag}(\cdots,-\frac{\eta_{i}(\phi)}{\beta_{i}(\phi)\gamma_{i}(\phi)-\alpha_{i}(\phi)\eta_{i}(\phi)},\cdots)&\mathrm{diag}(\cdots,\frac{\beta_{i}(\phi)}{\beta_{i}(\phi)\gamma_{i}(\phi)-\alpha_{i}(\phi)\eta_{i}(\phi)},\cdots)\\\
\mathrm{diag}(\cdots,\frac{\gamma_{i}(\phi)}{\beta_{i}(\phi)\gamma_{i}(\phi)-\alpha_{i}(\phi)\eta_{i}(\phi)},\cdots)&\mathrm{diag}(\cdots,-\frac{\alpha_{i}(\phi)}{\beta_{i}(\phi)\gamma_{i}(\phi)-\alpha_{i}(\phi)\eta_{i}(\phi)},\cdots)\end{bmatrix}.$
(53)

Knowing that
$\Re(\mathrm{e}^{\phi\overline{\lambda}_{i}})=\Re(\mathrm{e}^{\phi\lambda_{i}})$,
$\Re(\varpi_{i}^{2}(\overline{\lambda}_{i}))=\Re(\varpi_{i}^{2}(\lambda_{i}))$,
and $\Re(f(\overline{\lambda}_{i}))=\Re(f(\lambda_{i}))$, finally, we can
express $\alpha_{i}(\phi)$, $\beta_{i}(\phi)$, $\gamma_{i}(\phi)$,
$\eta_{i}(\phi)$ in terms of the hyperbolic sine and cosine in (40). For
$\alpha_{i}(\phi)$, it is given in (4.2), shown at the bottom of the next
page, and $\beta_{i}(\phi)$, $\gamma_{i}(\phi)$, and $\eta_{i}(\phi)$ are
obtained similarly.

$\displaystyle\alpha_{i}(\phi)=$ $\displaystyle
2\Re(\mathrm{e}^{\phi\lambda_{i}})\Re(\varpi_{i}^{2}(\lambda_{i}))\Re(f(\lambda_{i}))+2\Re(\mathrm{e}^{-\phi\lambda_{i}})\Re(\varpi_{i}^{2}(-\lambda_{i}))\Re(f(-\lambda_{i}))$
$\displaystyle=$ $\displaystyle
2\Re(\mathrm{e}^{\phi\lambda_{i}})\Re(\varpi_{i}^{2}(\lambda_{i}))\Big{(}\Re(f_{1}(\lambda_{i}))+\Re(f_{2}(\lambda_{i}))\Big{)}-2\Re(\mathrm{e}^{-\phi\lambda_{i}})\Re(\varpi_{i}^{2}(\lambda_{i}))\Big{(}\Re(f_{1}(\lambda_{i}))-\Re(f_{2}(\lambda_{i}))\Big{)}$
$\displaystyle=$ $\displaystyle
4\Re(\varpi_{i}^{2}(\lambda_{i}))\Big{[}\Re(f_{1}(\lambda_{i}))\frac{\Re(\mathrm{e}^{\phi\lambda_{i}})-\Re(\mathrm{e}^{-\phi\lambda_{i}})}{2}+\Re(f_{2}(\lambda_{i}))\frac{\Re(\mathrm{e}^{\phi\lambda_{i}})+\Re(\mathrm{e}^{-\phi\lambda_{i}})}{2}\Big{]}$
$\displaystyle=$ $\displaystyle
4\Re(\varpi_{i}^{2}(\lambda_{i}))\Big{[}\Re(f_{1}(\lambda_{i},\tau))\Re(\sinh(\phi\lambda_{i}))+\Re(f_{2}(\lambda_{i}))\Re(\cosh(\phi\lambda_{i}))\Big{]}$
(54)

Finally, multiplying $H(T-\tau-t)$ from (48) to $H^{-1}(T-\tau)$ from (53)
yields (40) and the proof is concluded.

For $t<\tau$, $\xi_{i}(t)=0$ for all $i\in\\{1,\cdots,m\\}$ and
$z(t)=(I+t\hat{A}_{0})y(0)$. For $t\geq\tau$, from (4),
$z(t+\tau)=(I+\tau\hat{A}_{0})y(t)$. Equivalently,

$\displaystyle z_{i}(t)=\left(\begin{bmatrix}1&t\\\ 0&1\end{bmatrix}\otimes
I_{q\times q}\right)y_{i}(0),\quad t<\tau,$ (55) $\displaystyle
z_{i}(t+\tau)=\left(\begin{bmatrix}1&\tau\\\ 0&1\end{bmatrix}\otimes
I_{q\times q}\right)y_{i}(t),\quad t\geq\tau.$ (56)

The control inputs $\xi_{i}(t)$ for $t\geq\tau$ are obtained from the double-
integrator relations,

$\displaystyle\xi_{i}(t+\tau)=\begin{bmatrix}0&I_{q\times
q}\end{bmatrix}\dot{y}_{i}(t),\quad t\geq\tau.$ (57)

As shown in Fig. 1, the proposed systematic approach to solving the DGG
problem accomplishes this by using the optimal control policies $\xi_{i}(t)$
and their corresponding state trajectories $z_{k}(t)$ for $\forall
i\in\\{1,\cdots,m\\}$ in the $m$-edge system to find the distributed explicit
open-loop Nash equilibrium actions $u_{j}(t)$ and their corresponding
distributed state trajectories $x_{j}(t)$ for $\forall j\in\\{1,\cdots,N\\}$
in the $N$-player DGG.

##  5 Illustrative Example

In this section, we present an illustrative example to verify the distributed
solution given in Theorem 1. Assume a consensus-seeking MAS
$\mathcal{V}=\\{0,1,2,3\\}$ and $\mathcal{E}=\\{(0,1),(1,2),(1,3)\\}$. The
communication graph is not complete, and agents 2 and 3 cannot acquire global
knowledge of the initial state vector $x_{0}$.

The lead agent is determined by $p_{0}(t)=[\cos(t),t]^{\top}$,
$\dot{p}_{0}(t)=[-\sin(t),1]^{\top}$, and
$\dot{u}_{0}(t)=[-\cos(t),0]^{\top}$. The initial states of the other agents
are $p_{1}(0)=[-1,1]^{\top}$, $\dot{p}_{1}(0)=[0,2]^{\top}$,
$p_{2}(0)=[4,4]^{\top}$, $\dot{p}_{2}(0)=[0,0]^{\top}$,
$p_{3}(0)=[6,9]^{\top}$, and $\dot{p}_{3}(0)=[2,0]^{\top}$. In the PIs,
$W_{1T}=W_{2T}=W_{3T}=I_{3\times 3}$, $W_{1}=\mathrm{diag}(1,0.7,0.5)$,
$W_{2}=\mathrm{diag}(0,0.7,0)$, $W_{3}=\mathrm{diag}(0,0,0.5)$,
$r_{1}=r_{2}=r_{3}=1$, and $T=8$.

Following the proposed approach in Fig. 1, the $3$-player DGG converts to the
TPBVP for which the explicit solution is given in (40). The distributed Nash
strategies and their associated distributed state trajectories are
$u_{1}(t)=u_{0}(t)+\xi_{1}(t)$, $u_{2}(t)=\xi_{1}(t)+u_{1}(t)$,
$u_{3}(t)=\xi_{2}(t)+u_{1}(t)$, $x_{1}(t)=x_{0}(t)+z_{1}(t)$,
$x_{2}(t)=x_{1}(t)+z_{2}(t)$, $x_{3}(t)=x_{2}(t)+z_{3}(t)$ where
$x_{i}(t)=[p_{i}^{\top}(t),\dot{p}_{i}^{\top}(t)]^{\top}$. For comparison, we
employ the non-distributed solution (4). Figs. 2-4 show the agents’
trajectories and time histories generated by both the distributed and non-
distributed solutions for delay-free DGG and delayed DGG. It is seen that the
agents $\\{1,2,3\\}$’ positions, velocities, and control inputs reach a
consensus with the lead agent.

From Figs. 2-4, we observe that the behavior of agent $1$ is identical under
both the non-distributed and distributed solutions. Notice from the
communication graph $\mathcal{E}$ that agent $1$ has access to the global
knowledge of all other agents, i.e., $\\{0,2,3\\}$, while they do not. As
seen, agents $2$ and $3$ have different behaviors under both solutions. When
they implement the distributed solution, these two agents converge to the lead
agent’s trajectory on a shorter path than when they implement the non-
distributed solution.

(a) delay-free trajectories

(b) delayed ($\tau=0.5$) trajectories

Figure 2: Agents’ trajectories under the non-distributed (solid gray lines)
and distributed solutions (dashed color lines).

Figure 3: Delay-free time histories under the non-distributed and distributed
solutions.

Figure 4: Delayed ($\tau=0.5$) time histories under the non-distributed and
distributed solutions.

##  6 Conclusion

This paper presents a fully distributed explicit solution for the finite-
planning horizon DGGs defined to govern the cooperative control of double-
integrator multi-agent systems in a $q$-dimensional space ($q\in\\{1,2,3\\}$).
The classical solution is associated with solving either a set of coupled
(asymmetric) Riccati differential equations or, equivalently, a TPBVP and is
non-distributed since it requires global state information. For future work,
the proposed systematic approach could be generalized to solving DGGs with the
feedback information structure and/or MAS with more complex dynamics.

Appendixes, if needed, appear before the acknowledgment.

##  7 Decomposition of M

The matrix $M$ is defective (see Appendix 10). Its Jordan decomposition is
given by (41) where

$J=\begin{bmatrix}I_{m(m-1)\times m(m-1)}&&&\\\ &J_{2}(0)&&\\\ &&\ddots&\\\
&&&J_{1}(\lambda_{i})\end{bmatrix}$ (7.1)

is the Jordan normal form of $M$ with $J_{i}(.)$ defined in (10.1) being the
Jordan block of size $i$ (see Appendix 11). The generalized modal matrices
$\Phi$ and $\Psi$ are constituted from vectors $v_{i}$ and $w_{i}$ as
following

$\Phi=\begin{bmatrix}0_{2m\times 2m(m-1)}&\cdots,v_{i},\cdots\\\
*\end{bmatrix},\Psi=\begin{bmatrix}0_{2m(m-1)\times 2m},*\\\ \vdots\\\
w_{i}\\\ \vdots\end{bmatrix}$

where * are the elements/blocks not to be concerned with (see Appendix 9).

Vectors $v_{i}$ and $w_{i}$, given at the top of the next page,

$\displaystyle v_{i}(\lambda_{i})$
$\displaystyle=\left[\left(\sigma_{i}^{\top}(\lambda_{i}),-\lambda_{i}\sigma_{i}^{\top}(\lambda_{i})\right),(0_{1\times
m},0_{1\times
m}),\cdots,\left((\frac{1}{\lambda_{i}}-\tau)\sigma_{i}^{\top}(\lambda_{i})W_{i},\frac{1}{\lambda_{i}^{2}}-(\tau^{2}+1)\sigma_{i}^{\top}(\lambda_{i})W_{i}\right),\cdots,(0_{1\times
m},0_{1\times m})\right]^{\top}$ (7.2) $\displaystyle w_{i}(\lambda_{i})$
$\displaystyle=\left[(\left(\frac{\mu_{i}}{\hat{r}_{i}}\tau\frac{1}{\lambda_{i}^{2}}+\frac{\mu_{i}}{\hat{r}_{i}}(\tau^{2}+1)\frac{1}{\lambda_{i}}-\lambda_{i}\right)\sigma_{i}^{\top}(\lambda_{i}),\sigma_{i}^{\top}(\lambda_{i})),(0_{m\times
1},0_{m\times
1}),\cdots,(\frac{1}{\lambda_{i}^{2}}\sigma_{i}^{\top}(\lambda_{i})\delta_{i},\frac{1}{\lambda_{i}}\sigma_{i}^{\top}(\lambda_{i})\delta_{i}),\cdots,(0_{m\times
1},0_{m\times 1})\right]$ (7.3)

are the right and left eigenvector associated with the eigenvalue
$\lambda_{i}$ of $M$, respectively (see Appendix 9).

The nonzero eigenvalues of $M$ are
$\lambda_{i},\overline{\lambda}_{i},-\lambda_{i},-\overline{\lambda}_{i},\cdots$
where

$\lambda_{i}=\sqrt{\frac{1}{2}\frac{\mu_{i}}{\hat{r}_{i}}(\tau^{2}+1)+\sqrt{\frac{1}{4}\left(\frac{\mu_{i}}{\hat{r}_{i}}(\tau^{2}+1)\right)^{2}-\frac{\mu_{i}}{\hat{r}_{i}}}}$
(7.4)

for $i=1,\cdots,m$ and $\overline{\lambda}_{i}$ denotes the conjugate of
$\lambda_{i}$ (see Appendix 8).

##  8 Eigenvalues of M

Eigenvalues of $M$ correspond to the roots of its characteristic polynomial

$\rho(\lambda)=\det(M-\lambda I)$ (8.1)

where $\lambda$ denotes the unknown eigenvalues.
$M=\begin{bmatrix}-\hat{A}&S\\\ Q&V\end{bmatrix}$ where
$V=\mathrm{diag}(\hat{A}^{\top},\cdots,\hat{A}^{\top})$,
$S=[S_{1},\cdots,S_{m}]$, and $Q=[Q_{1},\cdots,Q_{m}]^{\top}$.

The determinant of $M-\lambda I$ is [43]

$\displaystyle\det(M-\lambda I)=\det\begin{bmatrix}-\hat{A}-\lambda I&S\\\
Q&V-\lambda I\end{bmatrix}$ $\displaystyle=\det(-\hat{A}-\lambda
I)\det(V-\lambda I-Q(-\hat{A}-\lambda I)^{-1}S)$ (8.2)

where using the Kronecker property (2.1)

$\displaystyle\det(\hat{A}-\lambda I_{2m\times
2m})=\det(-\begin{bmatrix}0&1\\\ 0&0\end{bmatrix}\otimes I_{m\times m}-\lambda
I_{2}\otimes I_{m\times m})$ $\displaystyle=\det(-\begin{bmatrix}\lambda&1\\\
0&\lambda\end{bmatrix}\otimes I_{m\times
m})=\Big{(}\det(-\begin{bmatrix}\lambda&1\\\
0&\lambda\end{bmatrix})\Big{)}^{m}=\lambda^{2m}$ (8.3)

Also,

$\displaystyle(-$ $\displaystyle\hat{A}-\lambda I_{2m\times
2m})^{-1}=(-\begin{bmatrix}\lambda&1\\\ 0&\lambda\end{bmatrix}\otimes
I_{m\times m})^{-1}$ $\displaystyle=-\begin{bmatrix}\lambda&1\\\
0&\lambda\end{bmatrix}^{-1}\otimes I_{m\times
m}=\begin{bmatrix}-\frac{1}{\lambda}&\frac{1}{\lambda^{2}}\\\
0&-\frac{1}{\lambda}\end{bmatrix}\otimes I_{m\times m}.$ (8.4)

From (36)

$\displaystyle Q_{i}$ $\displaystyle=\begin{bmatrix}I&0\\\ \tau
I&I\end{bmatrix}\begin{bmatrix}W_{i}&0\\\
0&W_{i}\end{bmatrix}\begin{bmatrix}I&\tau I\\\
0&I\end{bmatrix}=\begin{bmatrix}1&\tau\\\ \tau&\tau^{2}+1\end{bmatrix}\otimes
W_{i}$ (8.5)

and from (37) $\delta_{i}=\frac{1}{\hat{r}_{i}}\hat{b}_{i}\hat{b}_{i}^{\top}$.

Using the expressions above, we have

$\displaystyle Q_{i}(-\hat{A}-\lambda I)^{-1}S_{i}$
$\displaystyle=\left(\begin{bmatrix}1&\tau\\\
\tau&\tau^{2}+1\end{bmatrix}\otimes W_{i}\right)$
$\displaystyle\left(\begin{bmatrix}-\frac{1}{\lambda}&\frac{1}{\lambda^{2}}\\\
0&-\frac{1}{\lambda}\end{bmatrix}\otimes I_{m\times
m}\right)\left(\begin{bmatrix}0&0\\\ 0&1\end{bmatrix}\otimes\delta_{i}\right)$
$\displaystyle=\begin{bmatrix}0&\frac{1}{\lambda^{2}}-\tau\frac{1}{\lambda}\\\
0&\tau\frac{1}{\lambda^{2}}-(\tau^{2}+1)\frac{1}{\lambda}\end{bmatrix}\otimes
W_{i}\delta_{i}$ (8.6)

where
$W_{i}\delta_{i}=\mathrm{diag}(0,\cdots,\frac{\mu_{i}}{\hat{r}_{i}},\cdots,0)$
and $W_{i}\delta_{j}=0~{}\forall i\neq j$. Similarly,

$\displaystyle Q(-\hat{A}$ $\displaystyle-\lambda I)^{-1}S=$
$\displaystyle\mathrm{diag}(\cdots,\begin{bmatrix}0&\frac{1}{\lambda^{2}}-\tau\frac{1}{\lambda}\\\
0&\tau\frac{1}{\lambda^{2}}-(\tau^{2}+1)\frac{1}{\lambda}\end{bmatrix}\otimes
W_{i}\delta_{i},\cdots).$ (8.7)

On the other hand,

$\displaystyle V-\lambda I$
$\displaystyle=\mathrm{diag}(\cdots,\hat{A}^{\top}-\lambda I_{2m\times
2m},\cdots)$ $\displaystyle=\mathrm{diag}(\cdots,\begin{bmatrix}-\lambda&0\\\
1&-\lambda\end{bmatrix}\otimes I_{m\times m},\cdots).$ (8.8)

Substituting (8), (8) and (8) in (8) yields (8), shown at the top of the next
page.

$\displaystyle\rho(\lambda)=$
$\displaystyle\lambda^{2m}\det\big{(}\mathrm{diag}(\cdots,\begin{bmatrix}-\lambda
I_{m\times m}&(-\frac{1}{\lambda^{2}}+\tau\frac{1}{\lambda})W_{i}\delta_{i}\\\
I_{m\times m}&-\lambda I_{m\times
m}+((\tau^{2}+1)\frac{1}{\lambda}-\tau\frac{1}{\lambda^{2}})W_{i}\delta_{i}\end{bmatrix},\cdots)\big{)}$
$\displaystyle=$
$\displaystyle\lambda^{2m}\prod_{i=1}^{m}\det\Bigg{(}\begin{bmatrix}-\lambda
I_{m\times m}&(-\frac{1}{\lambda^{2}}+\tau\frac{1}{\lambda})W_{i}\delta_{i}\\\
I_{m\times m}&-\lambda I_{m\times
m}+((\tau^{2}+1)\frac{1}{\lambda}-\tau\frac{1}{\lambda^{2}})W_{i}\delta_{i}\end{bmatrix}\Bigg{)}$
$\displaystyle=$ $\displaystyle\lambda^{2m}\prod_{i=1}^{m}\det(-\lambda
I_{m\times
m})\prod_{i=1}^{m}\det\left(\left((\tau^{2}+1)\frac{1}{\lambda}-\frac{1}{\lambda^{3}}\right)W_{i}\delta_{i}-\lambda
I_{m\times m}\right)$ $\displaystyle=$
$\displaystyle\lambda^{m(m+2)}\prod_{i=1}^{m}\det\left(-\lambda
I_{(m-1)\times(m-1)}\right)\prod_{i=1}^{m}\left(\left((\tau^{2}+1)\frac{1}{\lambda}-\frac{1}{\lambda^{3}}\right)\frac{\mu_{i}}{\hat{r}_{i}}-\lambda\right)$
$\displaystyle=$
$\displaystyle\lambda^{m(2m+1)}\prod_{i=1}^{m}\frac{1}{\lambda^{3}}\left(\frac{\mu_{i}}{\hat{r}_{i}}\Big{(}(\tau^{2}+1)\lambda^{2}-1\Big{)}-\lambda^{4}\right)=\lambda^{2m(m-1)}\prod_{i=1}^{m}\left(\lambda^{4}-\frac{\mu_{i}}{\hat{r}_{i}}(\tau^{2}+1)\lambda^{2}+\frac{\mu_{i}}{\hat{r}_{i}}\right)$
(8.9)

The quartic polynomial in (8) has four roots given by (7.4), two opposites in
sign, and two conjugates.

##  9 Eigenvectors of M

### Eigenvectors associated with the nonzero eigenvalues:

Matrix $M$ consists of $2(m+1)\times 2(m+1)$ blocks of size $m\times m$ as
follows

$M=\begin{bmatrix}0_{m\times m}&-I&0&0&\cdots&0&0\\\ 0&0_{m\times
m}&0&\delta_{1}&\cdots&0&\delta_{m}\\\ W_{1}&\tau W_{1}&0&0&\cdots&0&0\\\ \tau
W_{1}&(\tau^{2}+1)W_{1}&I&0&\cdots&0&0\\\ \vdots&\vdots&&&\ddots&&\\\
W_{m}&\tau W_{m}&0&0&\cdots&0&0\\\ \tau
W_{m}&(\tau^{2}+1)W_{m}&0&0&\cdots&I&0\\\ \end{bmatrix}.$ (9.1)

Any nonzero right eigenvector $v_{i}$ satisfies

$(M-\lambda_{i}I)v_{i}=0.$ (9.2)

Substituting (7.2) into the left-hand side of (9.2) yields

$\displaystyle(M-\lambda_{i}I)v_{i}$ $\displaystyle=\Bigg{[}\left(0_{1\times
m},\lambda_{i}^{2}+\frac{\mu_{i}}{\hat{r}_{i}}\left(\frac{1}{\lambda_{i}^{2}}-(\tau^{2}+1)\right)\sigma_{i}^{\top}(\lambda_{i})\right),$
$\displaystyle(0_{1\times m},0_{1\times m}),\cdots,(0_{1\times m},0_{1\times
m})\Bigg{]}^{\top}.$ (9.3)

Here, the expression
$\lambda_{i}^{2}+\frac{\mu_{i}}{\hat{r}_{i}}\left(\frac{1}{\lambda_{i}^{2}}-(\tau^{2}+1)\right)$
appears to be the quartic polynomial in (8) that $\lambda_{i}$ is a zero of
it. Therefore, (7.2) satisfies (9.2).

Any nonzero left eigenvector $w_{i}$ satisfies

$w_{i}(M-\lambda_{i}I)=0.$ (9.4)

Similarly, substituting (7.3) into (9.4), (9.4) is satisfied. By adopting
$\varpi_{i}(\lambda_{i})$ as (39), $v_{i}(\lambda_{i})$ and
$w_{i}(\lambda_{i})$ are normalized so that
$w_{i}(\lambda_{i})v_{i}(\lambda_{i})=1$.

### Eigenvectors associated with the zero eigenvalue:

From Appendix 8 we notice that $M$ has $2m(m-1)$ zero eigenvalues. Thereby,
the definition of the right and left eigenvectors in (9.2) and (9.4),
respectively, reduces to

$Mv_{0}=0,\quad w_{0}M=0.$ (9.5)

Vectors

$\displaystyle v_{0}$ $\displaystyle=[(0_{1\times m},0_{1\times
m}),(0_{1\times m},\vartheta_{1}^{\top}),\cdots,(0_{1\times
m},\vartheta_{m}^{\top})]^{\top}$ $\displaystyle w_{0}$
$\displaystyle=[(0_{1\times m},0_{1\times m}),(\vartheta_{1}^{\top},0_{1\times
m}),\cdots,(\vartheta_{m}^{\top},0_{1\times m})]$

where
$\vartheta_{i}=[\vartheta_{1}^{i},\cdots,[0]_{i},\cdots,\vartheta_{m}^{i}]^{\top}\in\mathbb{R}^{m}$
satisfy (9.5).

From Appendix 11, we notice that the Jordan blocks associated with the zero
eigenvalues are of size $2$ where each corresponds to a chain of generalized
eigenvectors of rank $2$. A generalized eigenvector of rank $1$ is an ordinary
eigenvector.

The generalized right and left eigenvectors of rank $2$ satisfy

$M\hat{v}_{0}=v_{0},\quad\hat{w}_{0}M=w_{0}$ (9.6)

where $\hat{v}_{0}\neq 0$ and $\hat{w}_{0}\neq 0$. It is verified that
$\hat{v}_{0}=w_{0}^{\top}$ and $\hat{w}_{0}=v_{0}^{\top}$. The right and the
right generalized eigenvector, as well as the left and the left generalized
eigenvector associated with the zero eigenvalues, have the form $[0_{1\times
2m},*]^{\top}$ and $[0_{1\times 2m},*]$, respectively.

##  10 M Is Defective

For a defective matrix, its Jordan normal form is a block-diagonal matrix
whose diagonally located blocks are associated with the eigenvalue of that
matrix in the form of

$J_{k}(\lambda_{i})=\begin{bmatrix}\lambda_{i}&1&0&&\\\ 0&\lambda_{i}&1&0\\\
&&\ddots&\ddots&\\\ &0&&\lambda_{i}&1\\\
&&&0&\lambda_{i}\end{bmatrix}_{k\times k}.$ (10.1)

To determine whether $M$ is defective or not, we inspect the geometric
multiplicity associated with the zero eigenvalues. The number of linearly
independent eigenvectors associated with an eigenvalue is its geometric
multiplicity. A defective matrix has an eigenvalue with its geometric
multiplicity less than its algebraic multiplicity [44].

Let $\mathcal{N}(M-\lambda I)$ be the dimension of the null space of
$M-\lambda I$. The geometric multiplicity of $\lambda$ is
$\mathrm{dim}\mathcal{N}(M-\lambda I)$ which is the number of free variables
in

$(M-\lambda I)v=0$ (10.2)

where $v$ is a nonzero vector.

Consider (9.1). Let
$v=[(\varrho_{0}^{\top},\vartheta_{0}^{\top}),(\varrho_{1}^{\top},\vartheta_{1}^{\top}),\cdots,(\varrho_{m}^{\top},\vartheta_{m}^{\top})]^{\top}$,
$\varrho_{i}=[\varrho_{1}^{i},\cdots,\varrho_{m}^{i}]^{\top}\in\mathbb{R}^{m\times
1}$,
$\vartheta_{i}=[\vartheta_{1}^{i},\cdots,\vartheta_{m}^{i}]^{\top}\in\mathbb{R}^{m\times
1}$. For $\lambda=0$, (10.2) reduces to

$\vartheta_{0}=\varrho_{i}=0,\quad
i=0,\cdots,m,\quad\sum_{i=1}^{m}\delta_{i}\vartheta_{i}=\begin{bmatrix}r_{1}^{-1}\varrho_{1}^{1}\\\
\vdots\\\ r_{m}^{-1}\varrho_{m}^{m}\end{bmatrix}=0.$ (10.3)

Clearly, $\vartheta_{i}^{i}$ in $\vartheta_{i}$ must be zero, thus, there are
$m-1$ free variables left in $\vartheta_{i}$. Consequently, the total number
of free variables is $m(m-1)$ and

$\mathrm{dim}\mathcal{N}(M)=m(m-1).$

As a result of the geometric multiplicity of the zero eigenvalues being less
than their algebraic multiplicity, $M$ is defective.

##  11 Jordan Blocks of M

By definition, the size of the Jordan block associated with $\lambda_{i}$,
i.e., $\zeta$, is the first integer for which
$\mathrm{dim}\mathcal{N}(M-\lambda_{i}I)^{\zeta}$ stabilizes. This is equal to
the number of free variables in

$(M-\lambda I)^{j}v=0.$ (11.1)

For the eigenvectors associated with the zero eigenvalues, we have
$\mathrm{dim}\mathcal{N}(M)^{1}=m(m-1)$. It can be verified that vector $v$
(as defined in Appendix 10) with $\varrho_{0}=\vartheta_{0}=0$ and
$\varrho_{i}=\vartheta_{i}=[\varrho_{1}^{i},\cdots,[0]_{i},\cdots,\varrho_{m}^{i}]^{\top}$,
$i=1,\cdots,m$ satisfies (11.1) for $j=2$ and $j=3$. Therefore,
$\mathrm{dim}\mathcal{N}(M)^{2}=\mathrm{dim}\mathcal{N}(M)^{3}=2m(m-1)$. It is
seen that $\zeta=2$ and thus, there are $m(m-1)$ Jordan blocks of size $2$.

For the eigenvectors associated with nonzero eigenvalues, the number of free
variables in the equation (11.1) for $j=1$ and $j=2$ are both equal to $m$.
Therefore, $\zeta=1$ and the corresponding Jordan block is of size $1$.

##  12 Decomposition of H

Substituting $\mathrm{e}^{\phi M}$ from (42) shows that

$H(\phi)=\begin{bmatrix}I_{2m\times
2m}&0&\cdots&0\end{bmatrix}\Phi\mathrm{e}^{\phi J}\Psi\begin{bmatrix}I\\\
Q_{1T}\\\ \vdots\\\ Q_{mT}\end{bmatrix}.$ (12.1)

This expression is broken apart into the expressions (12)-(12.4).

$\displaystyle\begin{bmatrix}I_{2m\times 2m}&0&\cdots&0\end{bmatrix}\Phi=$
$\displaystyle\qquad\qquad\begin{bmatrix}0_{2m\times
2m(m-1)}\cdots,\begin{pmatrix}1\\\
-\lambda_{i}\end{pmatrix}\otimes\sigma_{i}(\lambda_{i}),\cdots\end{bmatrix}$
(12.2) $\mathrm{e}^{\phi J}=\begin{bmatrix}I_{m(m-1)\times
m(m-1)}\otimes\mathrm{e}^{\phi J_{2}(0)}&0&\cdots&0\\\ 0&\ddots&0&\vdots\\\
\vdots&\cdots&\mathrm{e}^{\phi\lambda_{i}}&0\\\
0&\cdots&0&\ddots\end{bmatrix}$ (12.3)

$\Psi\begin{bmatrix}I\\\ Q_{1T}\\\ \vdots\\\
Q_{mT}\end{bmatrix}=\begin{bmatrix}*\\\ \vdots\\\
\left((\frac{\omega_{i}}{\hat{r}_{i}}+\frac{\mu_{i}}{\hat{r}_{i}}\tau)\frac{1}{\lambda_{i}^{2}}+(\frac{\omega_{i}}{\hat{r}_{i}}\tau+\frac{\mu_{i}}{\hat{r}_{i}}(\tau^{2}+1))\frac{1}{\lambda_{i}}-\lambda_{i}\right)\sigma_{i}^{\top}(\lambda_{i})\qquad\left(\frac{\omega_{i}}{\hat{r}_{i}}\tau\frac{1}{\lambda_{i}^{2}}+\frac{\omega_{i}}{\hat{r}_{i}}(\tau^{2}+1)\frac{1}{\lambda_{i}}+1\right)\sigma_{i}^{\top}(\lambda_{i})\\\
\vdots\end{bmatrix}$ (12.4)

Finally, by multiplying the right-hand sides of (12)-(12.4) we get (43).

## References

  * [1] M. A. Nowak, “Five rules for the evolution of cooperation,” Science,, vol. 314(5805), pp. 1560–1563, 2006.
  * [2] Z. Li, W. Ren, X. Liu, and L. Xie, “Distributed consensus of linear multi-agent systems with adaptive dynamic protocols,” Automatica, vol. 49, no. 7, pp. 1986–1995, 2013.
  * [3] K. H. Movric and F. L.Lewis, “Cooperative optimal control for multi-agent systems on directed graph topologies,” IEEE Trans. Autom. Control, vol. 59, no. 3, pp. 769–774, 2014.
  * [4] R. Olfati-Saber, “Flocking for multi-agent dynamic systems: algorithms and theory,” IEEE Trans. Autom. Control, vol. 51, no. 3, pp. 401–420, 2006.
  * [5] Z. Lin, L. Wang, Z. Han, and M. Fu, “Distributed formation control of multi-agent systems using complex Laplacian,” IEEE Trans. Autom. Control, vol. 59, no. 7, pp. 1765–1777, 2014.
  * [6] J. Wang, J. Qiao, G. Wen, Z. Duan, and T. Huang, “Rendezvous of heterogeneous multiagent systems with nonuniform time-varying information delays: an adaptive approach,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 51, no. 8, pp. 4848–4857, 2021.
  * [7] W. Lin, C. Li, Z. Qu, and M. A. Simaanm, “Distributed formation control with open-loop Nash strategy,” Automatica, vol. 106, pp. 266–273, 2019.
  * [8] L. Song, N. Wan, A. Gahlawat, C. Tao, N. Hovakimyan and E. A. Theodorou, “Generalization of safe optimal control actions on networked multiagent systems,” IEEE Trans. Control Netw. Syst., vol. 10, no. 1, pp. 491–502, 2023.
  * [9] N. Wan, A. Gahlawat, N. Hovakimyan, E. A. Theodorou and P. G. Voulgaris, “Cooperative path integral control for stochastic multi-agent systems,” in Proc. American Control Conf. (ACC), New Orleans, LA, USA, 2021, pp. 1262–1267.
  * [10] K. G. Vamvoudakis and F. L. Lewis, “Policy iteration algorithm for distributed networks and graphical games,” in Proc. 50th IEEE Conf. Decis. Control and European Control Conf. (CDC-ECC), Orlando, FL, USA, 2011, pp. 128–135.
  * [11] M. U. B. Niazi, A. B. Özgüler, and A. Yildiz, “Consensus as a Nash Equilibrium of a Dynamic Game,” in Proc. 12th Int. Conf. Signal-Image Technol. Internet-Based Sys. (SITIS), Naples, Italy, 2016, pp. 365–372.
  * [12] L. Zhou, Y. Zheng, Q. Zhao, F. Xiao and Y. Zhang, “Game-based coordination control of multi-agent systems,” Syst. Control Lett., vol. 169, pp. 105376, 2022.
  * [13] A. B. Özgüler and A. Yildiz, “Foraging swarms as Nash equilibria of dynamic games,” IEEE Trans. Cybern., vol. 44, no. 6, pp. 979–987, 2014.
  * [14] Y. Qian, M. Liu, Y. Wan, F. L. Lewis, and A. Davoudi, “Distributed adaptive Nash equilibrium solution for differential graphical games,” IEEE Trans. Cybern., pp. 1–13, 2021.
  * [15] M. Mazouchi, M. B. Naghibi-Sistani, and S. K. H. Sani, “A novel distributed optimal adaptive control algorithm for nonlinear multi-agent differential graphical games,” IEEE/CAA J. Automat. Sin., vol. 5, no. 1, pp. 331–341, 2018.
  * [16] J. Li, S. Chen, C. Li, and F. Wang, “Distributed game strategy for formation flying of multiple spacecraft with disturbance rejection,” IEEE Trans. Aerosp. Electron. Syst., vol. 57, pp. 119–128, 2021.
  * [17] V. G. Lopez, F. L. Lewis, Y. Wan, E. N. Sanchez, and L. Fan, “Solutions for multiagent pursuit-evasion games on communication graphs: finite-time capture and asymptotic behaviors,” IEEE Trans. Autom. Control, vol. 65, no. 5, pp. 1911–1923, 2020.
  * [18] W. Lin, Z. Qu, and M. A. Simaan, “Nash strategies for pursuit-evasion differential games involving limited observations,” IEEE Trans. Aerosp. Electron. Syst., vol. 51, no. 2, pp. 1347–1356, 2015.
  * [19] D. Cappello and T. Mylvaganam, “Distributed differential games for control of multi-agent systems,” IEEE Trans. Control Netw. Syst., vol. 9, no. 2, pp. 635–646, 2022.
  * [20] T. Mylvaganam, M. Sassano, and A. Astolfi, “A differential game approach to multi-agent collision avoidance,” IEEE Trans. Autom. Control, vol. 62, no. 8, pp. 4229–4235, 2017.
  * [21] J. Inga, E. Bischoff, T. L. Molloy, M. Flad and S. Hohmann, “Solution sets for inverse non-cooperative linear-quadratic differential games,” IEEE Control Syst. Lett., vol. 3, no. 4, pp. 871–876, 2019.
  * [22] Y. Shi, Y. Hua, J. Yu, X. Dong and Z. Ren, “Multi-agent differential game based cooperative synchronization control using a data-driven method,” Front. Inform. Technol. Electron. Eng., vol. 23, pp. 1043–1056, 2022.
  * [23] H. Li and Q. Wei, “Optimal synchronization control for multi-agent systems with input saturation: a nonzero-sum game,” Front. Inform. Technol. Electron. Eng., vol. 23, pp. 1010–1019, 2022.
  * [24] Y. Ren, Q. Wang and Z. Duan, “Optimal leader-following consensus control of multi-agent systems: a neural network based graphical game approach,” IEEE Trans. Netw. Sci. Eng., vol. 9, no. 5, pp. 3590–3601, 2022.
  * [25] J. Li, J. Ding, T. Chai and F. L. Lewis, “Nonzero-sum game reinforcement learning for performance optimization in large-scale industrial processes,” IEEE Trans. Cybern., vol. 50, no. 9, pp. 4132–4145, 2020.
  * [26] Y. Xu, H. Yang, B. Jiang and M. M. Polycarpou, “Distributed optimal fault estimation and fault-tolerant control for interconnected systems: a Stackelberg differential graphical game approach,” IEEE Trans. Autom. Control, vol. 67, no. 2, pp. 926–933, 2022.
  * [27] W. Lin, “Distributed UAV formation control using differential game approach,” Aerosp. Sci. Technol., vol. 35, pp. 54–62, 2014.
  * [28] D. Gu, “A differential game approach to formation control,” IEEE Trans. Control Syst., vol. 16, no. 1, pp. 85–93, 2008.
  * [29] T. Başar, and G. J. Olsder, Dynamic Noncooperative Game Theory, Society for Industrial and Applied Mathematics (SIAM), 2nd Edition, 1998.
  * [30] J.C. Engwerda, LQ Dynamic Optimization and Differential Games, John Wiley & Sons, 2005.
  * [31] M. Liu, Y. Wan, V. G. Lopez, F. L. Lewis, G. A. Hewer and K. Estabridis, “Differential graphical game with distributed global Nash solution,” IEEE Trans. Control Netw. Syst., vol. 8, no. 3, pp. 1371–1382, 2021.
  * [32] H. B. Jond and A. Yıldız, “Connected and automated vehicle platoon formation control via differential games,” IET Intell. Transp. Syst., pp. 1–15, 2022.
  * [33] A. Yildiz and A. B. Özgüler, “Opinion dynamics of stubborn agents under the presence of a troll as differential game,” Turk. J. Electr. Eng. Comput. Sci., vol. 29, no. 7, pp. 3259–3269, 2021.
  * [34] A. Yildiz and A. B. Özgüler, “Partially informed agents can form a swarm in a Nash equilibrium,” IEEE Trans. Autom. Control, vol. 60, no. 11, pp. 3089–3094, 2015.
  * [35] X. Ai, “Distributed Nash equilibrium seeking for integrated game and control of multi-agent systems with input delay,” Nonlinear Dyn., vol. 106, pp. 583–603, 2021.
  * [36] A.J. Laub, Matrix Analysis for Scientists and Engineers, Society for Industrial and Applied Mathematics (SIAM), 2005.
  * [37] J. Wang and M. Xin, “Integrated optimal formation control of multiple unmanned aerial vehicles,” IEEE Trans. Control Syst. Technol., vol. 21, no. 5, pp. 1731–1744, 2013.
  * [38] R. Wei and R.W. Beard, “Consensus seeking in multiagent systems under dynamically changing interaction topologies,” IEEE Trans. Autom. Control, vol. 50, no. 5, pp. 655–661, 2005.
  * [39] H. B. Jond and V. V. Nabiyev, “On the finite horizon Nash equilibrium solution in the differential game approach to formation control,” J. Syst. Eng. Electr., vol. 30, no. 6, pp. 1233–1242, 2019.
  * [40] E. van Damme, Refinements of the Nash Equilibrium Concept, Lecture Notes in Economics and Mathematical Systems, Springer Berlin Heidelberg, 2012.
  * [41] J. C. Engwerda, “On the open-loop Nash equilibrium in LQ-games,” J. Econ. Dyn. Control, vol. 22, no. 5, pp. 729–762, 1998.
  * [42] Y. Zhou and Z. Wang, “Optimal feedback control for linear systems with input delays revisited,” J. Optim. Theor. Appl., vol. 163, pp. 989–1017, 2014.
  * [43] J. R. Silvester, “Determinants of block matrices,” Math. Gaz., vol. 84, pp. 460–467, 2000.
  * [44] T.S. Shores, Applied Linear Algebra and Matrix Analysis, New York, NY, USA: Springer, 2007.