# Optimal Symmetric Strategies in Multi-Agent Systems with Decentralized
Information

Sagar Sudhakara and Ashutosh Nayyar S. Sudhakara and A. Nayyar are with the
Department of Electrical & Computer Engineering, University of Southern
California, Los Angeles, CA 90089 (e-mail<EMAIL_ADDRESS>ashutosn@usc.edu).

###### Abstract

We consider a cooperative multi-agent system consisting of a team of agents
with decentralized information. Our focus is on the design of _symmetric_
(i.e. identical) strategies for the agents in order to optimize a finite
horizon team objective. We start with a general information structure and then
consider some special cases. The constraint of using symmetric strategies
introduces new features and complications in the team problem. For example, we
show in a simple example that randomized symmetric strategies may outperform
deterministic symmetric strategies. We also discuss why some of the known
approaches for reducing agents’ private information in teams may not work
under the constraint of symmetric strategies. We then adopt the common
information approach for our problem and modify it to accommodate the use of
symmetric strategies. This results in a common information based dynamic
program where each step involves minimization over a single function from the
space of an agent’s private information to the space of probability
distributions over actions. We present specialized models where private
information can be reduced using simple dynamic program based arguments.

##  I Introduction

The problem of sequential decision-making by a team of collaborative agents
operating in an uncertain environment has received significant attention in
the recent control (e.g. [1, 2, 3, 4, 5]) and artificial intelligence (e.g.
[6, 7, 8, 9, 10]) literature. The goal in such problems is to design
decision/control strategies for the multiple agents in order to optimize a
performance metric for the team.

In some cooperative multi-agent (or team) problems, the agents are essentially
identical and interchangeable. For example, consider a team of autonomous
agents operating in an environment. The agents may have identical sensors that
they use to observe their local surroundings and they may have identical
action spaces. For teams with such identical agents, it may be convenient for
the designer to design identical decision/control strategies for the agents.
This would be particularly helpful if the number of agents is large –instead
of designing $n$ different strategies for $n$ agents in a team, the designer
needs to design just one strategy for all agents. Identical strategies may
also be necessary for other practical and regulatory reasons. For example, a
self-driving car company would be expected to have the same control algorithm
on all its cars. Another reason for using identical strategies arises in
situations where agents don’t have any individualized identities. This can
happen in settings where the population of the agents is not fixed and agents
are unaware of the total number of agents currently present or their own index
in the population. An example of such a situation for a multi-access
communication problem is described in [11]. When an agent doesn’t know its own
index (“Am I agent 1 or agent 2?”), it makes sense to use symmetric (i.e.
identical) strategies for all agents irrespective of their index. In this
paper, we will focus on the design of identical strategies for a team of
cooperative agents. We will refer to such strategies as _symmetric
strategies_.

Our focus is on designing symmetric strategies to optimize a finite horizon
team objective. We start with a general information structure and then
consider some special cases. The constraint of using symmetric strategies
introduces new features and complications in the team problem. For example,
when agents in a team are free to use individualized strategies, it is well-
known that agents can be restricted to deterministic strategies without loss
of optimality [12]. However, we show in a simple example that randomized
strategies may be helpful when the agents are constrained to use symmetric
strategies.

We adopt the common information approach [2] for our problem and modify it to
accommodate the use of symmetric strategies. This results in a common
information based dynamic program where each step involves minimization over a
single function from the space of an agent’s private information to the space
of probability distributions over actions. The complexity of this dynamic
program depends in large part on the size of the private information space. We
discuss some known approaches for reducing agents’ private information and why
they may _not_ work under the constraint of symmetric strategies. We present
two specialized models where private information can be reduced using simple
dynamic program based arguments.

Notation: Random variables are denoted by upper case letters (e.g. $X$), their
realization with lower case letters (e.g. $x$), and their space of
realizations by script letters (e.g. $\mathcal{X}$). Subscripts denote time
and superscripts denote agent index; e.g., $X^{i}_{t}$ denotes the state of
agent $i$ at time $t$. The short hand notation $X^{i}_{1:t}$ denotes the
collection $(X^{i}_{1},X^{i}_{2},...,X^{i}_{t})$. $\triangle(\mathcal{X})$
denotes the probability simplex for the space $\mathcal{X}$.
$\operatorname{{\mathds{P}}}(A)$ denotes the probability of an event $A$.
$\operatorname{\mathbb{E}}[X]$ denotes the expectation of a random variable
$X$. $\mathds{1}_{A}$ denotes the indicator function of event $A$. For
simplicity of notation, we use
$\operatorname{{\mathds{P}}}(x_{1:t},u_{1:t-1})$ to denote
$\operatorname{{\mathds{P}}}(X_{1:t}=x_{1:t},U_{1:t-1}=u_{1:t-1})$ and a
similar notation for conditional probability. For a strategy pair
$(g^{1},g^{2})$, we use $\operatorname{{\mathds{P}}}^{(g^{1},g^{2})}(\cdot)$
(resp. $\operatorname{\mathbb{E}}^{(g^{1},g^{2})}[\cdot]$) to indicate that
the probability (resp. expectation) depends on the choice of the strategy
pair. We use $-i$ to denote agent/agents other than agent $i$. $U\sim\lambda$
indicates that $U$ is randomly distributed according to the distribution
$\lambda$.

##  II Problem Formulation

Consider a discrete-time system with two agents. The system state consists of
three components - a shared state and two local states, one for each agent.
$X^{i}_{t}\in\mathcal{X}$ denotes the local state of agent $i$, $i=1,2$, at
time $t$ and $X^{0}_{t}\in\mathcal{X}^{0}$ denotes the shared state at time
$t$. $X_{t}$ denotes the triplet $(X^{0}_{t},X^{1}_{t},X^{2}_{t})$. Let
$U^{i}_{t}\in\mathcal{U}$ denote the control action of agent $i$ at time $t$.
$U_{t}$ denotes the pair $(U^{1}_{t},U^{2}_{t})$. The dynamics of the shared
and local states are as follows:

$X^{0}_{t+1}=f^{0}_{t}(X^{0}_{t},U_{t},W^{0}_{t}),$ (1)
$X^{i}_{t+1}=f_{t}(X^{i}_{t},X^{0}_{t},U_{t},W^{i}_{t}),\leavevmode\nobreak\
\leavevmode\nobreak\ i=1,2,$ (2)

where $W^{0}_{t}\in\mathcal{W}^{0}$ and $W^{i}_{t}\in\mathcal{W}$ are random
disturbances with $W^{0}_{t}$ having the probability distribution $p^{0}_{W}$
and $W^{i}_{t},i=1,2,$ having the probability distribution $p_{W}$. We use
$W_{t}$ to denote the triplet $(W^{0}_{t},W^{1}_{t},W^{2}_{t})$. Note that the
next local state of agent $i$ depends on its own current local state, the
shared state and the control actions of both the agents. Also note that the
function $f_{t}$ in (2) is the same for both agents. The initial states
$X^{0}_{1},X^{1}_{1},X^{2}_{1}$ are independent random variables with
$X^{0}_{1}$ having the probability distribution $\alpha^{0}$ and
$X^{i}_{1},i=1,2,$ having the probability distribution $\alpha$. The initial
states $X^{0}_{1},X^{1}_{1},X^{2}_{1}$ and the disturbances
$W^{0}_{t},W^{i}_{t},t\geq 1$, $i=1,2$, are independent discrete random
variables. These will be referred to as the primitive random variables of the
system.

###  II-A Information structure and strategies

The information available to agent $i$ , $i=1,2,$ at time $t$ consists of two
parts:

  1. 1.

Common information $C_{t}$ \- This information is available to both
agents111$C_{t}$ does not have to be the _entirety_ of information that is
available to both agents; it simply cannot include anything that is not
available to both agents.. $C_{t}$ takes values in the set $\mathcal{C}_{t}$.

  2. 2.

Private information $P^{i}_{t}$ \- Any information available to agent $i$ at
time $t$ that is not included in $C_{t}$ is included in $P^{i}_{t}$.
$P^{i}_{t}$ takes values in $\mathcal{P}_{t}$. (Note that the space of private
information is the same for both agents.) We use $P_{t}$ to denote the pair
$(P^{1}_{t},P^{2}_{t})$.

$C_{t}$ should be viewed as an ordered list (or row vector) of some of the
system variables that are known to both agents. Similarly, $P^{i}_{t}$ should
be viewed as an ordered list (or row vector).

We assume that $C_{t}$ is non-decreasing with time, i.e., any variable
included in $C_{t}$ is also included in $C_{t+1}$. Let $Z_{t+1}$ be the
increment in common information from time $t$ to $t+1$. We assume the
following dynamics for $Z_{t+1}$ and $P^{i}_{t+1}$ ($i=1,2$):

$\displaystyle
Z_{t+1}=\zeta_{t}(X_{t},P_{t},U_{t},W_{t});P^{i}_{t+1}=\xi^{i}_{t}(X_{t},P_{t},U_{t},W_{t}),$

Agent $i$ uses its information at time $t$ to select a probability
distribution $\delta{U}^{i}_{t}$ on the action space $\mathcal{U}$. We will
refer to $\delta{U}^{i}_{t}$ as agent $i$’s _behavioral action_ at time $t$.
The action $U^{i}_{t}$ is then randomly generated according to the chosen
distribution, i.e., $U^{i}_{t}\sim\delta{U}^{i}_{t}$. Thus, we can write

$\delta{U}^{i}_{t}=g^{i}_{t}(P^{i}_{t},C_{t}),$ (3)

where $g^{i}_{t}$ is a mapping from $\mathcal{P}_{t}\times\mathcal{C}_{t}$ to
$\Delta(\mathcal{U})$. The function $g^{i}_{t}$ is referred to as the control
strategy of agent $i$ at time $t$. The collection of functions
$g^{i}:=(g^{i}_{1},\ldots,g^{i}_{T})$ is referred to as the control strategy
of agent $i$. Let $\mathcal{G}$ denote the set of all possible strategies for
agent $i$. (Note that the set of all possible strategies is the same for the
two agents since the private information space, the common information space
and the action space are the same for the two agents.)

We use $(g^{1},g^{2})$ to denote the pair of strategies being used by agent 1
and agent 2 respectively. We are interested in the finite horizon total
expected cost incurred by the system which is defined as:

$J(g^{1},g^{2}):=\operatorname{\mathbb{E}}^{(g^{1},g^{2})}\left[\sum_{t=1}^{T}k_{t}({X}_{t},{U}_{t})\right],$
(4)

where $k_{t}$ is the cost function at time $t$. Our focus will be on the case
of _symmetric strategies_ , i.e., the case where both agents use the same
control strategy. When referring to symmetric strategies, we will drop the
superscript $i$ in $g^{i}$ and denote a symmetric strategy pair by $(g,g)$.

Symmetric strategy optimization problem (Problem P1): Our objective is to find
a symmetric strategy pair that achieves the minimum total expected cost among
all symmetric strategy pairs. That is, we are looking for a strategy
$g\in\mathcal{G}$ such that

$J(g,g)\leq J(h,h),\leavevmode\nobreak\ \leavevmode\nobreak\ \forall
h\in\mathcal{G}.$ (5)

We assume that the randomization at each agent is done independently over time
and independently of the other agent [13].

If the private information space, the common information space and the action
space are finite, then it can be shown that the strategy space $\mathcal{G}$
is a compact space and that $J(g,g)$ is a continuous function of
$g\in\mathcal{G}$. Thus, an optimal $g$ satisfying (5) exists.

We have formulated the problem with two agents for simplicity. The number of
agents can in fact be any positive integer $n$ or even a deterministic time-
varying sequence $n_{t}$. Our results extend to these cases with only
notational modifications.

Note that we are not claiming that use of symmetric strategies is always
optimal – it is not. We are simply focusing on the design of symmetric
strategies for reasons mentioned in the introduction.

###  II-B Some specific information structures

We will be particularly interested in the special cases of Problem P1
described below. Each case corresponds to a different information structure.
In each case, the shared state history until time $t$, $X^{0}_{1:t}$, and the
action history until $t-1$, $U_{1:t-1}$ are part of common information
$C_{t}$.

1\. _One-step delayed sharing information structure:_ In this case, each agent
knows its own local state history until time $t$ and the local state history
of the other agent until time $t-1$. Thus, the common and private information
available to agent $i$ at time $t$ is given by

$\displaystyle
C_{t}=(X^{0}_{1:t},U_{1:t-1},X^{1,2}_{1:t-1});\leavevmode\nobreak\
P^{i}_{t}=X^{i}_{t}.$ (6)

We refer to the instance of Problem P1 with this information structure as
Problem P1a.

2\. _Full local history information structure:_ In this case, each agent knows
its own local state history until time $t$ but does not observe the local
states of the other agent. Thus, the common and private information available
to agent $i$ at time $t$ is given by

$\displaystyle C_{t}=(X^{0}_{1:t},U_{1:t-1});\leavevmode\nobreak\
P^{i}_{t}=X^{i}_{1:t}.$ (7)

This information structure corresponds to the control sharing information
structure of [3]. We refer to the instance of Problem P1 with this information
structure as Problem P1b.

3\. _Reduced local history information structure:_ In this case, each agent
knows its own _current_ local state but does not recall its past local states
and does not observe the local states of the other agent. Thus, the common and
private information available to agent $i$ at time $t$ is given by

$\displaystyle C_{t}=(X^{0}_{1:t},U_{1:t-1});\leavevmode\nobreak\
P^{i}_{t}=X^{i}_{t}.$ (8)

We refer to the instance of Problem P1 with this information structure as
Problem P1c.

Another special case of Problem P1 that might be of interest is the following:
Consider a situation where the state dynamics are governed not by the vector
of agents’ actions but only by an aggregate effect of agents’ actions. Let
$A_{t}=a(U^{1}_{t},U^{2}_{t})$ denote the aggregate action. We refer to
$a(\cdot,\cdot)$ as the aggregation function. Some examples of $a$ could be
the sum or the maximum function. The state dynamics are as described in
equations (1) and (2) except with $U_{t}$ replaced by $A_{t}$. The agents only
observe the aggregate actions taken in the past but not the individual
actions. The common and private information are given as:

$\displaystyle C_{t}=(X^{0}_{1:t},A_{1:t-1});\leavevmode\nobreak\
P^{i}_{t}=X^{i}_{t}.$ (9)

We addressed this case in Appendix F,

###  II-C Why are randomized strategies needed?

In team problems, it is well-known that one can restrict agents to
deterministic strategies without loss of optimality [12]. However, since the
agents are restricted to use symmetric strategies in our setup, randomization
can help. This can be illustrated by the following simple example.

_Example 1:_ Let $T=1$ and let $(X^{0}_{1},X^{1}_{1},X^{2}_{1})=(0,0,0)$ with
probability $1$. The action space is $\mathcal{U}=\\{0,1\\}$. The information
structure is that of Problem P1c described in II-B. The cost at $t=1$ is given
by, $k_{1}({X}_{1},{U}_{1})=\mathds{1}_{\\{U^{1}_{1}=U^{2}_{1}\\}}$.

Note that the cost function penalizes the agents for taking the same action.
In this case, each agent has only two _deterministic_ strategies – taking
action $0$ or taking action $1$ at time $1$. If both agents use the same
deterministic strategy, then, clearly, $U^{1}_{1}=U^{2}_{1}$ and hence the
expected cost incurred is $1$.

Consider now the following randomized strategy for each agent: $U^{i}_{1}=1$
with probability $p$ and $U^{i}_{1}=0$ with probability $(1-p)$. When the two
agents use this randomized strategy, the expected cost is $p^{2}+(1-p)^{2}$.
With $p=0.5$, this cost is $0.5$ which is less than the expected cost achieved
by any deterministic symmetric strategy pair. Thus, when agents are restricted
to use the same strategy, they can benefit from randomization.

##  III Common information approach

We adopt the common information approach [2] for Problem P1. This approach
formulates a new decision-making problem from the perspective of a coordinator
that knows the common information. At each time, the coordinator selects
prescriptions that map each agent’s private information to its action. The
behavioral action of each agent in this problem is simply the prescription
evaluated at the current realization of its private information. Since Problem
P1 requires symmetric strategies for the two agents, we will require the
coordinator to select _identical prescriptions for the two agents_. To make
things precise, let $\mathcal{B}_{t}$ denote the space of all functions from
$\mathcal{P}_{t}$ to $\Delta(\mathcal{U})$. Let $\Gamma_{t}\in\mathcal{B}_{t}$
denote the prescription selected by the coordinator at time $t$. Then, the
behavioral action of agent $i$, $i=1,2,$ is given by: $\delta
U^{i}_{t}=\Gamma_{t}(P^{i}_{t})$.

As in Problem P1, agent $i$’s action $U^{i}_{t}$ is generated according to the
distribution $\delta U^{i}_{t}$ using independent randomization. The
coordinator selects its prescription at time $t$ based on the common
information at time $t$ and the history of past prescriptions. Thus, we can
write:

$\Gamma_{t}=d_{t}(C_{t},\Gamma_{1:t-1}),$ (10)

where $d_{t}$ is a mapping from
$\mathcal{C}_{t}\times\mathcal{B}_{1}\ldots\times\mathcal{B}_{t-1}$ to
$\mathcal{B}_{t}$. The collection of mappings $d:=(d_{1},\ldots,d_{T})$ is
referred to as the coordination strategy. The coordinator’s objective is to
choose a coordination strategy that minimizes the finite horizon total
expected cost:

$\mathcal{J}(d):=\operatorname{\mathbb{E}}^{d}\left[\sum_{t=1}^{T}k_{t}({X}_{t},{U}_{t})\right].$
(11)

The following lemma establishes the equivalence of the coordinator problem
formulated above and the problem Problem P1. The use of identical
prescriptions by the coordinator is needed to connect the coordinator’s
strategy to symmetric strategies for the agents in Problem P1. Problem P1 and
the coordinator’s problem are equivalent in the following sense:
(i) For any symmetric strategy pair $(g,g)$, consider the following
coordination strategy:

$d_{t}(C_{t})=g_{t}(\cdot,C_{t}).$

Then, $J(g,g)=\mathcal{J}(d)$. (ii) Conversely, for any coordination strategy
$d$, consider the symmetric strategy pair defined as follows:

$g_{t}(\cdot,C_{t})=d_{t}(C_{t},\Gamma_{1:t-1}),$

where $\Gamma_{k}=d_{k}(C_{k},\Gamma_{1:k-1})$ for $k=1,\ldots,t-1$.

###### Proof.

The proof is based on Proposition 3 of [2] and the fact that the use of
identical prescriptions for the two agents by the coordinator corresponds to
the use of symmetric strategies in Problem P1.

We now proceed with finding a solution for the coordinator’s problem. As shown
in [2], the coordinator’s belief on $(X_{t},P_{t})$ can serve as its
information state (sufficient statistic) for selecting prescriptions. At time
$t$, the coordinator’s belief is given as:

$\displaystyle\Pi_{t}(x,p)=\operatorname{{\mathds{P}}}(X_{t}=x,P_{t}=p|C_{t},\Gamma_{1:(t-1)}),$
(12)

for all
$x\in\mathcal{X}^{0}\times\mathcal{X}\times\mathcal{X},p\in\mathcal{P}_{t}\times\mathcal{P}_{t}$.
The belief can be sequentially updated by the coordinator as described in
Lemma F below. The lemma follows from arguments similar to those in Lemma 2 of
[13] (or Theorem 1 of [2]).

For any coordination strategy $d$, the coordinator’s belief $\Pi_{t}$ evolves
almost surely as

$\Pi_{t+1}=\eta_{t}(\Pi_{t},\Gamma_{t},Z_{t+1}),$ (13)

where $\eta_{t}$ is a fixed transformation that does not depend on the
coordination strategy.

Using the results in [2], we can write a dynamic program for the coordinator’s
problem. Recall that $\mathcal{B}_{t}$ is the space of all functions from
$\mathcal{P}_{t}$ to $\Delta(\mathcal{U})$. For a $\gamma\in\mathcal{B}_{t}$
and $p\in\mathcal{P}_{t}$, $\gamma(p)$ is a probability distribution on
$\mathcal{U}$. Let $\gamma(p;u)$ denote the probability assigned to
$u\in\mathcal{U}$ under the probability distribution $\gamma(p)$.

The value functions for the coordinator’s dynamic program are as follows:
Define $V_{T+1}(\pi_{T+1})=0$ for every $\pi_{T+1}$. For $t\leq T$ and for any
realization $\pi_{t}$ of $\Pi_{t}$, define

$\displaystyle V_{t}(\pi_{t})=\min_{\gamma_{t}\in\mathcal{B}_{t}}\mathbb{E}[$
$\displaystyle k_{t}(X_{t},U_{t})+$ $\displaystyle
V_{t+1}(\eta_{t}(\pi_{t},\gamma_{t},Z_{t+1}))|\Pi_{t}=\pi_{t},\Gamma_{t}=\gamma_{t}]$
(14)

The coordinator’s optimal strategy is to pick the minimizing prescription for
each time and each $\pi_{t}$.

###### Proof.

As noted in [2], the coordinator’s problem can be seen as a POMDP. The theorem
is simply the POMDP dynamic program for the coordinator.

The expectation in (14) should be interpreted as follows: $Z_{t+1}$ is given
by (II-A), $U^{i}_{t},i=1,2,$ is independently randomly generated according to
the distribution $\gamma_{t}(P^{i}_{t})$ and the joint distribution on
$(X_{t},P_{t})$ is $\pi_{t}$.

It can be established by backward induction that the term being minimized in
(14) is a continuous function of $\gamma_{t}$. This can be shown using an
argument very similar to the one used in the proof of Lemma 3 in [14]. This
continuity property along with the fact that $\mathcal{B}_{t}$ is a compact
set ensures that the minimum in (14) is achieved.

For the instances of Problem P1 described in Problems P1a - P1c (see Section
II), the private information of an agent includes its current local state.
Consequently, for these instances, the coordinator’s belief is just on the
private information of the agents and the current shared state. The following
lemma shows that this belief can be factorized into beliefs on each agent’s
private information and a degenerate belief on the shared state.

In Problems 1a - 1c, for any realization $x^{0}$ of the shared state and any
realizations $p^{1},p^{2}$ of the agents’ private information,

$\Pi_{t}(x^{0},p^{1},p^{2})=\delta_{X^{0}_{t}}(x^{0})\Pi^{1}_{t}(p^{1})\Pi^{2}_{t}(p^{2}),$
(15)

where $\Pi_{t}$ is the coordinator’s belief (see (12)),
$\Pi^{1}_{t},\Pi^{2}_{t}$ are the marginals of $\Pi_{t}$ for each agent’s
private information and $\delta_{X^{0}_{t}}(\cdot)$ is a delta distribution
located at $X^{0}_{t}$. (Recall that $X^{0}_{t}$ is part of the common
information in Problems P1a-P1c.)

Further, for any coordination strategy $d$, $\Pi^{i}_{t},i=1,2,$ evolves
almost surely as

$\Pi^{i}_{t+1}=\eta^{i}_{t}(X^{0}_{t},\Pi^{i}_{t},\Gamma_{t},Z_{t+1}),$ (16)

where $\eta^{i}_{t}$ is a fixed transformation that does not depend on the
coordination strategy.

###### Proof.

See Appendix I.

Because of the above lemma, we can replace $\Pi_{t}$ (and its realizations
$\pi_{t}$) by $(\Pi^{1}_{t},\Pi^{2}_{t},X^{0}_{t})$ (and the corresponding
realizations $(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t})$) in the dynamic program of
Theorem III for Problems P1a -P1c.

##  IV Comparison of Problems 1b and 1c

The information structures in Problems P1b and P1c differ only in the private
information available to the agents – in P1b, each agent know its entire local
state history whereas in P1c each agent knows only its current local state.
_If the agents were not restricted to use the same strategies,_ it is known
that the two information structures are equivalent. That is, if a (possibly
asymmetric) strategy pair $(g^{1},g^{2})$ is optimal for the information
structure in Problem P1c, then it is also optimal for the information
structure in Problem P1b [3]. This effectively means that agents can ignore
their past local states without any loss in performance. However, such an
equivalence of the two information structures may not hold when agents are
restricted to use symmetric strategies. In other words, an optimal symmetric
strategy in Problem P1c may not be optimal for Problem P1b; and the optimal
performance in Problem P1c may be strictly worse than the optimal performance
in Problem P1b. We explore this point in more detail below.

One approach for establishing that agents can ignore parts of their private
information that has been commonly used in prior literature on multi-
agent/decentralized systems is the agent-by-agent (or person-by-person
approach) [15, 2]. This approach works as follows: We start by fixing
strategies of all agents other than agent $i$ to arbitrary choices and then
show that agent $i$ can make decisions based on a subset or a function of its
private information without compromising performance. If this reduction in
agent $i$’s information holds for any arbitrary strategy of other agents, we
can conclude that this reduction would hold for globally optimal strategies as
well. By repeating this argument for all agents, one can reduce the private
information of all agents without losing performance. The problem with this
approach is that it cannot accommodate the restriction to symmetric
strategies. The reduced-information based strategies obtained using this
approach may or may not be symmetric. Thus, we cannot adopt this approach for
reducing agents’ private information in Problem P1b.

Another approach for reducing private information that has been used in some
game-theoretic settings [14] involves the use of conditional probabilities of
actions given reduced information. To see how this approach can be used, let’s
consider an arbitrary (possible asymmetric) strategy pair $(g^{1},g^{2})$ for
the information structure of Problem P1b and define the following conditional
probabilities for $i=1,2$:

$\displaystyle\operatorname{{\mathds{P}}}^{({g}^{1},{g}^{2})}[U^{i}_{t}=u|$
$\displaystyle X^{i}_{t}=x,C_{t}=c_{t}].$ (17)

Note that (17) specifies a probability distribution on $\mathcal{U}$ for each
$x$ and $c_{t}$. Thus, it can be viewed as a valid strategy for agent $i$
_under the information structure of Problem P1c_. This observation lets us
define the following reduced-information strategies for the agents:

$\displaystyle\bar{g}^{i}_{t}(x,c_{t}):=\operatorname{{\mathds{P}}}^{({g}^{1},{g}^{2})}[U^{i}_{t}=\cdot|$
$\displaystyle X^{i}_{t}=x,C_{t}=c_{t}],\leavevmode\nobreak\ i=1,2.$ (18)

Further, it can be shown that the above construction ensures that the joint
distributions of $(X_{t},U_{t},C_{t})$ under strategies $(g^{1},g^{2})$ and
$(\bar{g}^{1},\bar{g}^{2})$ are the same for all $t$. This, in turn, implies
that $J(\bar{g}^{1},\bar{g}^{2})=J(g^{1},g^{2})$. This argument establishes
that there is a reduced-information strategy pair with the same performance as
an arbitrary full-information strategy pair. Thus, the optimal performance
with reduced-information strategies must be the same as the optimal
performance with full-information strategies for the information structure of
Problem P1b.

We can try to use the above argument for symmetric strategy pairs. We start
with an arbitrary symmetric strategy pair $(g,g)$ in Problem P1b and use (18)
to define a reduced-information strategy pair that achieves the same
performance as $(g,g)$. The problem with this argument is that even though we
started with a symmetric strategy pair $(g,g)$, the reduced-information
strategy pair constructed by (18) need not be symmetric. Hence, this reduced-
information strategy pair may not be a valid solution for Problem P1c. We
illustrate this point in the following example.

_Example 2:_ Consider a setting where there is no shared state, the action
space is $\mathcal{U}=\\{a,b\\}$ and the local states are i.i.d. (across time
and across agents). Each local state is a Bernoulli (1/2) random variable.
Consider the symmetric strategy pair $(g,g)$ for Problem P1b where $g_{1}$
(the strategy at $t=1$) is:

$g_{1}(u^{i}_{1}=a|x^{i}_{1})=\begin{cases}\alpha,\leavevmode\nobreak\
\text{if}\leavevmode\nobreak\ x^{i}_{1}=0\\\ \beta,\leavevmode\nobreak\
\text{if}\leavevmode\nobreak\ x^{i}_{1}=1,\end{cases}$ (19)

where $0\leq\alpha,\beta\leq 1$. And $g_{2}$ (the strategy at $t=2$) is:

$g_{2}(u^{i}_{2}=a|x^{i}_{1},x^{i}_{2},u^{1}_{1},u^{2}_{1})=\begin{cases}\alpha,\leavevmode\nobreak\
\text{if}\leavevmode\nobreak\ x^{i}_{1}=x^{i}_{2}\\\
\beta,\leavevmode\nobreak\ \text{if}\leavevmode\nobreak\ x^{i}_{1}\neq
x^{i}_{2}.\end{cases}$ (20)

We now use (18) to define a reduced-information strategy. Even though we
started with a symmetric strategy pair for the two agents, the conditional
probability on the right hand side of (18) may be different for the two
agents. To see this, consider $t=2$ and $C_{2}=(U^{1}_{1},U^{2}_{1})=(a,b)$
and $X^{i}_{2}=0$. Then, for agent 1:

$\displaystyle\operatorname{{\mathds{P}}}^{(g,g)}(U^{1}_{2}=a|X^{1}_{2}=0,U^{1}_{1}=a,U^{2}_{1}=b)$
$\displaystyle=\operatorname{{\mathds{P}}}^{(g,g)}(U^{1}_{2}=a,X^{1}_{1}=0|X^{1}_{2}=0,U^{1}_{1}=a,U^{2}_{1}=b)$
$\displaystyle+\operatorname{{\mathds{P}}}^{(g,g)}(U^{1}_{2}=a,X^{1}_{1}=1|X^{1}_{2}=0,U^{1}_{1}=a,U^{2}_{1}=b)$
$\displaystyle=\alpha\operatorname{{\mathds{P}}}^{(g,g)}(X^{1}_{1}=0|X^{1}_{2}=0,U^{1}_{1}=a,U^{2}_{1}=b)$
$\displaystyle+\beta\operatorname{{\mathds{P}}}^{(g,g)}(X^{1}_{1}=1|X^{1}_{2}=0,U^{1}_{1}=a,U^{2}_{1}=b)$
$\displaystyle=\alpha\left(\frac{\alpha}{\alpha+\beta}\right)+\beta\left(\frac{\beta}{\alpha+\beta}\right)=\frac{\alpha^{2}+\beta^{2}}{\alpha+\beta}$
(21)

On the other hand, a similar calculation for agent 2 shows that:
$\operatorname{{\mathds{P}}}^{(g,g)}(U^{2}_{2}=a|X^{2}_{2}=0,U^{1}_{1}=a,U^{2}_{1}=b)$

$\displaystyle=\alpha\left(\frac{1-\alpha}{2-\alpha-\beta}\right)+\beta\left(\frac{1-\beta}{2-\alpha-\beta}\right)=\frac{\alpha+\beta-\alpha^{2}-\beta^{2}}{2-\alpha-\beta}.$
(22)

The expressions in (21) and (22) are clearly different. For example, with
$\alpha=1/4$ and $\beta=1/2$, (21) evaluates to $5/12$ while (22) evaluates to
$7/20$. Thus, the reduced-information strategies constructed by (18) are not
symmetric and, therefore, invalid for Problem P1c.

###  IV-A Special cases

In this section, we present two special cases under which Problems P1b and P1c
can be shown to be equivalent, i.e., we can show that an optimal strategy for
Problem P1c is also optimal for Problem P1b.

####  IV-A1 Specialized cost

We assume that the cost function at each time $t$ is non-negative, i.e.,
$k_{t}(X^{0}_{t},X^{1}_{t},X^{2}_{t},U^{1}_{t},U^{2}_{t})\geq 0$. Further, we
assume that for each possible local state $x^{i}$ of agent $i$ there exists an
action $m(x^{i})$ such that $k_{t}(x^{0},x^{1},x^{2},m(x^{1}),m(x^{2}))=0$ for
all $x^{0}\in\mathcal{X}^{0}$. An example of such a cost function is
$k_{t}(X^{0}_{t},X^{1}_{t},X^{2}_{t},U^{1}_{t},U^{2}_{t})=(X^{1}_{t}-U^{1}_{t})^{2}[(X^{2}_{t}-U^{2}_{t})^{2}+1]+(X^{2}_{t}-U^{2}_{t})^{2},$
where the states and actions are integer-valued.

Recall that in Problem P1b the prescription space at time $t$ is the space of
functions from $\mathcal{X}^{t}$ to $\Delta(\mathcal{U})$ and in Problem P1c
the prescription space is the space of functions from $\mathcal{X}$ to
$\Delta(\mathcal{U})$. Using the dynamic programs for Problems P1b and P1c
with the specialized cost above, we can show that optimal prescriptions in
both problems effectively coincide with the mapping $m$ from $\mathcal{X}$ to
$\mathcal{U}$222With a slight abuse of notation, the function $m$ from
$\mathcal{X}$ to $\mathcal{U}$ can be viewed as a deterministic prescription
from $\mathcal{X}$ to $\Delta(\mathcal{U})$ or from $\mathcal{X}^{t}$ to
$\Delta(\mathcal{U})$. .

The value functions for the coordinator’s dynamic programs in Problems P1b and
P1c can be written as follows: For $t\leq T$ and for any realization
$\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t}$ of $\Pi^{1}_{t},\Pi^{2}_{t},X^{0}_{t}$,

$\displaystyle V_{t}(\pi^{1}_{t}$
$\displaystyle,\pi^{2}_{t},x^{0}_{t}):=\min_{\gamma_{t}\in\mathcal{B}_{t}}Q_{{t}}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},\gamma_{t}),$
(23)

where the function $Q$ satisfies

$\displaystyle Q_{{t}}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},\gamma_{t})\geq
Q_{{t}}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},m)=0,$ (24)

Consequently, the coordinator’s optimal prescription is $m$ at each time.

###### Proof.

See Appendix C.

Since the coordinator’s optimal strategy is identical in Problems P1b and P1c,
it follows that the optimal symmetric strategy for the agents in the two
problems is also the same, namely $U^{i}_{t}=m(X^{i}_{t})$.

####  IV-A2 Specialized dynamics

We consider a specialized dynamics where the local states $X^{i}_{1:T},i=1,2,$
are $\mathrm{i.i.d.}$ uncontrolled random variables with probability
distribution $\alpha$ and there is no shared state. The following theorem
shows the equivalence between Problems P1b and P1c in terms of optimal
performance and strategies.

The optimal performance in Problem P1c is the same as the optimal performance
in Problem P1b. Further, the optimal symmetric strategy for Problem P1c is
optimal for Problem P1b as well.

###### Proof.

See Appendix D.

In summary, for the specialized cases described above, one can reduce the
private information of the agents without losing performance, even with the
restriction to symmetric strategies.

##  V Comparison of Problems 1a and 1c

The information structures in Problems P1a and P1c differ only in the common
information available to the agents – in P1a, each agent has an additional
part in the common information consisting of the local state history of both
agents. Since agents in Problem P1c have less information that their
counterparts in Problem P1a, an optimal symmetric strategy in Problem P1c may
not optimal for Problem P1a; and the optimal performance in Problem P1c may be
strictly worse then the optimal performance in Problem P1a.

_Example 3:_ Consider a setting where there is no shared state, the action
space is $\mathcal{U}=\\{0,1\\}$ and state space is $\mathcal{X}=\\{0,1\\}$.
Let $T=2$ and the local states of each agent are stationary across time. The
initial states $X^{1}_{1},X^{2}_{1}$ are independent random variables with
probability distribution Bernoulli (1/2). The cost at time $t=1$ is given by
$k_{1}(X_{1},U_{1})=10\mathds{1}_{\\{U^{1}_{1}\neq 0,U^{2}_{1}\neq 0\\}}$ and
cost at time $t=2$ is given by:

$k_{2}(X_{2},U_{2})=\begin{cases}0,\leavevmode\nobreak\
\text{if}\leavevmode\nobreak\ U^{1}_{2}=X^{2}_{2}\leavevmode\nobreak\
and\leavevmode\nobreak\ U^{2}_{2}=X^{1}_{2}\\\ 1,\leavevmode\nobreak\
\text{otherwise},\end{cases}$ (25)

Consider the symmetric strategy pair $(g,g)$ for Problem P1a where $g_{1}$
(the strategy at $t=1$) is: $U^{1}_{1}=0,U^{2}_{1}=0$. At time $t=2$, each
agent uses the following strategy: $U^{1}_{2}=X^{1}_{1},U^{2}_{2}=X^{2}_{1}$
if $X^{1}_{1}=X^{2}_{1}$ and $U^{1}_{2}=1-X^{1}_{1},U^{2}_{2}=1-X^{2}_{1}$ if
$X^{1}_{1}\neq X^{2}_{1}$. This results in optimal expected cost of $0$ in
Problem P1a. In Problem P1c, it can be shown that the optimal strategy at time
$t=1$ is $U^{1}_{1}=0,U^{2}_{1}=0$ and at time $t=2$, $U^{i}_{2}$ follows
probability distribution Bernoulli (1/2). The optimal expected cost is $0.75$
for the Problem P1c, which is strictly worse than optimal performance in
Problem P1a.

###  V-A Special Case

We present a special dynamics under which problems P1a and P1c can be shown to
be equivalent i.e. we can show that an optimal strategy for Problem P1a is
also optimal for Problem P1c. The dynamics of the shared and local states in
the specialized dynamics problem are as follows:

$X^{0}_{t+1}=f^{0}_{t}(X^{0}_{t},U_{t},W^{0}_{t}),$ (26)
$X^{i}_{t+1}=f_{t}(X^{0}_{t},U_{t},W^{i}_{t}),\leavevmode\nobreak\
\leavevmode\nobreak\ i=1,2.$ (27)

The shared and local state dynamics are similar to (1) and (2) except that in
local dynamics the next local state doesn’t depend on the current local state.
In this case, we have the following result:

For the specialized dynamics described in (26) - (27), an optimal symmetric
strategy in Problem P1c is also optimal for Problem P1a and, consequently, the
optimal performance in the two problems are the same.

###### Proof.

See Appendix E.

##  VI Conclusion

In this paper, we focused on designing symmetric strategies to optimize a
finite horizon team objective. We started with a general information structure
and then considered some special cases. We showed in a simple example that
randomized symmetric strategies may outperform deterministic symmetric
strategies. We also discussed why some of the known approaches for reducing
agents’ private information in teams may not work under the constraint of
symmetric strategies. We modified the common information approach to obtain
optimal symmetric strategies for the agents. This resulted in a common
information based dynamic program whose complexity depends in large part on
the size of the private information space. We presented two specialized models
where private information can be reduced using simple dynamic program based
arguments.

## References

  * [1] A. Nayyar, A. Mahajan, and D. Teneketzis, “Optimal control strategies in delayed sharing information structures,” IEEE Transactions on Automatic Control, vol. 56, no. 7, pp. 1606–1620, 2010.
  * [2] A. Nayyar, A. Mahajan, and D. Teneketzis, “Decentralized stochastic control with partial history sharing: A common information approach,” IEEE Transactions on Automatic Control, vol. 58, no. 7, pp. 1644–1658, 2013.
  * [3] A. Mahajan, “Optimal decentralized control of coupled subsystems with control sharing,” IEEE Transactions on Automatic Control, vol. 58, no. 9, pp. 2377–2382, 2013.
  * [4] S. Sudhakara, D. Kartik, R. Jain, and A. Nayyar, “Optimal communication and control strategies in a multi-agent MDP problem,” arXiv preprint arXiv:2104.10923, 2021.
  * [5] D. Kartik, S. Sudhakara, R. Jain, and A. Nayyar, “Optimal communication and control strategies for a multi-agent system in the presence of an adversary,” arXiv preprint arXiv:2209.03888, 2022.
  * [6] S. Seuken and S. Zilberstein, “Formal models and algorithms for decentralized decision making under uncertainty,” Autonomous Agents and Multi-Agent Systems, vol. 17, no. 2, pp. 190–250, 2008.
  * [7] A. Kumar, S. Zilberstein, and M. Toussaint, “Probabilistic inference techniques for scalable multiagent decision making,” Journal of Artificial Intelligence Research, vol. 53, pp. 223–270, 2015.
  * [8] T. Rashid, M. Samvelyan, C. Schroeder, G. Farquhar, J. Foerster, and S. Whiteson, “Qmix: Monotonic value function factorisation for deep multi-agent reinforcement learning,” in International Conference on Machine Learning, pp. 4295–4304, PMLR, 2018.
  * [9] H. Hu and J. N. Foerster, “Simplified action decoder for deep multi-agent reinforcement learning,” in International Conference on Learning Representations, 2019.
  * [10] D. Szer, F. Charpillet, and S. Zilberstein, “MAA* a heuristic search algorithm for solving decentralized POMDPs,” in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence, pp. 576–583, 2005.
  * [11] M. J. Neely, “Repeated games, optimal channel capture, and open problems for slotted multiple access,” arXiv preprint arXiv:2110.09638, 2021.
  * [12] S. Yüksel and T. Basar, “Stochastic networked control systems: Stabilization and optimization under information constraints.,” Springer Science & Business Media, 2013.
  * [13] D. Kartik and A. Nayyar, “Upper and lower values in zero-sum stochastic games with asymmetric information,” Dynamic Games and Applications, vol. 11, no. 2, pp. 363–388, 2021.
  * [14] D. Kartik, A. Nayyar, and U. Mitra, “Common information belief based dynamic programs for stochastic zero-sum games with competing teams,” arXiv preprint arXiv:2102.05838, 2021.
  * [15] Y.-C. Ho, “Team decision theory and information structures,” Proceedings of the IEEE, vol. 68, no. 6, pp. 644–654, 1980.

##  Appendix A Proof of Lemma III

To prove Lemma III, we first show that the private information of the agents
are conditionally independent given the common information under any
strategies. For Problem P1a, this is straightforward since the disturbances in
the dynamics are independent:

$\displaystyle\operatorname{{\mathds{P}}}(X^{1,2}_{t}$
$\displaystyle=x^{1,2}_{t}|C_{t}=(x^{1,2}_{1:t-1},x^{0}_{1:t},u_{1:t-1}))$
$\displaystyle=$
$\displaystyle\operatorname{{\mathds{P}}}(f_{t-1}(x^{1}_{t-1},x^{0}_{t-1},u_{t-1},W^{1}_{t-1})=x^{1}_{t})\times$
$\displaystyle\operatorname{{\mathds{P}}}(f_{t-1}(x^{2}_{t-1},x^{0}_{t-1},u_{t-1},W^{2}_{t-1})=x^{2}_{t}).$

For Problems P1b and P1c, we have the following lemma. [Conditional
independence property] Consider any arbitrary (symmetric or asymmetric) choice
of agents’ strategies in Problems P1b and P1c. Then, at any time $t$, the two
agents’ private information are conditionally independent given the common
information $C_{t}$. That is, if $c_{t}$ is the realization of the common
information at time $t$ then for any realization $p_{t}$ of private
information, we have

$\operatorname{{\mathds{P}}}^{(g^{1},g^{2})}({p}_{t}|c_{t})=\displaystyle\prod_{i=1}^{2}\operatorname{{\mathds{P}}}^{g^{i}}(p^{i}_{t}|c_{t}),$
(28)

Further, $\operatorname{{\mathds{P}}}^{g^{i}}(p^{i}_{t}|c_{t})$ depends only
on agent $i$’ strategy and not on the strategy of agent $-i$.

###### Proof.

The proof is analogous to the proof of [3, Proposition 1] except for the
possible randomization in agents’ strategies.

Using the above conditional independence property for Problems P1a-P1c, we can
now prove (15). At time $t$, the coordinator’s belief is given as:

$\Pi_{t}(x^{0}_{t},p^{1}_{t},p^{2}_{t})=\operatorname{{\mathds{P}}}(X^{0}_{t}=x^{0}_{t},P^{1}_{t}=p^{1}_{t},P^{2}_{t}=p^{2}_{t}|C_{t},\Gamma_{1:t-1})$
(29)

for any realization $x^{0}_{t}$ of the global state and any realizations
$p^{1}_{t},p^{2}_{t}$ of the agents’ private information. Since $X^{0}_{t}$ is
part of $C_{t}$, the coordinator’s belief can be factorized into:

$\displaystyle\operatorname{{\mathds{P}}}(x^{0}_{t},p^{1}_{t},p^{2}_{t}|C_{t},\Gamma_{1:t-1})=\delta_{X^{0}_{t}}(x^{0}_{t})\operatorname{{\mathds{P}}}(p^{1}_{t},p^{2}_{t}|C_{t},\Gamma_{1:t-1})$
$\displaystyle=\delta_{X^{0}_{t}}(x^{0}_{t})\operatorname{{\mathds{P}}}(p^{1}_{t}|C_{t},\Gamma_{1:t-1})\operatorname{{\mathds{P}}}(p^{2}_{t}|C_{t},\Gamma_{1:t-1})$
$\displaystyle=\delta_{X^{0}_{t}}(x^{0}_{t})\Pi^{1}_{t}(p^{1}_{t})\Pi^{2}_{t}(p^{2}_{t}),$

where we used the above-mentioned conditional independence.

We now prove (16) for Problems P1a-P1c.

####  A-1 Problem P1a

In Problem P1a, let $\pi^{i}_{t+1}$ be the realization of the coordinator’s
marginal belief $\Pi^{i}_{t+1}$ for each agent’s private information and
$c_{t+1}$ be the realization of common information at time $t+1$. The belief
for Problem P1a is given by:

$\displaystyle\pi^{i}_{t+1}(x^{i}_{t+1})=\operatorname{{\mathds{P}}}(x^{i}_{t+1}|c_{t+1}=(x^{1,2}_{1:t},x^{0}_{1:t+1},u_{1:t}),\gamma_{1:t})$
(30)

Using Bayes’ rule, we have

$\displaystyle\pi^{i}_{t+1}(x^{i}_{t+1})=\frac{\operatorname{{\mathds{P}}}(x^{i}_{t+1},x^{0}_{t+1}|x^{1,2}_{1:t},x^{0}_{1:t},u_{1:t},\gamma_{1:t})}{\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{1,2}_{1:t},x^{0}_{1:t},u_{1:t},\gamma_{1:t})}$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(f_{t}(x^{0}_{t},u_{t},W^{0}_{t})=x^{0}_{t+1})\operatorname{{\mathds{P}}}(f_{t}(x^{i}_{t},x^{0}_{t},u_{t},W^{i}_{t})=x^{i}_{t+1})}{\operatorname{{\mathds{P}}}(f_{t}(x^{0}_{t},u_{t},W^{0}_{t})=x^{0}_{t+1})}$
$\displaystyle=\operatorname{{\mathds{P}}}(f_{t}(x^{i}_{t},x^{0}_{t},u_{t},W^{i}_{t})=x^{i}_{t+1})$

Thus $\pi^{i}_{t+1}$ is determined by $x^{0}_{t}$ and the increment in common
information.

####  A-2 Problem P1b

In Problem P1b, let $\pi^{i}_{t+1}$ be the realization of the coordinator’s
marginal belief $\Pi^{i}_{t+1}$ for each agent’s private information and
$c_{t+1}$ be the realization of common information at time $t+1$. The belief
for agent $1$ is given by:

$\displaystyle\pi^{1}_{t+1}(x^{1}_{1:t+1})=\operatorname{{\mathds{P}}}(x^{1}_{1:t+1}|c_{t+1}=(x^{0}_{1:t+1},u_{1:t}),\gamma_{1:t})$
(31)

Using Bayes’ rule, we have $\pi^{1}_{t+1}(x^{1}_{1:t+1})$

$\displaystyle=\frac{\sum_{x^{2}_{1:t}}\operatorname{{\mathds{P}}}(x^{1}_{1:t+1},x^{0}_{t+1},u_{t},x^{2}_{1:t}|x^{0}_{1:t},u_{1:t-1},\gamma_{1:t})}{\sum_{\tilde{x}^{1}_{1:t+1}}\sum_{\tilde{x}^{2}_{1:t}}\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{1:t+1},x^{0}_{t+1},u_{t},\tilde{x}^{2}_{1:t}|x^{0}_{1:t},u_{1:t-1},\gamma_{1:t})}$
(32)

The numerator of (32) using state dynamics, coordinator prescription and
belief at time $t$, can be written as

$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{t+1}|x^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(x^{1}_{1:t})}$
$\displaystyle\times\pi^{1}_{t}(x^{1}_{1:t})\sum_{x^{2}_{1:t}}\mathds{1}_{u^{2}_{t}=\gamma(x^{2}_{1:t})}\pi^{2}_{t}(x^{2}_{1:t})$
(33)

Similarly the denominator can be written as

$\displaystyle\sum_{\tilde{x}^{1}_{1:t+1}}\sum_{\tilde{x}^{2}_{1:t}}$
$\displaystyle\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{t+1}|\tilde{x}^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(\tilde{x}^{1}_{1:t})}$
$\displaystyle\times\mathds{1}_{u^{2}_{t}=\gamma(\tilde{x}^{2}_{1:t})}\pi^{1}_{t}(\tilde{x}^{1}_{1:t})\pi^{2}_{t}(\tilde{x}^{2}_{1:t})$
(34)

Let $z^{b}_{t+1}:=(x^{0}_{t+1},u_{t})$ be the increment in the common
information in Problem P1b. Substituting equations (33), (34) in equation
(32), we derive $\pi^{1}_{t+1}(x^{1}_{1:t+1})$ as,

$\displaystyle\frac{\operatorname{{\mathds{P}}}(x^{1}_{t+1}|x^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(x^{1}_{1:t})}\pi^{1}_{t}(x^{1}_{1:t})}{\sum_{\tilde{x}^{1}_{1:t+1}}\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{t+1}|\tilde{x}^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(\tilde{x}^{1}_{1:t})}\pi^{1}_{t}(\tilde{x}^{1}_{1:t})}$
(35)

We denote the update rule described above with $\eta^{i}_{t}$, i.e.

$\eta^{i}_{t}(x^{0}_{t},\pi^{1}_{t},\gamma_{t},z^{b}_{t+1})$ (36)

####  A-3 Problem P1c

In problem P1c, let $\pi^{i}_{t+1}$ be the realization of the coordinator’s
marginal belief $\Pi^{i}_{t+1}$ for each agent’s private information and
$c_{t+1}$ be the realization of common information at time $t+1$. The belief
for agent $1$ is given by:

$\displaystyle\pi^{1}_{t+1}(x^{1}_{t+1})=\operatorname{{\mathds{P}}}(x^{1}_{t+1}|c_{t+1}=(x^{0}_{1:t+1},u_{1:t}),\gamma_{1:t})$
(37)

Using Bayes’ rule, we have $\pi^{1}_{t+1}(x^{1}_{t+1})$

$\displaystyle=\frac{\sum_{x^{1,2}_{t}}\operatorname{{\mathds{P}}}(x^{1}_{t+1},x^{0}_{t+1},u_{t},x^{1,2}_{t}|x^{0}_{1:t},u_{1:t-1},\gamma_{1:t})}{\sum_{\tilde{x}^{1}_{t+1},\tilde{x}^{1,2}_{t}}\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{t+1},x^{0}_{t+1},u_{t},\tilde{x}^{1,2}_{t}|x^{0}_{1:t},u_{1:t-1},\gamma_{1:t})}$
(38)

The numerator of (38) using state dynamics, coordinator prescription and
belief at time $t$, can be written as

$\displaystyle\sum_{x^{1,2}_{t}}\operatorname{{\mathds{P}}}(x^{1}_{t+1}|x^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(x^{1}_{t})}$
$\displaystyle\times\mathds{1}_{u^{2}_{t}=\gamma(x^{2}_{t})}\pi^{1}_{t}(x^{1}_{t})\pi^{2}_{t}(x^{2}_{t})$
(39)

Similarly the denominator can be written as

$\displaystyle\sum_{\tilde{x}^{1}_{t+1}}\sum_{\tilde{x}^{1,2}_{t}}$
$\displaystyle\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{t+1}|\tilde{x}^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(\tilde{x}^{1}_{t})}$
$\displaystyle\times\mathds{1}_{u^{2}_{t}=\gamma(\tilde{x}^{2}_{t})}\pi^{1}_{t}(\tilde{x}^{1}_{t})\pi^{2}_{t}(\tilde{x}^{2}_{t})$
(40)

Let $z^{c}_{t+1}:=(x^{0}_{t+1},u_{t})$ be the increment in the common
information in problem P1c. Substituting equations (39), (40) in equation
(38), we derive $\pi^{1}_{t+1}(x^{1}_{t+1})$ as

$\displaystyle\frac{\sum_{x^{1}_{t}}\operatorname{{\mathds{P}}}(x^{1}_{t+1}|x^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(x^{1}_{t})}\pi^{1}_{t}(x^{1}_{t})}{\sum_{\tilde{x}^{1}_{t+1},\tilde{x}^{1}_{t}}\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{t+1}|\tilde{x}^{1}_{t},x^{0}_{t},u_{t})\operatorname{{\mathds{P}}}(x^{0}_{t+1}|x^{0}_{t},u_{t})\mathds{1}_{u^{1}_{t}=\gamma(\tilde{x}^{1}_{t})}\pi^{1}_{t}(\tilde{x}^{1}_{t})}$
(41)

We denote the update rule described above with $\eta^{i}_{t}$, i.e.

$\eta^{i}_{t}(x^{0}_{t},\pi^{1}_{t},\gamma_{t},z^{c}_{t+1})$ (42)

##  Appendix B Proof of Lemma E

Then the coordinator’s belief state can serve as the sufficient statistic for
selecting prescriptions. Let $c_{t+1}:=(x_{1:t},x^{0}_{1:t+1},u_{1:t})$ be the
realization of common information $C_{t+1}$ and $\gamma_{1:t}$ be the
realization of the prescription $\Gamma_{1:t}$. The coordinator’s belief in
Problem P1a for all $x^{i}_{t+1}\in\mathcal{X}$ is given as:

$\displaystyle\pi^{i}_{t+1}(x^{i}_{t+1})=\operatorname{{\mathds{P}}}(X^{i}_{t+1}=x^{i}_{t+1}|x^{1,2}_{1:t},x^{0}_{1:t+1},u_{1:t},\gamma_{1:t}),$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(X^{i}_{t+1}=x^{i}_{t+1},X^{0}_{t+1}=x^{0}_{t+1}|x^{1,2}_{1:t},x^{0}_{1:t},u_{1:t},\gamma_{1:t})}{\operatorname{{\mathds{P}}}(X^{0}_{t+1}=x^{0}_{t+1}|x^{1,2}_{1:t},x^{0}_{1:t},u_{1:t},\gamma_{1:t})}$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(f^{0}_{t}(x^{0}_{t},u_{t},W^{0}_{t})=x^{0}_{t+1})\operatorname{{\mathds{P}}}(f_{t}(x^{0}_{t},u_{t},W^{i}_{t})=x^{i}_{t+1})}{\operatorname{{\mathds{P}}}(f^{0}_{t}(x^{0}_{t},u_{t},W^{0}_{t})=x^{0}_{t+1})}$
$\displaystyle=\operatorname{{\mathds{P}}}(f_{t}(x^{0}_{t},u_{t},W^{i}_{t})=x^{i}_{t+1}).$
(43)

Let $c_{t+1}:=(x^{0}_{1:t+1},u_{1:t})$ be the realization of common
information $C_{t+1}$ and $\gamma_{1:t}$ be the realization of the
prescription $\Gamma_{1:t}$. The coordinator belief in Problem P1c for all
$x^{i}_{t+1}\in\mathcal{X}$ is given as:

$\displaystyle\Pi^{i}_{t+1}(x^{i}_{t+1})=\operatorname{{\mathds{P}}}(X^{i}_{t+1}=x^{i}_{t+1}|x^{0}_{1:t+1},u_{1:t},\gamma_{1:t}),$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(X^{i}_{t+1}=x^{i}_{t+1},X^{0}_{t+1}=x^{0}_{t+1}|x^{0}_{1:t},u_{1:t},\gamma_{1:t})}{\operatorname{{\mathds{P}}}(X^{0}_{t+1}=x^{0}_{t+1}|x^{0}_{1:t},u_{1:t},\gamma_{1:t})}$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(f^{0}_{t}(x^{0}_{t},u_{t},W^{0}_{t})=x^{0}_{t+1})\operatorname{{\mathds{P}}}(f_{t}(x^{0}_{t},u_{t},W^{i}_{t})=x^{i}_{t+1})}{\operatorname{{\mathds{P}}}(f^{0}_{t}(x^{0}_{t},u_{t},W^{0}_{t})=x^{0}_{t+1})}$
$\displaystyle=\operatorname{{\mathds{P}}}(f_{t}(x^{0}_{t},u_{t},W^{i}_{t})=x^{i}_{t+1}).$
(44)

##  Appendix C Proof of Lemma IV-A1

We prove the lemma by backward induction. Let’s consider Problem P1b. The
value function for the coordinator’s dynamic program at time $T$ can be
written as follows: for any realization $\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T}$ of
$\Pi^{1}_{T},\Pi^{2}_{T},X^{0}_{T}$ respectively,

$\displaystyle V_{T}(\pi^{1}_{T},\pi^{2}_{T},$ $\displaystyle
x^{0}_{T})=\min_{\gamma_{T}\in\mathcal{B}_{T}}Q_{T}(\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T},\gamma_{T}),$
(45)

where

$\displaystyle Q_{T}(\pi^{1}_{T},\pi^{2}_{T},$ $\displaystyle
x^{0}_{T},\gamma_{T}):=\sum_{x_{1:T}}\sum_{u_{T}}k_{T}(x_{T},u_{T})\delta_{x^{0}_{T}}(x^{0})\pi^{1}_{T}(x^{1}_{1:T})$
$\displaystyle\times\pi^{2}_{T}(x^{2}_{1:T})\gamma_{T}(x^{1}_{1:T};u^{1}_{T})\gamma_{T}(x^{2}_{1:T};u^{2}_{T}).$
(46)

$Q_{T}(\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T},\gamma_{T})\geq 0$ because
$k_{T}(\cdot,\cdot)$ is a non negative function. The deterministic mapping $m$
from $\mathcal{X}$ to $\mathcal{U}$ can be viewed as a prescription
$\gamma\in\mathcal{{B}_{T}}$ with $\gamma(x^{i}_{1:T};m(x^{i}_{T}))=1$.

$\displaystyle Q_{T}(\pi^{1}_{T},\pi^{2}_{T},$ $\displaystyle
x^{0}_{T},m):=\sum_{x_{1:T}}\sum_{u_{T}}k_{T}(x_{T},m(x^{1}_{T}),m(x^{2}_{T}))$
$\displaystyle\times\pi^{1}_{T}(x^{1}_{1:T})\pi^{2}_{T}(x^{2}_{1:T})\delta_{x^{0}_{T}}(x^{0})=0,$
(47)

where we used the assumption on the cost function, namely,
$k_{T}(x_{T},m(x^{1}_{T}),m(x^{2}_{T}))=0$. Hence,

$\displaystyle Q_{T}(\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T},\gamma_{T})\geq
Q_{T}(\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T},m)=0,$ (48)

and therefore,

$\displaystyle
V_{T}(\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T})=\min_{\gamma_{T}\in\mathcal{B}_{T}}Q_{T}(\pi^{1}_{T},\pi^{2}_{T},x^{0}_{T},\gamma_{T})=0.$
(49)

_Induction hypothesis:_ Assume the coordinator’s value function
$V_{t+1}(\pi^{1}_{t+1},\pi^{2}_{t+1},x^{0}_{t+1})=0$ for any realization
$\pi^{1}_{t+1},\pi^{2}_{t+1},x^{0}_{t+1}$ at time $t+1$.

At time $t$ we define the function $Q_{t}$ as follows:

$\displaystyle
Q_{t}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},\gamma_{t}):=\sum_{x_{1:t}}\sum_{u_{t}}\pi^{1}_{t}(x^{1}_{1:t})\pi^{2}_{t}(x^{2}_{1:t})\delta_{x^{0}_{t}}(x^{0})\times$
$\displaystyle\gamma_{t}(x^{1}_{1:t};u^{1}_{t})\gamma_{t}(x^{2}_{1:t};u^{2}_{t})k_{t}(x_{t},u_{t})+\mathbb{E}[V_{t+1}(\delta_{x^{0}_{t+1}},$
$\displaystyle\Pi^{1}_{t+1},\Pi^{2}_{t+1})|(\Pi^{1}_{t},\Pi^{2}_{t},X^{0}_{t},\Gamma_{t})=(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},\gamma_{t})]$
(50)

Because of the induction hypothesis, the expectation of the value function at
time $t+1$ is $0$ and we can simplify $Q_{t}$ as follows:

$\displaystyle Q_{t}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},$
$\displaystyle\gamma_{t}):=\sum_{x_{1:{t}}}\sum_{u_{t}}\delta_{x^{0}_{t}}(x^{0})\pi^{1}_{t}(x^{1}_{1:{t}})\pi^{2}_{t}(x^{2}_{1:t})$
$\displaystyle\times\gamma_{t}(x^{1}_{1:t};u^{1}_{t})\gamma_{t}(x^{2}_{1:t};u^{2}_{t})k_{t}(x_{t},u_{t}).$
(51)

Using the same arguments as those used for $Q_{T}$, it follows that

$\displaystyle Q_{t}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},\gamma_{t})\geq
Q_{t}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t},m)=0,$ (52)

and therefore, $V_{t}(\pi^{1}_{t},\pi^{2}_{t},x^{0}_{t})=0$. Thus, the
induction hypothesis is true for all times.

It is clear from the above argument that the optimal prescription for the
coordinator in Problem P1b is $m$ at each time and for any realization of its
information state. Similar arguments can be repeated for the coordinator in
Problem P1c as well.

##  Appendix D Proof of Lemma IV-A2

Because of the specialized dynamics, the coordinator’s belief on each agent’s
private information at time $t$ is given by $\alpha^{1:t}$ for Problem P1b and
by $\alpha$ for Problem P1c.

At these beliefs, the value functions for the coordinators in Problems P1b and
P1c are as follows:

$\displaystyle
V^{b}_{t}(\alpha^{1:t},\alpha^{1:t})=\min_{\gamma^{b}_{t}\in\mathcal{B}_{t}}Q^{b}_{{t}}(\alpha^{1:t},\alpha^{1:t},\gamma^{b}_{t}),$
$\displaystyle
V^{c}_{t}(\alpha,\alpha)=\min_{\gamma^{c}_{t}\in\mathcal{B}_{t}}Q^{c}_{{t}}(\alpha,\alpha,\gamma^{c}_{t})$
(53)

where the functions $Q^{b}_{t}$ and $Q^{c}_{t}$ are defined as

$\displaystyle
Q^{b}_{t}(\alpha^{1:t},\alpha^{1:t},\gamma^{b}_{t}):=\sum_{x_{1:{t}}}\sum_{u_{t}}\alpha^{1:t}(x^{1}_{1:{t}})\alpha^{1:t}(x^{2}_{1:t})\gamma^{b}_{t}(x^{1}_{1:t};u^{1}_{t})$
$\displaystyle\times\gamma^{b}_{t}(x^{2}_{1:t};u^{2}_{t})k_{t}(x_{t},u_{t})+\mathbb{E}[V_{t+1}(\alpha^{1:t+1},\alpha^{1:t+1})],$
$\displaystyle
Q^{c}_{t}(\alpha,\alpha,\gamma^{c}_{t}):=\sum_{x_{{t}}}\sum_{u_{t}}\alpha(x^{1}_{t})\alpha(x^{2}_{t})\gamma^{c}_{t}(x^{1}_{t};u^{1}_{t})\gamma^{c}_{t}(x^{2}_{t};u^{2}_{t})$
$\displaystyle\times k_{t}(x_{t},u_{t})+\mathbb{E}[V_{t+1}(\alpha,\alpha)].$
(54)

Using a backward inductive argument, we can show that for any $\gamma^{b}_{t}$
there exists a $\gamma^{c}_{t}$ such that $Q^{b}_{t}$ and $Q^{c}_{t}$ defined
above are the same (such a $\gamma^{c}_{t}$ must satisfy equations of the
form:
$\sum_{x^{i}_{1:{t-1}}}\gamma^{b}_{t}(x^{i}_{1:t})\alpha^{1:t}(x^{i}_{1:t})=\alpha(x^{i}_{t})\gamma^{c}_{t}(x^{i}_{t})$).
Similarly, we can show that for any $\gamma^{c}_{t}$ there exists a
$\gamma^{b}_{t}$ such that $Q^{b}_{t}$ and $Q^{c}_{t}$ are the same (such a
$\gamma^{b}_{t}$ can be defined as
$\gamma^{b}_{t}(x^{i}_{1:t}):=\gamma^{c}_{t}(x^{i}_{t})$). This relationship
between the two $Q$-functions implies the following equation for the
corresponding value functions:

$\displaystyle V^{b}_{t}(\alpha^{1:t},\alpha^{1:t})=V^{c}_{t}(\alpha,\alpha).$
(55)

The optimal cost in each problem is the value function at time $t=1$ evaluated
at the prior belief $\alpha$. Therefore, (55) at $t=1$ implies that the two
problems have the same optimal performance. Consequently, an optimal symmetric
strategy in Problem P1c will achieve the optimal performance in Problem P1b as
well.

##  Appendix E Proof of Lemma V-A

To prove equivalence between Problems P1a and P1c, we show that the
coordinator’s dynamic program are identical in both cases.

Towards this we show that the coordinator belief is the same for both problems
in the following lemma. For $i=1,2$ and for each $t\geq 1$, if
$x^{0}_{t},u_{t}$ is the realization of shared state $X^{0}_{t}$ and action
$U_{t}$. The beliefs at time $t+1$ in the two problems are the same and given
as follows:

$\displaystyle\pi^{i}_{t+1}=\hat{\eta}^{i}_{t}(x^{0}_{t},u_{t})$ (56)

where $\hat{\eta}^{i}_{t}(x^{0}_{t},u_{t})$ is the probability distribution of
$f_{t}(x^{0}_{t},u_{t},W^{i}_{t})$.

###### Proof.

See Appendix II.

At time $t=1$, the beliefs are the same for both problems and the above lemma
proves that the beliefs are same for all time $t$.

Recall that the private information are same for both the problems, therefore
this leads to same prescription space $\mathcal{B}_{t}$. The Dynamic program
in Theorem III when applied to problem P1a and P1c are as follows: Define
$V^{a}_{T+1}(x^{0}_{t+1},\pi^{1}_{T+1},\pi^{2}_{T+1})=0$ and
$V^{c}_{T+1}(x^{0}_{t+1},\pi^{1}_{T+1},\pi^{2}_{T+1})=0$ for every
$x^{0}_{t+1},\pi^{1}_{T+1},\pi^{2}_{T+1}$. For $t\leq T$ and for any
realization $x^{0}_{t},\pi^{1}_{t},\pi^{2}_{t}$ of
$X^{0}_{t},\Pi^{1}_{t},\Pi^{2}_{t}$, define

$\displaystyle
V^{a}_{t}(x^{0}_{t},\pi^{1}_{t},\pi^{2}_{t})=\min_{\gamma_{t}\in\mathcal{B}_{t}}\mathbb{E}[k_{t}(X_{t},U_{t})+V^{a}_{t+1}(\delta_{X^{0}_{t+1}},$
(57)
$\displaystyle\hat{\eta}^{1}_{t}(x^{0}_{t},U_{t}),\hat{\eta}^{2}_{t}(x^{0}_{t},U_{t}))|(X^{0}_{t},\Pi^{1}_{t},\Pi^{2}_{t},\Gamma_{t})=(x^{0}_{t},\pi^{1}_{t},\pi^{2}_{t},\gamma_{t})]$
$\displaystyle
V^{c}_{t}(x^{0}_{t},\pi^{1}_{t},\pi^{2}_{t})=\min_{\gamma_{t}\in\mathcal{B}_{t}}\mathbb{E}[k_{t}(X_{t},U_{t})+V^{c}_{t+1}(\delta_{X^{0}_{t+1}},$
(58)
$\displaystyle\hat{\eta}^{1}_{t}(x^{0}_{t},U_{t}),\hat{\eta}^{2}_{t}(x^{0}_{t},U_{t}))|(X^{0}_{t},\Pi^{1}_{t},\Pi^{2}_{t},\Gamma_{t})=(x^{0}_{t},\pi^{1}_{t},\pi^{2}_{t},\gamma_{t})]$

Using the above Corollary, we show that in both the problems value functions
are the same for any $\pi\in\delta(x)$. For $t\leq T$, let $V^{a}_{t}(\cdot)$
be the value function for the coordinator’s dynamic program in problem P1a and
$V^{c}_{t}(\cdot)$ be the value function for the coordinator’s dynamic program
in problem P1c. We show that $V^{a}_{t}(\pi)=V^{c}_{t}(\pi)$ for any
$\pi\in\delta(x)$.

###### Proof.

By definition value functions for both problems at time $T+1$ are same (equal
to $0$). At time $T$, the value functions evaluated at the same belief leads
to $(X_{T},U_{T})$ has a pair having the same probability distribution.
Therefore, the expectations are the same and the minimization is over
identical prescription space proves that the value functions are the same in
both the problems. So using the above argument inductively backwards in time.
At any time $t$, the expectation over the cost function are same in both
problems because $(X_{t},U_{t})$ has a pair have the same probability
distribution when evaluated over the same belief. Using Lemma E, the beliefs
at time $t+1$ in the two problems are the same and from the inductive step
value functions evaluated at same belief are equal. Hence the expectations are
the same and minimization over identical prescription space proves that the
value functions are same for both the problems.

The optimal cost in each problem is the expectation of value function at time
$t=1$ evaluated at prior belief and initial shared state $X^{0}_{1}$. Since
value functions are same for both the problems P1a and P1c, optimal costs are
equal.

##  Appendix F State dynamics influenced by an aggregate of agent actions
rather than individual actions.

For problem P1d Lemma A doesn’t hold true.

$\operatorname{{\mathds{P}}}^{(g^{1},g^{2})}(x^{1}_{t},x^{2}_{t}|c_{t})\neq\displaystyle\prod_{i=1}^{2}\operatorname{{\mathds{P}}}^{g^{i}}(x^{i}_{t}|c_{t}),$
(59)

The proof of the same is given below:

###### Proof.

$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{t},x^{2}_{t}|c_{t})=\operatorname{{\mathds{P}}}(x^{1}_{t},x^{2}_{t}|x^{0}_{1:t},a_{1:t-1})$
$\displaystyle=\frac{\sum_{x^{1,2}_{t-1}}\sum_{u_{t-1}}\operatorname{{\mathds{P}}}(x^{1,2}_{t},x^{0}_{t},a_{t-1},x^{1,2}_{t-1},u_{t-1}|x^{0}_{1:t-1},a_{1:t-2})}{\operatorname{{\mathds{P}}}(x^{0}_{t},a_{t-1}|x^{0}_{1:t-1},a_{1:t-2})}$
$\displaystyle=\frac{\sum_{x^{1,2}_{t-1}}\sum_{u_{t-1}}\operatorname{{\mathds{P}}}(x^{1,2}_{t},x^{0}_{t},a_{t-1},x^{1,2}_{t-1},u_{t-1}|x^{0}_{1:t-1},a_{1:t-2})}{\sum_{\tilde{x}^{1,2}_{t-1}}\sum_{\tilde{u}_{t-1}}\operatorname{{\mathds{P}}}(x^{0}_{t},a_{t-1},\tilde{x}^{1,2}_{t-1},\tilde{u}_{t-1}|x^{0}_{1:t-1},a_{1:t-2})}$

The numerator of the above equation can be written as:

$\displaystyle\sum_{x^{1,2}_{t-1}}\sum_{u_{t-1}}\operatorname{{\mathds{P}}}(x^{1}_{t}|x^{1}_{t-1},x^{0}_{t-1},a_{t-1})\operatorname{{\mathds{P}}}(x^{2}_{t}|x^{2}_{t-1},x^{0}_{t-1},a_{t-1})$
$\displaystyle\operatorname{{\mathds{P}}}(x^{0}_{t}|x^{0}_{t-1},a_{t-1})\operatorname{{\mathds{P}}}(a_{t-1}|u_{t-1})\operatorname{{\mathds{P}}}(u^{1}_{t-1}|x^{1}_{t-1},c_{t})$
$\displaystyle\operatorname{{\mathds{P}}}(u^{2}_{t-1}|x^{2}_{t-1},c_{t})\operatorname{{\mathds{P}}}(x^{1}_{t-1}|x^{0}_{1:t-1},a_{1:t-2})\operatorname{{\mathds{P}}}(x^{2}_{t-1}|x^{0}_{1:t-1},a_{1:t-2})$

Similarly the denominator can be written as:

$\displaystyle\sum_{\tilde{x}^{1,2}_{t-1}}\sum_{\tilde{u}_{t-1}}\operatorname{{\mathds{P}}}(x^{0}_{t}|x^{0}_{t-1},a_{t-1})\operatorname{{\mathds{P}}}(a_{t-1}|\tilde{u}_{t-1})\operatorname{{\mathds{P}}}(\tilde{u}^{1}_{t-1}|\tilde{x}^{1}_{t-1},c_{t})$
$\displaystyle\operatorname{{\mathds{P}}}(\tilde{u}^{2}_{t-1}|\tilde{x}^{2}_{t-1},c_{t})\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{t-1}|x^{0}_{1:t-1},a_{1:t-2})\operatorname{{\mathds{P}}}(\tilde{x}^{2}_{t-1}|x^{0}_{1:t-1},a_{1:t-2})$

The problem arises because of the term
$\operatorname{{\mathds{P}}}(a_{t-1}|u_{t-1})$ which does not factorize into a
separate agent’s action. Therefore, problem P1d Lemma A doesn’t hold true.

One example to prove Lemma A doesn’t hold true for problem P1d is: Consider a
scenario in which there is no shared state, the action space is denoted as
$\mathcal{U}=\\{0,1\\}$, and the state space is represented by
$\mathcal{X}=\\{0,1\\}$. Let $T=2$, and it is assumed that the local state of
agent $1$ remains stationary over time. The dynamics of agent $2$ are defined
by the transition probabilities
$\operatorname{{\mathds{P}}}(x^{2}_{t}=a|x^{2}_{t-1}=a)$ = 0.8, for
$a\in\\{0,1\\}$. The initial states $X^{1}_{1}$ and $X^{2}_{1}$ are
independent random variables with probability distribution Bernoulli (1/2).
The aggregate action $a_{t}$ is the sum of individual agents actions
$u^{1}_{t}$and $u^{2}_{t}$. Consider the strategy pair $(g,g)$ for Problem P1d
where $g_{t}$ (the strategy at time $t$) is:

$g_{t}(u^{i}_{t}=0|x^{i}_{t})=\begin{cases}1,\leavevmode\nobreak\
\text{if}\leavevmode\nobreak\ x^{i}_{t}=0\\\ 0,\leavevmode\nobreak\
\text{if}\leavevmode\nobreak\ x^{i}_{t}=1,\end{cases}$ (60)

To disprove Lemma 7, we first need to find the joint probability conditioned
on the common information. Here in this problem the common information
($C_{t}$) is the aggregate action $A_{t}$. The joint probability at time $t=2$
is computed under the assumption that the common information at time $t=2$ is
$a_{1}=u^{1}_{1}+u^{2}_{1}=1$. Specifically, we are interested in evaluating
the probability of the event $x^{1}_{2}=0,x^{2}_{2}=0$ conditioned on
$a_{1}=1$. Thus, the expression can be written as:

$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{2}_{2}=0|c_{2})=\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{2}_{2}=0|a_{1}=1)$
$\displaystyle=\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{2}_{2}=0,x^{1,2}_{1},u^{1,2}_{1}|a_{1}=1)$
$\displaystyle=\frac{\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{2}_{2}=0,x^{1,2}_{1},u^{1,2}_{1},a_{1}=1)}{\sum_{\tilde{x}^{1,2}_{1}}\sum_{\tilde{u}^{1,2}_{1}}\operatorname{{\mathds{P}}}(\tilde{x}^{1,2}_{1},\tilde{u}^{1,2}_{1},a_{1}=1)}$

Consider the numerator in the above equation,

$\displaystyle\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}$
$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{2}=0|x^{1}_{1})\operatorname{{\mathds{P}}}(x^{2}_{2}=0|x^{2}_{1})\operatorname{{\mathds{P}}}(a_{1}=1|u^{1}_{1},u^{2}_{1})$
$\displaystyle\times\operatorname{{\mathds{P}}}(u^{1}_{1}|x^{1}_{1})\operatorname{{\mathds{P}}}(u^{2}_{1}|x^{2}_{1})\operatorname{{\mathds{P}}}(x^{1}_{1},x^{2}_{1})$

and the denominator is,

$\displaystyle\sum_{\tilde{x}^{1,2}_{1}}\sum_{\tilde{u}^{1,2}_{1}}\operatorname{{\mathds{P}}}(a_{1}=1|\tilde{u}^{1}_{1},\tilde{u}^{2}_{1})\operatorname{{\mathds{P}}}(\tilde{u}^{1}_{1}|\tilde{x}^{1}_{1})\operatorname{{\mathds{P}}}(\tilde{u}^{2}_{1}|\tilde{x}^{2}_{1})\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{1},\tilde{x}^{2}_{1})$

The joint probability is evaluated to,

$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{2}_{2}=0|a_{1}=1)=\frac{0.2*0.25}{0.5}=0.1$
(61)

Now considering the marginal conditional probability,

$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{2}=0|a_{1}=1)=\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{1,2}_{1},u^{1,2}_{1}|a_{1}=1)$
$\displaystyle=\frac{\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{1,2}_{1},u^{1,2}_{1},a_{1}=1)}{\sum_{\tilde{x}^{1,2}_{1}}\sum_{\tilde{u}^{1,2}_{1}}\operatorname{{\mathds{P}}}(\tilde{x}^{1,2}_{1},\tilde{u}^{1,2}_{1},a_{1}=1)}$

The numerator of the above equation can be written as,

$\displaystyle\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}\operatorname{{\mathds{P}}}(x^{1}_{2}=0|x^{1}_{1})$
$\displaystyle\operatorname{{\mathds{P}}}(a_{1}=1|u^{1}_{1},u^{2}_{1})\operatorname{{\mathds{P}}}(u^{1}_{1}|x^{1}_{1})\operatorname{{\mathds{P}}}(u^{2}_{1}|x^{2}_{1})$
$\displaystyle\times\operatorname{{\mathds{P}}}(x^{1}_{1},x^{2}_{1})$

and the denominator is,

$\displaystyle\sum_{\tilde{x}^{1,2}_{1}}\sum_{\tilde{u}^{1,2}_{1}}\operatorname{{\mathds{P}}}(a_{1}=1|\tilde{u}^{1}_{1},\tilde{u}^{2}_{1})\operatorname{{\mathds{P}}}(\tilde{u}^{1}_{1}|\tilde{x}^{1}_{1})\operatorname{{\mathds{P}}}(\tilde{u}^{2}_{1}|\tilde{x}^{2}_{1})\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{1},\tilde{x}^{2}_{1})$

the marginal conditional probability is evaluated to,

$\displaystyle\operatorname{{\mathds{P}}}(x^{1}_{2}=0|a_{1}=1)=0.5$ (62)

Similarly evaluating the marginal conditional probability for agent $2$,

$\displaystyle\sum_{x^{1,2}_{1}}\sum_{u^{1,2}_{1}}\operatorname{{\mathds{P}}}(x^{2}_{2}=0|x^{2}_{1})\operatorname{{\mathds{P}}}(a_{1}=1|u^{1}_{1},u^{2}_{1})\operatorname{{\mathds{P}}}(u^{1}_{1}|x^{1}_{1})\operatorname{{\mathds{P}}}(u^{2}_{1}|x^{2}_{1})$
$\displaystyle\times\frac{\operatorname{{\mathds{P}}}(x^{1}_{1},x^{2}_{1})}{\sum_{\tilde{x}^{1,2}_{1}}\sum_{\tilde{u}^{1,2}_{1}}\operatorname{{\mathds{P}}}(a_{1}=1|\tilde{u}^{1}_{1},\tilde{u}^{2}_{1})\operatorname{{\mathds{P}}}(\tilde{u}^{1}_{1}|\tilde{x}^{1}_{1})\operatorname{{\mathds{P}}}(\tilde{u}^{2}_{1}|\tilde{x}^{2}_{1})\operatorname{{\mathds{P}}}(\tilde{x}^{1}_{1},\tilde{x}^{2}_{1})}$

which evaluates to,

$\displaystyle\operatorname{{\mathds{P}}}(x^{2}_{2}=0|a_{1}=1)=0.5$ (63)

Hence proved that,
$\operatorname{{\mathds{P}}}(x^{1}_{2}=0,x^{2}_{2}=0|a_{1})\neq\displaystyle\prod_{i=1}^{2}\operatorname{{\mathds{P}}}(x^{i}_{2}=0|a_{1}).$

In Problem P1d For any realization $x^{0}$ of the global state and any
realizations $x^{1}$ and $x^{2}$ of the agents’ private information,

$\pi_{t}(x^{0},x^{1},x^{2})=\delta_{X^{0}_{t}}{(x^{0})}\pi^{1,2}_{t}(x^{1},x^{2})$
(64)

For any coordination strategy $d$, the coordinator’s belief $\Pi_{t}$ in
problem P1d evolves almost surely as

$\Pi^{1,2}_{t+1}=\eta_{t}(X^{0}_{t},\Pi^{1,2}_{t},\Gamma_{t},A_{t}),$ (65)

where $\eta_{t}$ is a fixed transformation that does not depend on the
coordination strategy.

###### Proof.

The coordinator’s belief state can serve as a sufficient statistic for
selecting prescriptions. Let $c_{t+1}:=(x^{0}_{t+1},a_{1:t})$ be the
realization of common information $C_{t+1}$ and $\gamma_{1:t}$ be the
realization of the prescription $\Gamma_{1:t}$. The coordinator belief in
Problem P1d for all $x^{i}_{t+1}\in\mathcal{X}$ is given as:

$\displaystyle\pi^{1,2}_{t+1}(x^{1,2}_{t+1})=\operatorname{{\mathds{P}}}(x^{1,2}_{t+1}|c_{t+1}=(x^{0}_{t+1},a_{1:t}),\gamma_{1:t})$
$\displaystyle=\frac{\operatorname{{\mathds{P}}}(x^{1,2}_{t+1},a_{t}|x^{0}_{t},a_{1:t-1},\gamma_{1:t})}{\operatorname{{\mathds{P}}}(a_{t}|x^{0}_{t},a_{1:t-1},\gamma_{1:t})}$
$\displaystyle=\frac{\sum_{x^{1,2}_{t}}\sum_{u_{t}}\operatorname{{\mathds{P}}}(x^{1,2}_{t+1},a_{t},x^{1,2}_{t},u_{t}|x^{0}_{t},a_{1:t-1},\gamma_{1:t})}{\sum_{\tilde{x}^{1,2}_{t}}\sum_{\tilde{u}_{t}}\operatorname{{\mathds{P}}}(a_{t},\tilde{x}^{1,2}_{t},\tilde{u}_{t}|x^{0}_{t},a_{1:t-1},\gamma_{1:t})}$

The numerator of the above equation can be written as:

$\displaystyle\sum_{x^{1,2}_{t}}\sum_{u_{t}}\operatorname{{\mathds{P}}}(x^{1}_{t+1}|x^{1}_{t},x^{0}_{t},a_{t})\operatorname{{\mathds{P}}}(x^{2}_{t+1}|x^{1}_{t},x^{0}_{t},a_{t})\operatorname{{\mathds{P}}}(a_{t}|u_{t})$
$\displaystyle\times\gamma(x^{1}_{t};u^{1}_{t})\gamma(x^{2}_{t};u^{2}_{t})\pi^{1,2}_{t}(x_{t})$

Similarly, the denominator can be written as:

$\displaystyle\sum_{x^{1,2}_{t}}\sum_{u_{t}}\operatorname{{\mathds{P}}}(a_{t}|u_{t})\gamma(x^{1}_{t};u^{1}_{t})\gamma(x^{2}_{t};u^{2}_{t})\pi^{1,2}_{t}(x_{t})$

Thus $\pi_{t+1}(\cdot)$ is determined by
$x^{0}_{t},\pi^{1,2}_{t},\gamma_{t},a_{t}$. We denote the update rule
described above with $\eta_{t}$.


11institutetext:  Department of Physics, College of Science, Qassim
University, P.O. Box 6644, Buraidah 51452, Saudi Arabia

Quantum mechanics Foundations of Quantum mechanics Quantum Formalism
Phenomenological theories

# Quantum mechanics with quaternionic mass

A. I<EMAIL_ADDRESS>
(today; today)

###### Abstract

Quantum mechanics with quaternionic mass is considered. The momentum eigen-
value equation with quaternionic mass yields the Klein-Gordon equation with a
mass consisting of longitudinal and traverse masses. The scalar field total
mass is found to be a sum of these masses. This field appears to be connected
with two subfields conserving linear momentum. It is found that a particle
with real mass satisfies the quantum Telegraph equation, whereas that one with
quaternionic mass satisfies the Klein-Gordon equation. A quantum force acting
on the particle is found to be proportional to its velocity. When the particle
field is coupled to an electromagnetic field, an additional term in the
particle’s energy appears reflecting the interaction of the particle’s angular
momentum with the magnetic field.

###### pacs:

03.65.-w

###### pacs:

03.65.Ta

###### pacs:

03.65.Ca

###### pacs:

74.20.De

##  1 Introduction

Maxwell, Planck, de Broglie, and Einstein have laid down the foundational
principles of the elementary particles and their fields [1]. In Maxwell’s
theory, the electric and magnetic fields are treated as one entity if the
theory is invariant under the duality transformation. Under such
circumstances, Maxwell equations are reduced to two equations. Einstein states
that energy and mass are equivalent so that each one can be transformed into
another. Lorentz had found that space and time are one entity. They can be
transformed into each other by employing Lorentz transformations. Since matter
is inscribed into mass and energy into a wave, then matter and wave are
equivalent too. This is normally coined in the wave-particle duality. In
quaternionic jargon, the quantum equation reflecting the wave-particle duality
is the momentum eigen-value equation. This yields a new quantum wave equation
exhibiting this duality aspect [2, 3].

The mathematical formalism in which the above principles are realized is via
equations of motion. At their highest level, these are the Maxwell, Klein-
Gordon, and Dirac equations. Maxwell equations describe pure wave phenomena,
and Klein-Gordon and Dirac describe the particle phenomena. However an
equation that describes the evolutionary state is found in [2, 3]. The two
forms of existence are described by fields. Any change in this field will be
exhibited as a wave. Besides the dynamical evolution of a quantum object, an
aggregation of particles in an ensemble is found to be governed by specific
statical laws. Further, it is found that the dynamics and statistics are
connected. Fermions that are governed by the Dirac equation follow different
statistics than those of bosons which are described by the Klein-Gordon
equation. Besides, it is also found that Maxwell equations govern another kind
of bosons with different statistics. In effect, all particles find their way
to be described by appropriate dynamical as well as statistical equations.

Objects like molecules (atoms) follow other statistics known as the Maxwell-
Boltzmann statistics. Therefore, the physical properties of microparticles are
intimately connected. It is well known that all objects (particles) have
inertia expressing the extent to which they resist a change of state (motion
to rest or rest to motion). Inertia can be modeled as some kind of friction.
In quantum mechanics, a quantum particle is modeled by a wave packet in such a
way that its group velocity is equal to the particle velocity. However, the
phase velocity of a single wave is different from the particle and group
velocities. In our anticipated description, we think of a particle and its
wave as two manifestations of one object. When the particle behaves as a wave
its momentum is equal to that when it behaves like a normal particle. The wave
has a vector mass whereas the particle has a scalar mass. The two entities are
connected, however.

Space can not be compatible with the notion of inertia, however. A fluid
(ether) filling the whole space was promoted by physicists but later rejected
by the famous Michelson–Morley experiment. The experiment did not rule out the
presence of the fluid but rather excluded its existence to carry the
electromagnetic wave. The electromagnetic wave was thought of as a mechanical
wave that requires a priori a medium to propagate in. The modern understating
of the space, today, allows some kind of minimal field to be pervading the
whole space. This expresses a background due to quantum fluctuations in the
field. It is called the _vacuum_ that today bears a physical existence.

Mass is a property that distinguishes an object. The wave inertia however is
not characterized by its mass. Therefore, the notion of mass has to be
extended to incorporate particles as well as waves. There exists now a need to
distinguish different kinds of mass that could be due to the field of force
under which the object interacts. With the same token, an object can have two
kinds of charge too playing analogous roles. One mass (charge) represents the
matter nature while the other one characterizes the wave nature. A mass
besides being the source of matter-wave could also act as a source for the
electromagnetic field.

We have found that Quaternions are the appropriate mathematical tools that can
deal with all of the above identification of the object. This is owing to the
spectacular algebra of Quaternions. Treating the particle mass as quaternion
yields very interesting results. Since a quaternion consists of scalar and
vector components, a quaternionic mass embodies the particle (scalar) and wave
(vector) aspects. In the de Broglie theory, a photon wave, with zero mass, is
used to draw a parallel with that one having a mass.

In the present formalism, we however employ a different prescription. We would
like to explore these possibilities by applying the quaternionic mass in Dirac
formalism to manifest this. This recipe turned out to be equivalent to
introducing an interaction of the quantum particle with an external field.
Previously, the normal mechanism to do so is via the minimal coupling method.

This paper is scheduled as follows: we introduce in Section 2 quaternionic
quantum mechanics with real and quaternionic mass. In this section, we derived
the energy conservation (continuity) equation of the quantum system. We then
showed that the duality aspect is guaranteed by deriving a particle-wave
momentum equation connecting the wave momentum to that of the particle. This
relationship reveals that a quaternionic mass representation of a quantum
particle realizes the duality principle due to de Broglie. While the ordinary
quaternionic momentum eigen-value equation with real mass yields a Telegraph
equation, that one with quaternionic mass yields Klein-Gordon equation for a
field whose mass consists of two masses. In Section 3 we study the interaction
of a quantum particle with an electromagnetic field. An interaction term
showing the coupling of a particle’s angular momentum with the magnetic field
appears in the particle’s Hamiltonian. A Lorentz-like force deemed to govern
the quantum particle motion is presented in Section 4. We end our paper by
concluding remarks in Section 5.

##  2 Quantum mechanics with quaternionic mass

The Dirac (momentum eigen-value) equation with real mass, $m$, was studied
before yielding [2, 3]

$\frac{1}{c^{2}}\,\frac{\partial\psi_{0}}{\partial
t}+\vec{\nabla}\cdot\vec{\psi}+\frac{m}{\hbar}\,\psi_{0}=0\,,$ (1)
$\frac{\partial\vec{\psi}}{\partial
t}+\vec{\nabla}\psi_{0}+\frac{mc^{2}}{\hbar}\,\vec{\psi}=0\,\,,$ (2)
$\vec{\nabla}\times\vec{\psi}=0\,.$ (3)

Equations (1) - (3) yield the wave equations

$\frac{1}{c^{2}}\frac{\partial^{2}\psi_{0}}{\partial
t^{2}}-\nabla^{2}\psi_{0}+\frac{2m}{\hbar}\frac{\partial\psi_{0}}{\partial
t}+\left(\frac{mc}{\hbar}\right)^{2}\psi_{0}=0\,,\qquad\frac{1}{c^{2}}\frac{\partial^{2}\vec{\psi}}{\partial
t^{2}}-\nabla^{2}\vec{\psi}+\frac{2m}{\hbar}\frac{\partial\vec{\psi}}{\partial
t}+\left(\frac{mc}{\hbar}\right)^{2}\vec{\psi}=0\,.$ (4)

These are the undistorted Telegraph equations that are known to describe the
propagation of the electric current and voltage in transmission lines.
Interestingly, the inertial (quantum) and electromagnetic waves of the
electron are coherent. The solution of the above quantum Telegraph equation is
a wavepacket consisting of waves traveling to the left and right with the
speed of light. This is an advantage of the quantum Telegraph equation over
the Schrodinger equation where the solution of the latter is a plane wave. For
the Schrodinger equation to describe the motion of a particle two plane waves
have to be added to obtain a wavepacket solution.

We would like here to entertain the possibility of studying the momentum
eigen-value equation with quaternionic mass. To this aim let us write the
quaternionic momentum eigen-value generalizing the ordinary one. It is
expressed in the form [2, 3]

$\tilde{P}\,\tilde{\Psi}^{*}=c\tilde{M}\,\tilde{\Psi}^{*}\,,\qquad\tilde{P}=\left(\frac{i}{c}\,\hat{E}\,,\hat{\vec{p}}\,\right)\,,\qquad\tilde{\Psi}=\left(\frac{i}{c}\,\psi_{0}\,,\vec{\psi}\,\right)\,,\qquad\tilde{M}=\left(im_{\ell}\,,\vec{m}_{t}\,\right)\,$
(5)

Here, $\hat{E}$ and $\hat{\vec{p}}$ are the energy and momentum operators. The
two masses $m_{\ell}$ and $\vec{m}_{t}$ are the longitudinal and transverse
masses describing the wave-particle duality. Here $\tilde{\Psi}$ describes the
state of the quantum particle consisting of a vector and a scalar part that is
analogous to the scalar and vector potentials describing the electromagnetic
field. The quaternion mass, $\tilde{M}$, can be thought of as a mass field.
The particle interacts with the outside world by coupling this mass field with
the other fields in which the particle exists. The physical meaning of the
masses, $\vec{m}_{t}$ and $m_{\ell}$ will be given later on.

Applying the quaternionic product rule, and equating the real and imaginary
parts in the two sides of the resulting equations, one finds [4]

$\frac{1}{c^{2}}\,\frac{\partial\psi_{0}}{\partial
t}+\vec{\nabla}\cdot\vec{\psi}=0\,,$ (6) $\frac{\partial\vec{\psi}}{\partial
t}+\vec{\nabla}\psi_{0}+\frac{c^{2}}{\hbar}\,\vec{m}_{t}\times\vec{\psi}=0\,\,,$
(7)
$\vec{\nabla}\times\vec{\psi}+\hbar^{-1}(m_{\ell}c\,\vec{\psi}-\,\vec{m}_{t}\psi_{0})=0\,,$
(8)

and

$m_{\ell}\psi_{0}-c\,\vec{m}_{t}\cdot\vec{\psi}=0\,.$ (9)

Now equations (7) and (9) yield

$\frac{\partial\rho}{\partial
t}+\vec{\nabla}\cdot\vec{J}=0\,,\qquad\vec{J}=c\,\vec{m}\psi_{0}\,,\qquad\rho=m_{\ell}\psi_{0}\,.$
(10)

Thus if $\vec{J}=\rho\,\vec{v}$, then

$m_{\ell}\vec{v}=c\,\vec{m}_{t}\,.$ (11)

Equation (11) can be seen as momentum conservation of the two masses
representing the quantum particle. Since in de Broglie’s theory a particle has
a wave nature, the term $c\,\vec{m}_{t}$ in eq.(11) could account for the
momentum of the wave associated with the particle. If we say that the matter-
wave momentum is $\vec{p}_{W}=c\vec{m}_{t}$, then using Einstein’s mass-energy
relationship, $E=mc^{2}$, one finds the value of the matter-wave momentum to
be $p_{W}=E/c$ . This agrees with the standard representation of the momentum
of light (photon). The vector mass, $\vec{m}_{t}$, points along the direction
in which matter-wave travels. However, in the standard quantum mechanics, a
particle is represented by a wavepacket that travels at a velocity (group
velocity) equals to the particle’s velocity.

Recall that when a charged particle interacts with an electromagnetic field,
the field imparts momentum $\vec{p}_{A}=q\vec{A}$ such that the total momentum
the charge receives is $\vec{p}\,^{\prime}=\vec{p}+q\vec{A}$. This allows us
to deduce that the momentum provided to the particle is
$\vec{p}_{f}=c\,\vec{m}_{t}$. When a charged particle interacts with an
electromagnetic field then, $q\vec{A}=c\,\vec{m}_{t}$. Hence, the transverse
(vector) mass describes the momentum that a particle receives while moving. If
the particle moves freely, $\vec{m}_{t}=0$. This urges us to define $m_{\ell}$
as the mass of the free particle. It is thus very intriguing to encapsulate
the two masses in a single representation, the quaternionic mass.

Now eqs.(6) and (7) yield

$\frac{\partial u}{\partial
t}+\vec{\nabla}\cdot\vec{S}=0\,,\qquad\qquad\vec{S}=\psi_{0}\vec{\psi}\,,\qquad\qquad
u=\frac{\psi^{2}}{2}+\frac{\psi_{0}^{2}}{2c^{2}}\,,$ (12)

which represents the energy conservation equation of the system. Here $u$
represents the energy density of the particle field and $\vec{S}$ is its
energy flux. The quantum particle/de Broglie wave travels/propagates in space
is like the fluid flow. It was Maxwell who thought of electromagnetic field
(wave) propagation as fluid flow. It is interesting to observe that the
introduction of the quaternionic mass removes the dissipation in the system
that is associated with eqs.(1) - (3).

Multiply eq.(7) by $\psi_{0}$ and employ eqs.(6) and (8) to get the momentum
(force) equation

$\frac{1}{c^{2}}\,\frac{\partial\vec{S}}{\partial
t}+(\vec{\psi}\vec{\nabla}\cdot\vec{\psi}+\vec{\psi}\cdot\vec{\nabla}\vec{\psi})+\vec{\nabla}\left(\frac{\psi_{0}^{2}}{2c^{2}}-\frac{\psi^{2}}{2}\right)=0\,,$
(13)

In components form, eq.(13) can be expressed as

$\frac{1}{c^{2}}\,\frac{\partial S_{i}}{\partial
t}+\partial_{j}(\psi_{i}\psi_{j})+\partial_{i}\left(\frac{\psi_{0}^{2}}{2c^{2}}-\frac{\psi^{2}}{2}\right)=0\,,$
(14)

which can be written as

$\frac{1}{c^{2}}\,\frac{\partial S_{i}}{\partial
t}+\partial_{j}(\psi_{i}\psi_{j}+\delta_{ij}(\frac{\psi_{0}^{2}}{2c^{2}}-\frac{\psi^{2}}{2}))=0\,,\qquad-\frac{\partial
g_{i}}{\partial t}+\partial_{j}\sigma_{ij}=0\,,$ (15)

where

$\sigma_{ij}=-\psi_{i}\psi_{j}-\delta_{ij}\left(\frac{\psi_{0}^{2}}{2c^{2}}-\frac{\psi^{2}}{2}\right)\,,$
(16)

is a symmetric stress-tensor of the matter field. Interestingly, eq.(15) is
the momentum conservation equation that is analogous to the electromagnetic,
where the momentum of the matter wave is defined by
$\vec{g}=\frac{\vec{S}}{c^{2}}$. Thus, eq.(15) describes the force density
that the matter wave impinges. It is similar to Euler’s equation of fluid
motion. This indicates that the motion of a quantum particle is analogous to
fluid motion.

Eqs.(6) - (9) can be shown to represent a wave equation (matter wave) [4]

$\frac{1}{c^{2}}\,\frac{\partial^{2}\psi_{0}}{\partial
t^{2}}-\nabla^{2}\psi_{0}+\left(\frac{Mc}{\hbar}\right)^{2}\psi_{0}=0,\qquad\frac{1}{c^{2}}\,\frac{\partial^{2}\vec{\psi}}{\partial
t^{2}}-\nabla^{2}\vec{\psi}+\left(\frac{Mc}{\hbar}\right)^{2}\vec{\psi}=0\,,$
(17)

where

$M^{2}=m_{t}^{2}-m_{\ell}^{2}\,.$ (18)

Equation (17) is but the Klein-Gordon equation of a scalar field whose
effective mass is $M$ which I prefer to call the invariant mass. It is thus
interesting that the momentum eigen-value equation with a quaternion mass
gives the Klein-Gordon equation. It seems that the Klein-Gordon field is a
superposition of two kinds of fields. A tachyonic Klein-Gordon equation
results when $m_{t}<|m_{\ell}|$. As evident from eq.(18), if the scalar field
has the same mass components, it then behaves like a massless field (photon).
It is quite remarkable to see that a particle described by a real mass
satisfies the quantum Telegraph equation while that one described by a
quaternionic mass yields the Klein-Gordon equation. Note that the Dirac
equation can be obtained from the quantum Telegraph equation by taking the
mass to be imaginary [2].

The quaternionic momentum eigen-value equation associated with real mass was
found to lead to the Telegraph equation (or a dissipative Klein-Gordon
equation) [5]. The dissipative term is removed when quaternionic mass is
proposed instead.

Applying eq.(11) in eq.(18) yields

$m_{\ell}=\frac{M}{\sqrt{1-\frac{v^{2}}{c^{2}}}}\,,\qquad\qquad\vec{m}_{t}=\left(\frac{\vec{v}}{c}\,\right)m_{\ell}\,.$
(19)

The longitudinal mass $m_{\ell}$ is but the relativistic mass and $M$ is the
field rest-mass. Notice that in general the value of the transverse mass is
smaller than the longitudinal one. We could now say that the transverse mass
is the _active mass_ whereas the longitudinal mass is a _passive mass_ of the
particle. When the particle is not interacting its transverse vanishes and its
longitudinal mass remains. This new concept of masses should find application
in scattering problems of microscopic particles having particle and wave
natures. In such cases, the conservation of energy and momentum should be
taken in the association of a quaternionic perspective.

Interestingly, Eq.(19) associates inertia with motion. In the ordinary
Newton’s laws, the inertia of a macroscopic object is connected with its
acceleration. However, in quantum mechanics the force is directly probational
to velocity [6]. The second equation in eq.(19) can be expressed as the
Einstein’s relativistic equation between velocity, momentum and energy that is
valid for matter as well as waves, $\vec{v}=(c^{2}/E)\,\vec{p}$, upon
expressing it as $\vec{v}=[c^{2}/(m_{\ell}c^{2})]\vec{m}_{t}c$ , with
$E=m_{\ell}c^{2}$ and $\vec{p}=\vec{m}_{t}c$ . The latter momentum can be seen
as a momentum of the wave-packet mimicking the particle. Hence, this analogy
provides a faithful realization of our theory. Thus, for particle interactions
the momentum and energy conservations are conservations of the vector mass and
relativistic masses, $\vec{m}_{t}$ and $m_{\ell}$ , respectively. The relation
in eq.(19) was already derived in the context of a different eigen-value
equation of a quantum particle [7].

Defining $\rho=\psi^{2}_{0}/c^{2}$ and $\vec{J}=\vec{\psi}$, equations (6) and
(7) can be seen as the continuity and momentum conservation equations
governing matter flow. It is valid for mass as well as charge flow. It
therefore fully characterizes the moving quantum particle.

##  3 Interacting quantum particle with an electromagnetic field

The interaction of a charged ($q$) quantum particle with an electromagnetic
field can be found by using the minimal coupling recipe, where the particle
momentum and energy are replaced, respectively, by,
$\vec{p}\rightarrow\vec{p}-q\vec{A}$ and $E\rightarrow E-q\varphi$ [1]. This
is found to alter eqs.(1) - (3) to read [8]

$\frac{1}{c^{2}}\,\frac{\partial\psi_{0}}{\partial
t}+\vec{\nabla}\cdot\vec{\psi}+\frac{m}{\hbar}\,\psi_{0}=0\,,$ (20)
$\frac{\partial\vec{\psi}}{\partial
t}+\vec{\nabla}\psi_{0}+\frac{mc^{2}}{\hbar}\,\vec{\psi}+\frac{cq}{\hbar}\,\vec{A}\times\vec{\psi}=0\,\,,$
(21)
$\vec{\nabla}\times\vec{\psi}+\hbar^{-1}(c^{-1}q\varphi\,\vec{\psi}-c^{-1}q\vec{A}\psi_{0})=0\,,$
(22)

and

$c^{-2}q\varphi\,\psi_{0}-q\vec{A}\cdot\vec{\psi}=0\,,$ (23)

where $\varphi$ and $\vec{A}$ are the scalar and vector potentials of the
electromagnetic field. An interesting analogy between eqs.(20) - (23) and
eqs.(6) - (9) can be drawn. This is as follows: $m=0$,
$c\,\vec{m}_{t}\rightarrow q\vec{A}$, and $m_{\ell}c^{2}\rightarrow
q\,\varphi$. Therefore, introducing a quaternionic mass in a theory is
equivalent to dealing with interaction of a particle with an external field.

Equations (20) - (23) reveal that $\psi_{0}$ satisfies the Telegraph’s wave
equation

$\frac{1}{c^{2}}\frac{\partial^{2}\psi_{0}}{\partial
t^{2}}-\nabla^{2}\psi_{0}+\frac{2m}{\hbar}\frac{\partial\psi_{0}}{\partial
t}+\left(\frac{m^{2}c^{2}}{\hbar^{2}}+\frac{q^{2}}{\hbar^{2}}(A^{2}-\frac{\varphi^{2}}{c^{2}})\right)\psi_{0}=\frac{cq}{\hbar}\vec{\psi}\cdot\vec{B}\,,$
(24)

and

$\frac{1}{c^{2}}\frac{\partial^{2}\vec{\psi}}{\partial
t^{2}}-\nabla^{2}\vec{\psi}+\frac{2m}{\hbar}\frac{\partial\vec{\psi}}{\partial
t}+\left(\frac{m^{2}c^{2}}{\hbar^{2}}+\frac{q^{2}}{\hbar^{2}}(A^{2}-\frac{\varphi^{2}}{c^{2}})\right)\vec{\psi}=\frac{q}{c\hbar}(\vec{E}\times\vec{\psi}+\psi_{0}\vec{B})\,,$
(25)

with variable mass and source terms that couple to the magnetic field. It is
intriguing that the above two equations are coupled to each other. The right-
hand sides of eqs.(24) and (25) show that the electromagnetic field couple to
the particle’s fields (wavefunctions), $\psi_{0}$ and $\vec{\psi}$ . For
homogenous wave equations, eqs.(24) and (25) imply that

$\vec{\psi}\cdot\vec{B}=0\,,\qquad\qquad\vec{B}=\frac{\vec{\psi}}{\psi_{0}}\times\vec{E}\,.$
(24a)

These show that the electromagnetic field is perpendicular to the particle’s
vector wavefunction, $\vec{\psi}$ . Equation (24a) reduces to the
electrodynamics of a moving charge for, $\vec{v}\,\psi_{0}=c^{2}\vec{\psi}$ .
With the conditions in eq.(24a), eqs.(24) and (25) decouple.

It would be interesting if we connect the vector wavefunction to the angular
momentum, $\vec{L}$, of the particle by

$c\hbar\,\vec{\psi}=\vec{L}\psi_{0}\,,$ (26)

so that the right-hand side of eq.(25) will describe a particle’s orbital
interaction with the magnetic field of the form

$\frac{1}{c^{2}}\frac{\partial^{2}\psi_{0}}{\partial
t^{2}}-\nabla^{2}\psi_{0}+\frac{2m}{\hbar}\frac{\partial\psi_{0}}{\partial
t}+\left(\frac{m^{2}c^{2}}{\hbar^{2}}+\frac{q^{2}A^{2}}{\hbar^{2}}-\frac{q^{2}\varphi^{2}}{\hbar^{2}c^{2}}-\frac{q}{\hbar^{2}}\vec{L}\cdot\vec{B}\right)\psi_{0}=0\,.$
(27)

Equation (27) can be expressed in an energy equation as

$\frac{\hbar^{2}}{2mc^{2}}\frac{\partial^{2}\psi_{0}}{\partial
t^{2}}-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{0}+\hbar\frac{\partial\psi_{0}}{\partial
t}+\left(\frac{mc^{2}}{2}+\frac{q^{2}A^{2}}{2m}-\frac{q^{2}\varphi^{2}}{2mc^{2}}-\frac{q}{2m}\vec{L}\cdot\vec{B}\right)\psi_{0}=0\,.$
(28)

Under the rotation of space and time coordinates by an angle of $-\pi/2$,
_i.e.,_ ($t\rightarrow-it$ and $r\rightarrow-ir$) [9], eq.(28) becomes

$i\hbar\frac{\partial\psi_{0}}{\partial
t}=\frac{\hbar^{2}}{2mc^{2}}\frac{\partial^{2}\psi_{0}}{\partial
t^{2}}-\frac{\hbar^{2}}{2m}\nabla^{2}\psi_{0}-\left(\frac{mc^{2}}{2}+\frac{q^{2}A^{2}}{2m}-\frac{q^{2}\varphi^{2}}{2mc^{2}}-\frac{q}{2m}\vec{L}\cdot\vec{B}\right)\psi_{0}\,.$
(28a)

The last term in the left-hand side of eq.(28a) embodies the interaction of
the particle’s angular momentum with the external magnetic field. The first
term in eq.(28a) will have a small contribution in the low-frequency limit.
For such a case eq.(28a) approximates to Schrodinger equation. If we chose in
eq.(26) the spin angular momentum instead of the orbital angular momentum,
then $c\hbar\,\vec{\psi}=2\vec{S}\psi_{0}$, where $2\vec{S}=\vec{\sigma}$ are
Pauli’s matrices, then the energy term in eq.(28a) will describe the
particle’s spin angular momentum. Notice that the vector wavefunction
equation, eq.(25), doesn’t involve such a term.

For massive electrodynamics, one multiplies eq.(25) by
$\frac{\varepsilon_{0}c\hbar}{q}$ and choose, $\vec{\psi}=\vec{A}$ and
$\psi_{0}=\varphi$, so that eq.(25) becomes

$\frac{1}{c^{2}}\frac{\partial^{2}\vec{Y}}{\partial
t^{2}}-\nabla^{2}\vec{Y}+\frac{2m}{\hbar}\frac{\partial\vec{Y}}{\partial
t}+\left(\frac{m^{2}c^{2}}{\hbar^{2}}+\frac{q^{2}}{\hbar^{2}}(A^{2}-\frac{\varphi^{2}}{c^{2}})\right)\vec{Y}=\vec{s}\,,$
(29)

where

$\vec{s}=\varepsilon_{0}(\vec{E}\times\vec{A}+\varphi\vec{B})\,,\qquad\qquad\vec{Y}=\frac{\varepsilon_{0}c\hbar}{q}\vec{A}\,.$
(30)

The quantity, $\vec{s}$, is the spin density of the electromagnetic field and
the field $\vec{Y}$ is a linear momentum vector field [10]. This later
momentum is new to electrodynamics. Equation (29) shows that the spin
$\vec{s}$ is the source for the momentum vector field, $\vec{Y}$ . Equation
(27) will yield a nonlinear Telegraph’s equation

$\frac{1}{c^{2}}\frac{\partial^{2}\varphi}{\partial
t^{2}}-\nabla^{2}\varphi+\frac{2m}{\hbar}\frac{\partial\varphi}{\partial
t}+\left(\frac{m^{2}c^{2}}{\hbar^{2}}+\frac{q^{2}A^{2}}{\hbar^{2}}-\frac{q}{\hbar^{2}}\vec{L}\cdot\vec{B}\right)\varphi-\frac{q^{2}}{\hbar^{2}c^{2}}\,\varphi^{3}=0\,.$
(31)

Equation (31) can be expressed as an energy equation as

$\frac{\hbar^{2}}{2mc^{2}}\frac{\partial^{2}\varphi}{\partial
t^{2}}-\frac{\hbar^{2}}{2m}\nabla^{2}\varphi+\hbar\,\frac{\partial\varphi}{\partial
t}+\left(\frac{mc^{2}}{2}+\frac{q^{2}A^{2}}{2m}-\frac{q}{2m}\vec{L}\cdot\vec{B}\right)\varphi-\frac{q^{2}}{2mc^{2}}\,\varphi^{3}=0\,.$
(32)

Upon space and time rotation by $-\pi/2$ ($t\rightarrow-it$ and $r\rightarrow-
ir$) [9], eq.(32) yields

$i\hbar\,\frac{\partial\varphi}{\partial
t}=\frac{\hbar^{2}}{2mc^{2}}\frac{\partial^{2}\varphi}{\partial
t^{2}}-\frac{\hbar^{2}}{2m}\nabla^{2}\varphi-\left(\frac{mc^{2}}{2}+\frac{q^{2}A^{2}}{2m}-\frac{q}{2m}\vec{L}\cdot\vec{B}\right)\varphi+\frac{q^{2}}{2mc^{2}}\,\varphi^{3}\,.$
(33)

Equation (33) can be seen as a late-time nonlinear Schrodinger equation of a
charged particle interacting with an electromagnetic field with $\varphi$ as a
wavefunction. The fifth term on the left-hand side of eq.(33) would represent
the interaction orbital angular momentum with the magnetic field, where the
particle spin $\vec{S}=0$ .

##  4 Extended Lorentz force

The Lorentz force describing the force acting of the quantum particle can be
extended to read

$\tilde{F}=c\tilde{M}^{*}\tilde{V}\tilde{\nabla}\tilde{\Psi}=\left(\frac{i}{c}\,P\,,\vec{f}\right),\qquad\qquad\tilde{V}=(ic\,,\vec{v})\,,$
(34)

where $\tilde{\Psi}$ is considered to be dimensionless. Using the quaternion
product rule mentioned above and equating the real and imaginary components in
the two sides of the resulting equation, yield

$\vec{f}=-M\frac{\psi_{0}}{c}\,\frac{\vec{v}}{\tau}+\frac{M^{2}c^{3}}{\hbar}\,\vec{\psi}\,,\,\,P=\frac{Mc}{\tau}\,(\psi_{0}-\vec{v}\cdot\vec{\psi})\,,\,\,\vec{m}_{t}\cdot\vec{v}=m_{\ell}c\,,\,\,\tau=\frac{\hbar}{Mc^{2}}\,,$
(35)

upon employing eqs.(6) - (9). The first term in the force is analogous to the
electric force and the second one to the friction force that occurs in a
conductor due to scattering between electrons. It is also like a viscous force
that requires the presence of a fluid that permeates the whole space. The time
$\tau$ is a characteristic time reflecting the jittery (trembling) motion of a
quantum particle as observed by Schrodinger for the electron free-motion [11].
It seems a quantum particle jitters because it experiences a quantum (viscous)
force as it travels in space. The jitter frequency here is twice that due to
the electron in the Dirac theory. The quantum resistive force may explain the
origin of inertia that any particle experiences as it starts to move.

##  5 Concluding remarks

We have extended in this work our earlier work on quantum mechanics where the
mass is treated as a quaternion. The quaternionic momentum eigen-value
equation leads to a Telegraph equation when a real mass is used, it however
leads to the Klein-Gordon equation when a quaternionic mass is used instead.
This scalar field has an effective mass that is a combination of its two
masses. The vector and scalar masses connect the particle matter-wave duality
where the wave’s momentum is equal to the particle’s momentum. This is
expressed by the equation, $\vec{m}\,c=m_{0}\vec{v}$, where $\vec{v}$ is the
particle’s velocity. When the quantum particle interacts with an
electromagnetic field, the particle’s angular momentum is coupled to the
magnetic field. A quantum frictional force acting on the moving particle is
found to be proportional to its velocity. The interaction of the quantum
particle with the electromagnetic field can be obtained from that of a free
particle with quaternionic mass provided one defines $c\,\vec{m}_{t}\equiv
q\vec{A}$ and $m_{\ell}c^{2}\equiv q\varphi$. A new linear momentum field is
found to be associated with the spin of the electromagnetic field. A charged
particle interacting with an electromagnetic field is found to be governed by
a Schrodinger-like equation where the particle spin (orbital) angular momentum
is coupled to the electromagnetic field.

## References

  * [1] Bjorken, J. D., Drell, S. D., _Relativistic Quantum Fields_ , McGraw-Hill College; First Edition edition (1965).
  * [2] Arbab, A. I., _Derivation of Dirac, Klein-Gordon, Schrodinger, Diffusion and quantum heat transport equations from a universal quantum wave equation_ , EPL 92, 40001 (2010).
  * [3] Arbab, A. I., _The Quaternionic Quantum Mechanics_ , Applied Physics Research, Vol.3, No. 2, 160 (2011).
  * [4] Arbab, A. I., Widatallah, H. M., and Khalafalla, M. A. H., _Toward a Complete Wave-Particle Duality: Do Matter WavesHave Inertia?_ , Advances in Natural Science, Vol.5, No. 3, 32 (2012).
  * [5] Arbab, A. I., _Quantum Telegraph equation: New matter-wave equation_ , Optik 140, 1010 (2017).
  * [6] Arbab, A. I., _Bohmian quantum mechanics revisited_ , Inter. J. Quant. Found. 4, 235 (2018).
  * [7] Arbab, A. I., Widatallah, H. M., Khalafalla, M. A. H., _Toward a Complete Wave-Particle Duality: Do Matter Waves Have Inertia?_ , Advances in Natural Science, 5, 32 (2012).
  * [8] Arbab, A. I., _Coupling of a biquaternionic Dirac field to a bosonic field_ , Theor. Math. Phys. 203, 631 (2020).
  * [9] Arbab, A. I. and Widatallah, H. M., _The mass-extended ’t Hooft-Nobbenhuis complex transformations and their consequences_ , Europhys. Lett. 92, 23002 (2010).
  * [10] Griffiths, D. J., _Introduction to Electrodynamics_ , Cambridge University Press (2017).
  * [11] Schrodinger, E., _Uber die Kraftfreie Bewegung in der Relativistischen Quantenmechanik,_ Sitzungb. Preuss. Akad. Wiss. Phys.-Math. Kl. 24, 418 (1930).