# On the rank of $\pi_{1}(\textup{Ham})$

Andrés Pedroza  Facultad de Ciencias
Universidad de Colima
Bernal Díaz del Castillo No. 340
Colima, Col., Mexico 28045<EMAIL_ADDRESS>
###### Abstract.

We show that for any positive integer $k$ there exists a closed symplectic
$4$-manifold, such that the rank of the fundamental group of the group of
Hamiltonian diffeomorphisms is at least $k.$

##  1\. Introduction

The problem of determining the homotopy type of the group of Hamiltonian
diffeomorphisms for a closed symplectic manifold is nowadays a far reaching
problem in symplectic topology. In order to have an idea of the limited
knowledge in the subject, absolutely nothing is known about the homotopy type
of the group $\textup{Ham}(\mathbb{T}^{4},\omega)$ where
$(\mathbb{T}^{4},\omega)$ is the $4$-dimensional torus with the standard
symplectic form. Note that we stated an example of a 4-dimensional manifold,
since for 2-dimensional symplectic manifolds the problem is partially
understood due to the fact that in two dimensions symplectic geometry agrees
with area and orientation preserving geometry. See for instance [11, Sec.
7.2]. In this direction,
$\textup{Ham}(S^{2},\omega)=\textup{Symp}_{0}(S^{2},\omega)$ has the same
homotopy type as $SO(3)$ [13]; and for the surface of genus $g\geq 1$,
$\textup{Ham}(M_{g},\omega)$ is simply connected.

In higher dimensions there are some cases where the homotopy type of
$\textup{Ham}(M,\omega)$ is completely understood, due the techniques of
holomorphic curves. For instance, $\textup{Ham}(\mathbb{C}P^{2},\omega_{FS})$
has the homotopy type of $PU(3)$;
$\textup{Ham}(\mathbb{C}P^{1}\times\mathbb{C}P^{1},\omega_{FS}\oplus\omega_{FS})$
has the homotopy type of $SO(3)\times SO(3)$. These results are due to M.
Gromov [4]. The rational homotopy type for the case of one-point blow up of
$(\mathbb{C}P^{2},\omega_{\textup{FS}})$ was settled by M. Abreu and D. McDuff
in [1]. For more examples, see the work of F. Lalonde and M. Pinsonnault [6];
and J. Evans [3].

Leave behind the problem of determining the homotopy type and focus on first
stage of the problem: the fundamental group of $\textup{Ham}(M,\omega)$.
Recall that the fundamental group of a topological group is an abelian group.
Hence is natural to ask: Given any positive integer $k$, does there exists a
symplectic manifold such that the free part of
$\pi_{1}(\textup{Ham}(M,\omega))$ is isomorphic to $\mathbb{Z}^{k}$. Using
cartesian products of symplectic manifolds together with Seidel’s
representation [12] or Weinstein’s morphism [14] if possible to provide a weak
answer to the problem. Namely, is possible to construct a symplectic manifold
$(M,\omega)$ such that the rank of $\pi_{1}(\textup{Ham}(M,\omega))$ is at
least $k$. See for instance [9], where Seidel’s morphism on cartesian products
is studied. In this note we arrive to the same conclusion but on 4-dimensional
symplectic manifolds. That is, in the smallest possible dimension.

######  Theorem 1.1.

Given a positive integer $k$, there exists a closed, connected and simply
connected symplectic $4$-manifold $(M,\omega)$ such that

$\displaystyle\textup{rank }\pi_{1}(\textup{Ham}(M,\omega))\geq k.$

The proof that we provide is a hands-on proof. The symplectic $4$-manifold of
the theorem turns out to be the blow up of
$(\mathbb{C}P^{2},\omega_{\textup{FS}})$ at $k$ points of distinct weights.
The techniques used throughout this note are soft techniques of symplectic
topology, where Weinstein’s morphism plays a key role.

If $\textup{Symp}_{0}(M,\omega)$ stands for the connected component of
$\textup{Symp}(M,\omega)$ that contains the identity map, then the inclusion
$\textup{Ham}(M,\omega)\subset\textup{Symp}_{0}(M,\omega)$ induces an
injective map

$\pi_{1}(\textup{Ham}(M,\omega))\to\pi_{1}(\textup{Symp}_{0}(M,\omega))$

due to the Flux morphism, [8, Ch. 10]. Therefore, Theorem 1.1 also holds if
the group $\textup{Ham}(M,\omega)$ is replaced by
$\textup{Symp}_{0}(M,\omega)$.

Unlike the group of Hamiltonian diffeomorphisms of a closed symplectic
manifold, the group of symplectic diffeomorphisms is not necessarily
connected. Therefore, following the same line of ideas is natural to ask if
given a positive number $k$ does there exists a closed connected symplectic
manifold $(M,\omega)$ such that the number of connected components of
$\textup{Symp}(M,\omega)$ is equal to $k$. Recently, D. Aroux and I. Smith
solved this problem in [2, Thm 1.3] via Floer-theoretic arguments.

As a byproduct of the arguments used to prove the main result, we are also
able to show that Calabi’s morphism on the one-point blow up of
$(\mathbb{R}^{4},\omega_{0})$ is non trivial. The first examples of open
manifolds whose Calabi’s morphism is non trivial are due to A. Kislev [5].
Alongside we prove that the rank of the fundamental group of
$\textup{Ham}(\widetilde{M},\widetilde{\omega}_{\kappa})$ is positive where
$(\widetilde{M},\widetilde{\omega}_{\kappa})$ is the one-point blow of weight
$\kappa$ of $(M,\omega)$. Hence our result improves the one obtained by D.
McDuff [7], since information about $\pi_{1}(M)$ and $\pi_{2}(M)$ is
irrelevant in our arguments.

######  Theorem 1.2.

Let $(\widetilde{M},\widetilde{\omega}_{\kappa})$ be the one-point blow up of
weight $\kappa$ of the closed manifold $(M,\omega)$. Then for infinitely many
values of $\kappa$, the rank of
$\pi_{1}(\textup{Ham}(\widetilde{M},\widetilde{\omega}_{\kappa}))$ is
positive.

It is worth mentioning the case $M=\mathbb{T}^{4}$ with the standard
symplectic form. Therefore,
$\pi_{1}(\textup{Ham}(\widetilde{\mathbb{T}}^{4},\widetilde{\omega}_{\kappa}))$
has positive rank. However, nothing is known about the group
$\pi_{1}(\textup{Ham}({\mathbb{T}}^{4},\omega))$.

We are grateful to L. Polterovich from bringing [5] to our attention.

##  2\. Preliminary computations

###  2.1. A compactly supported path of Hamiltonian diffeomorphisms on
$(\mathbb{R}^{4},\omega_{0})$

Consider $1$-periodic smooth functions
$a_{1},a_{2},a_{3},a_{4}:\mathbb{R}\to\mathbb{R}$ such that
$a_{1}(0)=a_{4}(0)=1$ and $a_{2}(0)=a_{3}(0)=0$. Let
$\alpha:\mathbb{R}\to\mathbb{R}$ be also a smooth function such that
$\alpha(0)\in\mathbb{Z}$. Furthermore, the functions $a_{j}$ are also subject
to the condition that

(1) $\displaystyle A_{t}:=\begin{pmatrix}a_{1}(t)e^{2\pi
i\,\alpha(t)}&a_{2}(t)e^{-2\pi it}\\\ a_{3}(t)e^{2\pi
i\,\alpha(t)}&a_{4}(t)e^{-2\pi it}\end{pmatrix}\ \ \ \ t\in[0,1]$

is a $2\times 2$ unitary matrix. Notice that the only constraint on $\alpha$
is $\alpha(0)\in\mathbb{Z}$, it does not have to be periodic unlike the
functions $a_{j}$. Thus $\\{A_{t}\\}_{0\leq t\leq 1}$ is a path in $U(2)$ that
stars at the identity. Let $\psi^{\bf
a}_{t}:(\mathbb{C}^{2},\omega_{0})\to(\mathbb{C}^{2},\omega_{0})$ be the path
of Hamiltonian diffeomorphism induced by $\\{A_{t}\\}$. A direct computation
yields the Hamiltonian $H^{\bf a}_{t}$ and time-dependent vector field $X^{\bf
a}_{t}$ induced by $\\{\psi^{\bf a}_{t}\\}$.

######  Lemma 2.3.

The path $\\{\psi^{\bf a}_{t}\\}_{0\leq t\leq 1}$ induced by the path of
unitary matrices $\\{A_{t}\\}$ induces in $(\mathbb{R}^{4},\omega_{0})$ the
time-dependent vector field

$\displaystyle X^{\bf a}_{t}$ $\displaystyle=$ $\displaystyle\left\\{2\pi
y_{1}(a_{2}^{2}-a_{1}^{2}\alpha^{\prime})+x_{2}(a_{1}^{\prime}a_{3}+a_{2}^{\prime}a_{4})+2\pi
y_{2}(a_{2}a_{4}-a_{1}a_{3}\alpha^{\prime})\right\\}\frac{\partial}{\partial
x_{1}}$ $\displaystyle+\left\\{2\pi
x_{1}(a_{1}^{2}\alpha^{\prime}-a_{2}^{2})+2\pi
x_{2}(a_{1}a_{3}\alpha^{\prime}-a_{2}a_{4})+y_{2}(a_{1}^{\prime}a_{3}+a_{2}^{\prime}a_{4})\right\\}\frac{\partial}{\partial
y_{1}}$
$\displaystyle+\left\\{x_{1}(a_{1}a_{3}^{\prime}+a_{2}a_{4}^{\prime})+2\pi
y_{1}(a_{2}a_{4}-a_{1}a_{3}\alpha^{\prime})+2\pi
y_{2}(a_{4}^{2}-a_{3}^{2}\alpha^{\prime})\right\\}\frac{\partial}{\partial
x_{2}}$ $\displaystyle+\left\\{2\pi
x_{1}(a_{1}a_{3}\alpha^{\prime}-a_{2}a_{4})-y_{1}(a_{1}a_{3}^{\prime}+a_{2}a_{4}^{\prime})+2\pi
x_{2}(a_{3}^{2}\alpha^{\prime}-a_{4}^{2})\right\\}\frac{\partial}{\partial
y_{2}},$

and Hamiltonian function

$\displaystyle H^{\bf a}_{t}(x_{1},y_{1},x_{2},y_{2})$ $\displaystyle=$
$\displaystyle\pi(-a_{1}^{2}\alpha^{\prime}+a_{2}^{2})(x_{1}^{2}+y_{1}^{2})+\pi(-a_{3}^{2}\alpha^{\prime}+a_{4}^{2})(x_{2}^{2}+y_{2}^{2})$
$\displaystyle+2\pi(a_{2}a_{4}-a_{1}a_{3}\alpha^{\prime})(x_{1}x_{2}+y_{1}y_{2})$
$\displaystyle+(a_{1}a_{3}^{\prime}+a_{2}a_{4}^{\prime})(x_{1}y_{2}-x_{2}y_{1}).$

For $r>0$ denote by $B_{r}$ the open ball of radius $r$ in $\mathbb{R}^{4}$
centered at the origin. Using the fact that the functions $a_{j}$ define the
unitary matrix (1), it follows that the integral of $H^{\bf a}_{t}$ over the
ball $B_{r}$ only depends on $\alpha$ and $r$.

######  Lemma 2.4.

If $H^{\bf a}_{t}$ is the Hamiltonian function of Lemma 2.3, then

$\displaystyle\int_{0}^{1}\int_{B_{r}}H^{\bf
a}_{t}\,\omega_{0}^{2}\,dt=\left(\frac{\pi^{3}r^{6}}{6}\right)(1+\alpha(0)-\alpha(1)).$

###### Proof.

The integral over $B_{r}$ of the last two terms of $H^{\bf a}_{t}$ are zero.
Now since the functions $a_{j}$ are the entries of a unitary matrix, it
follows that

$\displaystyle\int_{0}^{1}\int_{B_{r}}H^{\bf a}_{t}\,\omega_{0}^{2}\,dt$
$\displaystyle=$
$\displaystyle\pi\int_{B_{r}}(x_{1}^{2}+y_{1}^{2})\,\omega_{0}^{2}\cdot\int_{0}^{1}-a_{1}^{2}\alpha^{\prime}+a_{2}^{2}-a_{3}^{2}\alpha^{\prime}+a_{4}^{2}\,dt$
$\displaystyle=$
$\displaystyle\pi\left(\frac{\pi^{2}r^{6}}{6}\right)\int_{0}^{1}1-\alpha^{\prime}\,dt$
$\displaystyle=$
$\displaystyle\pi\left(\frac{\pi^{2}r^{6}}{6}\right)(1+\alpha(0)-\alpha(1)).$

∎

We are interested in the case when the above integral is non zero. Thereby, by
the last result we neglect the functions $a_{j}$ and focus our attention on
$\alpha$.

Next we used a bump function to obtain a compactly supported Hamiltonian path.
This is the reason that we considered a Hamiltonian path instead of a
Hamiltonian loop. If a Hamiltonian function induces a loop, then multiplying
it by a bump function will no longer induce a loop. To that end fix
$r_{0},R_{0}\in\mathbb{R}$ such that $0<r_{0}<R_{0}$ and also fix a bump
function $\rho:\mathbb{R}^{4}\to\mathbb{R}$ such that $\rho\equiv 1$ on
$B_{r_{0}}$ and $\rho\equiv 0$ on $\mathbb{R}^{4}\setminus B_{R_{0}}$.
Consider the compactly supported smooth function $H^{\bf
a,\rho}_{t}:(\mathbb{R}^{4},\omega_{0})\to\mathbb{R}$ defined as

$H^{\bf a,\rho}_{t}:=\rho\cdot H^{\bf a}_{t}$

and let $\\{\psi^{\bf a,\rho}_{t}\\}_{0\leq t\leq 1}$ be the induced
Hamiltonian path. Observe that $\psi^{\bf
a,\rho}_{t}\in\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ and that on the
ball $B_{r_{0}}$ we have that $\psi^{\bf a,\rho}_{t}=\psi^{\bf a}_{t}$ and
$H^{\bf a,\rho}_{t}=H^{\bf a}_{t}$ for all $t\in[0,1]$.

###  2.2. A compactly supported loop of Hamiltonian diffeomorphisms on
$(\mathbb{R}^{4},\omega_{0})$

Next we define a loop in $\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ based
at the identity by concatenating two paths of the kind defined above. Fix
$a_{1},\ldots a_{4},b_{1},\ldots b_{4}$ smooth 1-periodic functions and
$\alpha$ and $\beta$ also smooth functions, such that all functions are
subject to the conditions previously imposed. Further, we also impose the
condition that the two path of matrices $\\{A_{t}\\}$ and $\\{B_{t}\\}$ agree
in a neighborhood of $t=1$. Therefore, the exponent functions satisfy
$\alpha(1)-\beta(1)\in\mathbb{Z}$.

Define the Hamiltonian loop $\psi=\\{\psi_{t}\\}_{0\leq t\leq 2}$ in
$\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ as

(4) $\displaystyle\psi_{t}:=\left\\{\begin{array}[]{ll}\psi^{\bf
a,\rho}_{t}&t\in[0,1]\\\ \psi^{\bf b,\rho}_{1-t}&t\in[1,2].\end{array}\right.$

Thus, its compactly supported Hamiltonian function is given by

(7) $\displaystyle H_{t}:=\left\\{\begin{array}[]{ll}H^{\bf
a,\rho}_{t}&t\in[0,1]\\\ H^{\bf b,\rho}_{1-t}&t\in[1,2].\end{array}\right.$

The above observations on the paths $\\{\psi^{\bf a,\rho}_{t}\\}$ and
$\\{\psi^{\bf b,\rho}_{t}\\}$ and the computation of Lemma 2.4 imply the
following facts about the Hamiltonian loop $\psi.$

######  Proposition 2.5.

Given $r_{0},R_{0}\in\mathbb{R}$ such that $R_{0}>r_{0}>0$ there is a
Hamiltonian loop $\psi=\\{\psi_{t}\\}_{0\leq t\leq 2}$ defined as in (4) such
that it is supported in $B_{R_{0}}$ and on $B_{r_{0}}$ it agrees with a loop
of unitary matrices. Moreover, its Hamiltonian function $H_{t}$ satisfies

(8)
$\displaystyle\int_{0}^{2}\int_{B_{r_{0}}}H_{t}\,\omega_{0}^{2}\,dt=\left(\frac{\pi^{3}{r_{0}}^{6}}{6}\right)(\alpha(0)-\alpha(1)-\beta(0)+\beta(1)).$

The condition that the paths $\\{\psi^{\bf a,\rho}_{t}\\}$ and $\\{\psi^{\bf
b,\rho}_{t}\\}$ agree in a neighborhood of 1, imply that
$\alpha(0)-\alpha(1)-\beta(0)+\beta(1)$ is a integer. Thus Eq. (8) is a kind
of winding number for the loop of unitary matrices. Furthermore, it is
possible to choose the functions $\alpha$ and $\beta$ so that the paths agree
in a neighborhood of 1 and $\alpha(0)-\alpha(1)-\beta(0)+\beta(1)$ is non
zero. From now on we fix the Hamiltonian loop $\psi$ in
$\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ defined in (4) so that
$\alpha(0)-\alpha(1)-\beta(0)+\beta(1)$ is equal to 1. The actual value is not
important, what is relevant at this point is that it is a non zero integer.

On an open manifold $(M,\omega)$ the Calabi morphism
$\textup{Cal}:\pi_{1}(\textup{Ham}^{c}(M,\omega))\to\mathbb{R}$ is defined as

$\displaystyle\textup{Cal}(\phi):=\int_{0}^{1}\int_{M}F_{t}\,\omega^{n}\,dt,$

where the loop $\phi$ is generated by the compactly supported Hamiltonian
$F_{t}$. Moreover, if $(M,\omega)$ is exact then Calabi’s morphism is
identically zero. Consequently, for the the loop $\psi$ defined in (4) we have
that

(9) $\displaystyle
0=\int_{0}^{2}\int_{\mathbb{R}^{4}}H_{t}\,\omega_{0}^{2}\,dt,$

in contrast with the integral over $B_{r_{0}}$ that is non zero.

Equation (9) will be useful when we consider the loop $\psi$ in a Darboux
chart on a closed manifold. For, in this case the normalization condition
corresponds to zero mean.

##  3\. A loop of Hamiltonian diffeomorphisms in the one-point blow up of
infinite order

In [10] it is proved that if a closed symplectic manifold admits a Hamiltonian
circle action, then after blowing up one point the fundamental group of the
group of Hamiltonian diffeomorphisms has positive rank. In this section we
prove that the above result always holds, namely we prove Theorem 1.2.
Henceforth, the hypothesis about the Hamiltonian circle action is no longer
required. We restrict to the four-dimensional case in view of the main result
of this paper; however the result presented in this section holds on any
symplectic manifold of dimension greater than or equal than four.

This section can be considered has the first step of the proof of the main
theorem. Recall that for any $R_{0}>0$, the loop $\psi=\\{\psi_{t}\\}_{0\leq
t\leq 2}$ defined in (4) is supported in the open ball
$B_{R_{0}}\subset(\mathbb{R}^{4},\omega_{0})$. Let $(M,\omega)$ be a closed
rational symplectic 4-manifold. For $R_{0}>0$ small enough, by Darboux’s
theorem the loop $\psi\in\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ can be
regarded in $\textup{Ham}(M,\omega)$.

Since we are in a closed symplectic manifold $(M,\omega)$, we must normalized
the Hamiltonian function $H_{t}:(M,\omega)\to\mathbb{R}$. Thus, for
$t\in[0,1]$ define

$c_{t}:=\frac{1}{\textup{Vol}(M,\omega^{2})}\int_{M}H_{t}\,{\omega^{2}}.$

Afterwards define $H^{\bf N}_{t}:M\to\mathbb{R}$ as $H^{\bf
N}_{t}:=H_{t}-c_{t}$. Thus, $H^{\bf N}_{t}$ is the normalized Hamiltonian
function that induces the same loop $\psi$ in $\textup{Ham}(M,\omega)$.

Call $\iota B_{r_{0}}\subset M$ the image of $B_{r_{0}}\subset\mathbb{R}^{4}$
under the Darboux embedding. Then the loop $\psi=\\{\psi_{t}\\}$ satisfies the
following:

  * •

$\psi_{t}(\iota(0)))=\iota(0)$ for all $t\in[0,2]$

  * •

$\psi_{t}$ behaves like a unitary matrix on $\iota B_{r_{0}}$ for all
$t\in[0,2]$.

Using the embedded ball $\iota B_{r_{0}}\subset M$, define
$(\widetilde{M},\widetilde{\omega}_{r_{0}})$ to be the one-point blow up at
$\iota(0)$ of $(M,\omega)$ of weight $r_{0}$. From the above remarks on the
loop $\psi$, it follows from [10, Sec. 3] that $\psi$ induces a Hamiltonian
loop $\widetilde{\psi}=\\{\widetilde{\psi}_{t}\\}_{0\leq t\leq 2}$ in
$\textup{Ham}(\widetilde{M},\widetilde{\omega}_{r_{0}})$. For appropriate
values of $r_{0}$, we claim that $[\widetilde{\psi}]$ has infinite order in
$\pi_{1}(\textup{Ham}(\widetilde{M},\widetilde{\omega}_{r_{0}})).$ We prove
this using Weinstein’s morphism

$\mathcal{A}:\pi_{1}(\textup{Ham}(\widetilde{M},\widetilde{\omega}_{r_{0}}))\to\mathbb{R}/\mathcal{P}(\widetilde{M},\widetilde{\omega}_{r_{0}}),$

where $\mathcal{P}(\widetilde{M},\widetilde{\omega}_{r_{0}})$ is the period
group.

Next we prove Theorem 1.2 that was stated at the Introduction. We give a more
precise statement of the theorem in terms of the loop of Hamiltonian
diffeomorphisms $\psi$ defined above and the weight of the blow up. Keep in
mind that we state this result for four-dimensional symplectic manifolds, but
the same argument works in higher dimensions.

######  Theorem 3.6.

Let $r_{0}>0$ such that $\pi r_{0}^{2}$ is a transcendental number. If
$(M,\omega)$ is a rational symplectic 4-manifold, then the induced loop
$[\widetilde{\psi}]$ has infinite order in
$\pi_{1}(\textup{Ham}(\widetilde{M},\widetilde{\omega}_{r_{0}})).$

###### Proof.

Since $(M,\omega)$ is rational, there are $q_{1},\ldots,q_{s}\in\mathbb{Q}$
such that $\mathcal{P}(M,\omega)=\mathbb{Z}\langle q_{1},\ldots,q_{s}\rangle$.
In fact,
$\mathcal{P}(\widetilde{M},\widetilde{\omega}_{r_{0}})=\mathbb{Z}\langle
q_{1},\ldots,q_{s},\pi r_{0}^{2}\rangle$ since the area of the line in the
exceptional divisor is $\pi r_{0}^{2}$.

From [10, Thm. 1.1], it is possible to compute $\mathcal{A}(\widetilde{\psi})$
in terms solely of the loop $\psi$. Namely,

$\displaystyle\mathcal{A}(\widetilde{\psi})=\left[\mathcal{A}(\psi)+\frac{1}{\textup{Vol}(\widetilde{M},\widetilde{\omega}_{r_{0}}^{2})}\int_{0}^{2}\int_{\iota
B_{r_{0}}}H^{\bf N}_{t}\,{\omega^{2}}\,dt\right].$

Since $\\{\psi_{t}\\}$ is null homotopic, it follows that
$\mathcal{A}(\psi)=0$ in $\mathbb{R}/\mathcal{P}(M,\omega)$. Observe that the
integral of $H^{\bf N}_{t}$ over $\iota B_{r_{0}}\subset M$ is the same as the
integral over $B_{r_{0}}\subset\mathbb{R}^{4}$. Therefore, from Proposition
2.5 we have that

$\displaystyle\int_{0}^{2}\int_{\iota B_{r_{0}}}H^{\bf
N}_{t}\,{\omega^{2}}\,dt$ $\displaystyle=$
$\displaystyle\int_{0}^{2}\int_{\iota B_{r_{0}}}H_{t}-c_{t}\,{\omega^{2}}\,dt$
$\displaystyle=$ $\displaystyle\left(\frac{\pi^{3}{r_{0}}^{6}}{6}\right)\cdot
1-\textup{Vol}(B_{r_{0}},\omega^{2})\int_{0}^{2}c_{t}\,dt.$

The functions $H_{t}$ are supported $B_{R_{0}}\subset\mathbb{R}^{4}$, thus
they can also be considered as a functions on $M$. Hence using Eq. (9) it
follows that

$\displaystyle\int_{0}^{2}c_{t}\,dt$ $\displaystyle=$
$\displaystyle\int_{0}^{2}\frac{1}{\textup{Vol}(M,\omega^{2})}\int_{M}H_{t}\,{\omega^{2}}\,dt$
$\displaystyle=$
$\displaystyle\frac{1}{\textup{Vol}(M,\omega^{2})}\int_{0}^{2}\int_{M}H_{t}\,{\omega^{2}}\,dt=0.$

If $V$ stands for $\textup{Vol}(M,\omega^{2})$, then
$\textup{Vol}(\widetilde{M},\widetilde{\omega}_{r_{0}}^{2})=V-\pi^{2}r_{0}^{4}/2$.
Substituting the above computations, we have

$\displaystyle\mathcal{A}(\widetilde{\psi})=\left[\frac{1}{V-\pi^{2}r_{0}^{4}/2}\left(\frac{\pi^{3}{r_{0}}^{6}}{6}\right)\right]\in\mathbb{R}/\mathbb{Z}\langle
q_{1},\ldots,q_{s},\pi r_{0}^{2}\rangle.$

Then the equation $\mathcal{A}(\widetilde{\psi}^{m})=0$, for $m\in\mathbb{N}$,
is equivalent to a polynomial equation on $\pi r_{0}^{2}$ with rational
coefficients. Recall that $V\in\mathbb{Q}.$ Since $\pi r_{0}^{2}$ is
transcendental, the Hamiltonian loop
$\widetilde{\psi}=[\\{\widetilde{\psi}_{t}\\}_{0\leq t\leq 2}]$ has infinite
order in $\pi_{1}(\textup{Ham}(\widetilde{M},\widetilde{\omega}_{r_{0}})).$ ∎

##  4\. Proof of the main theorem

The proof of Theorem 3.6 gives the blueprint that we follow in order to prove
the main result. The proof of the main theorem deals with $k$ distinct
Hamiltonian loops supported in $k$ mutually disjoint balls. Thereby, the
hypothesis in Theorem 3.6, about $\pi r_{0}$ being transcendental, is replaced
by the following lemma.

######  Lemma 4.7.

Given $k\in\mathbb{N}$ there exist $k$ distinct real numbers
$y_{1},\ldots,y_{k}$ such that for any $j,s\in\\{1,\ldots,k\\}$ the equation

$(a+q_{1}y_{1}+\cdots+q_{k}y_{k})(b-(y_{1}^{2}+\cdots+y_{k}^{2}))+cy_{j}^{3}+y_{s}^{3}=0$

has no solution for any $q_{1},\ldots,q_{k},a,b,c\in\mathbb{Q}$.

The proof of the lemma is a consequence of the fact that $\mathbb{R}$ is an
infinite dimensional $\mathbb{Q}$-vector space. Next we provide the proof of
the main theorem builded on the ideas of the proof of Theorem 3.6.

For the proof of the main theorem is important to consider the complex
2-dimensional projective space $(\mathbb{C}P^{2},\omega_{FS})$ endowed with
its standard Fubini-Study symplectic form. If $\omega_{FS}$ is normalized so
that $(\mathbb{C}P^{2}\setminus\mathbb{C}P^{1},\omega_{FS})$ is
symplectomorphic to the open ball of radius $R$ in
$(\mathbb{R}^{4},\omega_{0})$, then the area of a complex line is $\pi R^{2}$.
Therefore, the period group of $(\mathbb{C}P^{2},\omega_{FS})$ is
$\mathbb{Z}\langle\pi R^{2}\rangle$ and
$\textup{Vol}(\mathbb{C}P^{2},\omega_{FS}^{2})=(\pi R^{2})^{2}/2$.

###### Proof of Theorem 1.1.

Given $k>0$, it follows from Lemma 4.7 that there are $k$ distinct numbers
$r_{1},\ldots,r_{k}\in\mathbb{R}_{>0}$ where $y_{j}:=\pi r_{j}^{2}$ for
$j\in\\{1,\ldots,n\\}$.

Let $R_{1},\ldots,R_{k}\in\mathbb{R}$ be any numbers such that $R_{j}>r_{j}$
for all $j\in\\{1,\ldots,k\\}$. On $\mathbb{R}^{4}$, fix $k$ mutually disjoint
balls $B_{1},\ldots,B_{k}$ centered at $p_{1},\ldots,p_{k}$ and of radii
$R_{1},\ldots,R_{k}$ respectively. Inside each ball $B_{j}$ fix a a smaller
ball $B_{r_{j}}$ of radius $r_{j}$ centered at $p_{j}$. Next, consider the
complex 2-dimensional projective space $(\mathbb{C}P^{2},\omega_{FS})$ such
that the symplectic form is normalized so that
$(\mathbb{C}P^{2}\setminus\mathbb{C}P^{1},\omega_{FS})$ is symplectomorphic to
an open ball of radius $R$ in $(\mathbb{R}^{4},\omega_{0})$ that contains all
the balls $B_{1},\ldots,B_{k}$ and $\pi R^{2}$ is a rational number. Therefore
the period group of $(\mathbb{C}P^{2},\omega_{FS})$ is $\mathbb{Z}\langle\pi
R^{2}\rangle$.

Call $\iota_{j}B_{r_{j}}\subset\mathbb{C}P^{2}$ the image of the fixed ball
$B_{r_{j}}\subset\mathbb{R}^{4}$. Hence in $(\mathbb{C}P^{2},\omega_{FS})$
there are $k$ mutually disjoint embedded balls
$\iota_{1}B_{r_{1}},\ldots,\iota_{k}B_{r_{k}}$. Denote by
$(\mathbb{C}P^{2}\\#_{k}\overline{\mathbb{C}P}^{2},\widetilde{\omega}_{\bf
r})$ the symplectic manifold that is obtained by blowing up the $k$ points
$\iota_{1}(p_{1}),\ldots,\iota_{k}(p_{k})$ in $(\mathbb{C}P^{2},\omega_{FS})$
where the weight at $\iota_{j}(p_{j})$ is $r_{j}.$ That is, the embedded ball
$\iota_{j}B_{r_{j}}$ is removed from the torus for $j\in\\{1,\ldots,k\\}.$
$(\mathbb{C}P^{2}\\#_{k}\overline{\mathbb{C}P}^{2},\widetilde{\omega}_{\bf
r})$ is the desired closed symplectic 4-manifold of the main theorem. Notice
that it is simply connected.

Next we define $k$ Hamiltonian loops. From Proposition 2.5 for each
$j\in\\{1,\ldots,k\\}$ there is a loop $\psi^{(j)}=\\{\psi^{(j)}_{t}\\}_{0\leq
t\leq 2}$ in $\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ supported in the
ball of radius $R_{j}$ such that inside the ball of radius $r_{j}$ it agrees
with a loop of unitary matrices. Therefore, the $k$ loops in
$\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$ induced $k$ loops
$\textup{Ham}(\mathbb{C}P^{2},\omega_{\textup{FS}})$ that we also denoted by
$\psi^{(1)},\ldots,\psi^{(k)}$.

For every $j\in\\{1,\ldots,k\\}$, the loop $\psi^{(j)}$ on
$(\mathbb{C}P^{2},\omega_{\textup{FS}})$ behaves as a loop of unitary matrices
on each of the embedded ball $\iota_{s}B_{r_{s}}$. Moreover, if $s\neq j$ it
is the constant loop on $\iota_{s}B_{r_{s}}$. In any case, the loop
$\psi^{(j)}$ fixes the points $\iota_{1}(p_{1}),\ldots,\iota_{k}(p_{k})$.
Therefore, from [10, Sec. 3] it follows that $\psi^{(j)}$ induces a loop
$\widetilde{\psi}^{(j)}$ in
$\textup{Ham}(\mathbb{C}P^{2}\\#_{k}\overline{\mathbb{C}P}^{2},\widetilde{\omega}_{\bf
r})$. We claim that the loops
$\widetilde{\psi}^{(1)},\ldots,\widetilde{\psi}^{(k)}$ generate a subgroup
isomorphic to $\mathbb{Z}^{k}$ in
$\pi_{1}(\textup{Ham}(\mathbb{C}P^{2}\\#_{k}\overline{\mathbb{C}P}^{2},\widetilde{\omega}_{\bf
r}))$.

Fix $j\in\\{1,\ldots k\\}$. The corresponding Hamiltonian $H_{t}^{(j)}$ of
$\psi^{(j)}$ is compactly supported, as before let $H_{t}^{(j),{\bf
N}}:(\mathbb{C}P^{2},\omega_{\textup{FS}})\to\mathbb{R}$ be its normalization.
As in the proof of Theorem 3.6 we have that

$\displaystyle\int_{0}^{2}\int_{\iota_{j}B_{r_{j}}}H_{t}^{(j),{\bf
N}}\,{\omega^{2}}\,dt=\left(\frac{\pi^{3}{r_{j}}^{6}}{6}\right)\cdot 1$

and

$\displaystyle\mathcal{A}(\widetilde{\psi}^{(j)})=\left[\frac{1}{V-\pi^{2}(r_{1}^{4}+\cdots+r_{k}^{4})/2}\left(\frac{\pi^{3}{r_{j}}^{6}}{6}\right)\right]\in\mathbb{R}/\mathbb{Z}\langle\pi
R^{2},\pi r_{1}^{2},\ldots,\pi r_{k}^{2}\rangle.$

where
$V:=\textup{Vol}({\mathbb{C}P}^{2},\omega_{\textup{FS}}^{2})\in\mathbb{Q}$.

Since $\pi R^{2}\in\mathbb{Q}$ and the numbers $\pi r_{1}^{2},\ldots,\pi
r_{k}^{2}$ were chosen according to Lemma 4.7, it follows that for any
$m\in\mathbb{Z}_{>0}$ the equation
$\mathcal{A}((\widetilde{\psi}^{(j)})^{m})=0$ does not hold. Therefore
$[\widetilde{\psi}^{(j)}]$ has infinite order in
$\pi_{1}(\textup{Ham}(\mathbb{C}P^{2}\\#_{k}\overline{\mathbb{C}P}^{2},\widetilde{\omega}_{\bf
r}))$. The same reasoning implies that
$\mathcal{A}((\widetilde{\psi}^{(j)})^{m})\neq\mathcal{A}((\widetilde{\psi}^{(s)})^{n})$
for distinct $j,s\in\\{1,\ldots,k\\}$ and any $m,n\in\mathbb{Z}_{>0}$.
Henceforth the rank of
$\pi_{1}(\textup{Ham}(\mathbb{C}P^{2}\\#_{k}\overline{\mathbb{C}P}^{2},\widetilde{\omega}_{\bf
r}))$ is at least $k$. ∎

##  5\. Calabi’s morphism

As noted before, Calabi’s morphism on
$\pi_{1}(\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0}))$ is trivial. However,
the arguments used before show that Calabi’s morphism on the one-point blow up
of $(\mathbb{R}^{4},\omega_{0})$ is non trivial

Fix $r>0$, let $(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r})$ be the
blowup of the origin in $(\mathbb{R}^{4},\omega_{0})$ of weight $r$. As we
have seen the loop $\psi$ in $\textup{Ham}^{c}(\mathbb{R}^{4},\omega_{0})$
induces the loop $\widetilde{\psi}$ in
$\textup{Ham}^{c}(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r})$.

######  Proposition 5.8.

For any $r>0$, the Calabi morphism on
$\textup{Ham}^{c}(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r})$ does not
vanish.

###### Proof.

Let $\psi$ be the Hamiltonian loop in
$\textup{Ham}^{c}({\mathbb{R}^{4}},\omega_{0})$ defined in Eq. 4 and let
$\widetilde{\psi}$ be the induced loop in
$\textup{Ham}^{c}(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r})$.
According to [10, Sec. 1], $\textup{Cal}(\widetilde{\psi})$ and
$\textup{Cal}(\psi)$ are related as

(10)
$\displaystyle\textup{Cal}(\widetilde{\psi})=\textup{Cal}(\psi)-\frac{1}{2}\int_{0}^{2}\int_{B_{r}}H_{t}\,\omega^{2}\,dt$

where $H_{t}:(\mathbb{R}^{4},\omega)\to\mathbb{R}$ is the compactly supported
Hamiltonian function of the loop $\psi$ defined in Eq. (7). Thus from
Proposition 2.5,

$\displaystyle\textup{Cal}(\widetilde{\psi})$ $\displaystyle=$
$\displaystyle-\frac{\pi^{3}{r}^{6}}{12}\cdot 1.$

∎

Remark. The above result is true for any dimension; Calabi’s morphism on
$\textup{Ham}^{c}(\widetilde{\mathbb{R}^{2n}},\widetilde{\omega}_{r})$ does
not vanish for $n\geq 2$ and $r>0$.

Finally, note that since Calabi’s morphism does not vanish on
$\pi_{1}(\textup{Ham}^{c}(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r}))$
it does not descend to
$\textup{Ham}^{c}(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r})$.
Furthermore, for the Hamiltonian loop $\widetilde{\psi}$ in
$\textup{Ham}^{c}(\widetilde{\mathbb{R}^{4}},\widetilde{\omega}_{r})$ that
appears in the previous proof we have that

$\displaystyle\ell(\widetilde{\psi})\geq\frac{\pi^{3}{r}^{6}}{12}.$

Here $\ell(\cdot)$ stands for the Hofer length of the class in
$\pi_{1}(\textup{Ham}(\cdot))$. For the precise definition of $\ell(\cdot)$,
see [11, Sec. 7.3].

## References

  * [1] Abreu, M., and McDuff, D. Topology of symplectomorphism groups of rational ruled surfaces.  J. Amer. Math. Soc. 13, 4 (2000), 971–1009 (electronic).
  * [2] Auroux, D., and Smith, I. Fukaya categories of surfaces, spherical objects, and mapping class groups, arXiv.2006.09689.
  * [3] Evans, J. D. Symplectic mapping class groups of some Stein and rational surfaces.  J. Symplectic Geom. 9, 1 (2011), 45–82.
  * [4] Gromov, M. Pseudoholomorphic curves in symplectic manifolds.  Invent. Math. 82, 2 (1985), 307–347.
  * [5] Kislev, A. Compactly supported Hamiltonian loops with a non-zero Calabi invariant.  Electron. Res. Announc. Math. Sci. 21 (2014), 80–88.
  * [6] Lalonde, F., and Pinsonnault, M. The topology of the space of symplectic balls in rational 4-manifolds.  Duke Math. J. 122, 2 (2004), 347–397.
  * [7] McDuff, D. The symplectomorphism group of a blow up.  Geom. Dedicata 132 (2008), 1–29.
  * [8] McDuff, D., and Salamon, D. Introduction to symplectic topology, second ed.  Oxford Mathematical Monographs. The Clarendon Press, Oxford University Press, New York, 1998.
  * [9] Pedroza, A. Seidel’s representation on the Hamiltonian group of a Cartesian product.  Int. Math. Res. Not. IMRN, 14 (2008), Art. ID rnn049, 19.
  * [10] Pedroza, A. Hamiltonian loops on the symplectic blow up.  J. Symplectic Geom. 16, 3 (2018), 839–856.
  * [11] Polterovich, L. The geometry of the group of symplectic diffeomorphisms.  Lectures in Mathematics ETH Zürich. Birkhäuser Verlag, Basel, 2001\.
  * [12] Seidel, P. $\pi_{1}$ of symplectic automorphism groups and invertibles in quantum homology rings.  Geom. Funct. Anal. 7, 6 (1997), 1046–1095.
  * [13] Smale, S. Diffeomorphisms of the $2$-sphere.  Proc. Amer. Math. Soc. 10 (1959), 621–626.
  * [14] Weinstein, A. Cohomology of symplectomorphism groups and critical values of Hamiltonians.  Math. Z. 201, 1 (1989), 75–82.


# When is every non central-unit a sum of two nilpotents?

Simion Breaz and Yiqiang Zhou  Department of Mathematics, Faculty of
Mathematics and Computer Science, Babes-Bolyai University, Ro-400084 Cluj-
Napoca, Romania<EMAIL_ADDRESS>Department of Mathematics and
Statistics, Memorial University of Newfoundland, St.John’s, NL A1C 5S7, Canada
<EMAIL_ADDRESS>
###### Abstract.

A ring is said to satisfy the $2$-nil-sum property if every non central-unit
is the sum of two nilpotents. We prove that a ring satisfies the $2$-nil-sum
property iff it is either a simple ring with the $2$-nil-sum property or a
commutative local ring with nil Jacobson radical, and we provide an example of
a simple rings with the $2$-nil-sum property that is not commutative.
Moreover, a simple right Goldie ring has the $2$-nil-sum property iff it is a
field.

###### Key words and phrases:

Nilpotent, unit, $2$-nil-sum property, matrix ring, simple ring

###### 2020 Mathematics Subject Classification:

16U99; 16S50

##  1\. Introduction

Throughout, rings are associative with identity. We start by recalling three
special types of elements in a ring. An element in a ring is $2$-good if it is
a sum of two units (see [18]), is fine if it is a sum of a unit and a
nilpotent (see [4]), and is $2$-nilgood if it is a sum of two nilpotents (see
[3]). By the terminology of Vámos [18], a ring is $2$-good if each element is
$2$-good. The study of $2$-good rings (also called rings with the $2$-sum
property in the literature), initiated by Wolfson [19] and Zelinsky [20], has
attracted considerable interest (see, for example, [2], [11], [14], [17], [18]
and the references there). A ring is called a fine ring if each nonzero
element is fine (the zero element being fine implies that the ring is
trivial). Fine rings were introduced and extensively investigated by
Călugăreanu and Lam [4].

Motivated by the notions of $2$-good rings and fine rings, we define a ring to
satisfy the $2$-nil-sum property if every element that is not a central unit
is $2$-nilgood (a central unit in a ring being $2$-nilgood implies that the
ring is trivial). These rings belong to a larger class of rings for which
every element is either a unit or a sum of two units. But, we will see that
the $2$-nil-sum property is very restrictive. For instance, a ring without
nonzero nilpotents satisfies the $2$-nil-sum property iff it is a field. The
rings which is additively generated by nilpotents have been studied by several
authors with connections to rings additively generated by commutators. We
refer to [5] and [10] for more details. As far as we are aware, the $2$-nil-
sum property of rings has not been discussed in the literature. Here a study
of this topic is conducted. The main results proved here are the following: a
ring satisfies the $2$-nil-sum property iff it is either a simple ring with
the $2$-nil-sum property or a commutative local ring with nil Jacobson radical
(Theorem 2.2); there exists simple rings with the $2$-nil-sum property that
are not commutative (Example 3.1); a simple right Goldie ring has the $2$-nil-
sum property iff it is a field (Theorem 3.8).

For a ring $R$, we denote by $C(R)$, $J(R)$, $U(R)$, and ${\rm nil}(R)$ the
center, the Jacobson radical, the group of units of $R$, and the set of
nilpotents of $R$, respectively. We write $\mathbb{M}_{n}(R)$ for the ring of
$n\times n$ matrices over $R$ whose identity is denoted by $I_{n}$. By a non
central-unit we mean an element that is not a central unit. For
$i,j\in\\{1,2,\ldots,n\\}$, we denote by $E_{ij}$ the $n\times n$ matrix whose
$(i,j)$-entry is $1$ and all other entries are $0$.

##  2\. Basic properties and the reduction theorem

We start by recording some basic properties.

######  Lemma 2.1.

Let $R$ be a ring with the $2$-nil-sum property. The following hold:

  1. (1)

$C(R)\backslash U(R)\subseteq{\rm nil}(R)$. Consequently, $C(R)$ is a local
ring with nil Jacobson radical.

  2. (2)

For every proper ideal $I$ of $R$ we have $I\subseteq C(R)\cap{\rm nil}(R)$.

  3. (3)

All proper ideals of $R$ are contained in $J(R)$.

  4. (4)

$J(R)$ is nil and $J(R)\subseteq C(R)$.

  5. (5)

For any ideal $I$ of $R$, central units of $R/I$ can be lifted to central
units of $R$.

###### Proof.

(1) Assume that $r$ is a central element that is not a unit. Then
$r=b_{1}+b_{2}$, where $b_{1},b_{2}\in{\rm nil}(R)$. Since $r$ is central,
$b_{1}$ and $b_{2}$ commute. So $r=b_{1}+b_{2}\in{\rm nil}(R)$.

(2) Assume that $I\backslash C(R)\neq\varnothing$. Let $a\in I\backslash
C(R)$. Then $1+a$ is a sum of two nilpotents, so $\bar{1}\in R/I$ is a sum of
two nilpotents, a contradiction. Hence $I\subseteq C(R)\backslash U(R)$, so
$I\subseteq{\rm nil}(R)$ by (1).

(3) and (4) are clear by (2).

$(5)$ Let $\bar{u}:=u+I$ be a central unit of $R/I$. We claim that $u\in R$ is
a central unit. Otherwise, $u$ is a sum of two nilpotents in $R$. Hence
$\bar{u}$ is a sum of two nilpotents in $R/I$, a contradiction. ∎

######  Theorem 2.2.

(Reduction Theorem) A ring $R$ satisfies the $2$-nil-sum property iff $R$ is
either a simple ring with the $2$-nil-sum property or a commutative local ring
with nil Jacobson radical.

###### Proof.

The sufficiency is clear. For the necessity, suppose that $R$ satisfies the
$2$-nil-sum property but $R$ is not simple. Then, by Lemma 2.1(3),
$J(R)\not=0$. Let $0\not=x\in J(R)$. If $a,b\in R$ then, noting $J(R)\subseteq
C(R)$, we have

$(ab)x=a(bx)=(bx)a=b(xa)=b(ax)=(ba)x,$

hence $(ab-ba)x=0$. It follows that $ab-ba$ belongs to the ideal
$Ann(x)=\\{r\in R\mid rx=0\\}$. Since $Ann(x)\not=R$, $Ann(x)\subseteq J(R)$.
It follows that $R/J(R)$ is a commutative ring with the $2$-nil-sum property,
and hence is local by Lemma 2.1(1). It follows that $R/J(R)$ is a field. Thus,
by Lemma 2.1(5), for any $a\in R\backslash J(R)$, $a=u+j$ where $u$ is a
central unit and $j\in J(R)$. It follows that $a$ is central. Hence, $R$ is
commutative, and so $R$ is a commutative local ring with $J(R)$ nil. ∎

If $R$ is a commutative ring with the $2$-nil-sum property then every non
central-unit $a\in R$ is nilpotent, so it is a sum of the form $a=b+c$ with
$b^{1}=0$ and $c=a$ is nilpotent. Thus, $R$ is a ring with the $2$-nil-sum
property of type $(1,\infty)$ as defined below. The $2$-nil-sum property is
refined as follows: let $p,q$ be integers with $p\geq 1$ and $q\geq 1$. We say
that a ring $R$ has the $2$-nil-sum property of type $(p,q)$ if, for each non
central-unit $a$ in $R$, $a=b+c$ where $b^{p}=c^{q}=0$. The ring $R$ has the
$2$-nil-sum property of type $(p,\infty)$ if, for each non central-unit $a$ in
$R$, $a=b+c$ where $b^{p}=0$ and $c$ is nilpotent. Clearly, a ring $R$ is a
field iff $R$ has the $2$-nil-sum property of type $(1,1)$.

######  Proposition 2.3.

Suppose that $R$ has the $2$-nil-sum property of type $(2,\infty)$. Then $R$
is commutative.

###### Proof.

We observe that $R$ has no non-trivial idempotents: if $1\not=e^{2}=e\in R$ is
a sum of a square-zero element and a nilpotent it follows by [7, Proposition
2] that $e=0$.

Suppose that $R$ is not commutative. Then $R$ is a simple ring by Theorem 2.2,
hence $0$ is the only central nilpotent. But ${\rm nil}(R)\not=0$, hence there
exists a square-zero element $x$ that is not central. Since $R$ is of type
$(2,\infty)$, there exist $y,z\in R$ such that

$1-x=y+z,$

where $y^{2}=0$ and $z^{n}=0$ for some $n\geq 1$. Then $(1-x-y)^{n}=0$. It
follows that

$1=n_{1}(x+y)+n_{2}(xy+yx)+n_{3}(xyx+yxy)+n_{4}(xyxy+yxyx)+\cdots,$

where $n_{1}=n$ and each $n_{i}$ is an integer. Multiplying both sides of this
equation from the left by $x$ gives

$\begin{split}x&=n_{1}xy+n_{2}xyx+n_{3}xyxy+n_{4}xyxyx+\cdots\\\ \end{split}$

Multiplying both sides of this equality from the right by $y$ gives

$xy=n_{2}xyxy+n_{4}xyxyxy+\cdots=a(xy)^{2}=(xy)^{2}a,$

where $a=n_{2}+n_{4}xy+\cdots$. It follows that $a(xy)$ is an idempotent.
Since $x^{2}=0$, we have $a(xy)\not=1$. We obtain $a(xy)=0$, hence $xy=0$. It
follows that $x=0$, a contradiction. ∎

######  Proposition 2.4.

Suppose that $R$ is of characteristic 0 and has the $2$-nil-sum property of
type $(p,q)$ where $p\leq 3$ and $q\leq 5$. Then $R$ is commutative.

###### Proof.

We can assume that $p=3$ and $q=5$. Suppose that $R$ is not commutative. As in
the proof of Proposition 2.3, there exists a square-zero element $x$ that is
not central. So, there exist $y,z\in R$ such that $1-x=y+z$ and
$y^{3}=z^{5}=0$. This contradicts [16, Proposition 9]. ∎

We close this section with the following

Open Question. If a ring $R$ has the $2$-nil-sum property of type $(p,q)$ with
$p\in\mathbb{N}$ and $q\in\mathbb{N}\cup\\{\infty\\}$, is it commutative?

##  3\. Simple rings

######  Example 3.1.

There exists a simple ring with the $2$-nil-sum property that is not
commutative.

###### Proof.

Let $\mathbb{F}_{2}$ be the field of $2$ elements. For each integer $n\geq 0$
we consider the diagonal morphism

$\varphi_{n}:\mathbb{M}_{2^{n}}(\mathbb{F}_{2})\to\mathbb{M}_{2^{n+1}}(\mathbb{F}_{2}),A\mapsto\mathrm{diag}(A,A).$

This is an inductive system, and we denote its colimit by $R$, while
$\varphi_{n}:\mathbb{M}_{2^{n}}(\mathbb{F}_{2})\to R$ denote the canonical
ring morphisms. Then $R$ is a simple ring (see [8, Example 8.1]).

Let $0\neq r\in R$ a non central-unit of $R$. Then we can view it as an image
$r=\varphi_{n}(A)$ for some $0\neq A\in\mathbb{M}_{2^{n}}(\mathbb{F}_{2})$.
Note that $A\neq I_{2^{n}}$ since otherwise $r$ is the identity of $R$.
Observe that $r=\varphi_{n+1}(\mathrm{diag}(A,A))$. From $A\neq I_{2^{n}}$ it
follows that $\mathrm{diag}(A,A)$ is not a scalar matrix. Since its trace is
zero, there exist two nilpotent matrices $N_{1}$ and $N_{2}$ in
$\mathbb{M}_{2^{n+1}}(\mathbb{F}_{2})$ such that
$\mathrm{diag}(A,A)=N_{1}+N_{2}$, see [3, Proposition 3(ii)]. Then
$r=\varphi_{n+1}(N_{1})+\varphi_{n+1}(N_{2})$ is a sum of two nilpotents. ∎

Similar examples can be obtained by using other fields (of positive
characteristic). We do not know other kind of examples of non-commutative
rings with the $2$-nil-sum property. In fact, we will prove that if a simple
ring with the $2$-nil-sum property satisfies some standard finiteness
conditions then it is a field. A basic example is the following.

######  Example 3.2.

Let $F$ be a field and $n\geq 2$ an integer. Since the traces of all nilpotent
matrices in $\mathbb{M}_{n}(F)$ are $0$, it follows that $\mathbb{M}_{n}(F)$
does not have the $2$-nil-sum property. This can be easily extended to
matrices over commutative rings (since in this case the trace of a nilpotent
matrix has to be nilpotent).

Below we will study matrix rings over division rings. In this case, the above
argument does not work because nilpotent matrices over division rings may have
non-zero traces and, moreover, the trace is no longer invariant under
similarity. For instance, if $A=\begin{pmatrix}\bf i&\bf j\\\ -\bf j&\bf
i\end{pmatrix}\in{\mathbb{M}}_{2}(\mathbb{H})$, where $\mathbb{H}$ is the ring
of real quaternions, then $A^{2}=0$ and $\begin{pmatrix}1&0\\\ \bf
k&1\end{pmatrix}A\begin{pmatrix}1&0\\\ -\bf
k&1\end{pmatrix}=\begin{pmatrix}0&\bf j\\\ 0&0\end{pmatrix}$.

We start with some elementary lemmas. We include the details of the proofs for
the reader’s convenience.

######  Lemma 3.3.

Let $D$ be a division ring. If $A\in{\mathbb{M}}_{n}(D)$ is nilpotent, then
$A^{n}=0$.

###### Proof.

We identify $A$ with a linear transformation of the right vector space
$V:=D^{n}$ over $D$. Then the chain

$Im(A)\supset Im(A^{2})\supset Im(A^{3})\supset\cdots$

should be strictly decreasing (otherwise, $A$ is not nilpotent). Since ${\rm
dim}(V_{D})=n$, it follows that $A^{n}=0$. ∎

######  Lemma 3.4.

Let $R$ be a ring and $n\geq 2$. If
$A=\begin{pmatrix}a_{11}&\cdots&\cdots&a_{1n}\\\ 1&\ddots&\ddots&\vdots\\\
&\ddots&\ddots&\vdots\\\ &&1&a_{nn}\end{pmatrix}\in{\mathbb{M}}_{n}(R)$, then

$A^{k}=\left(\begin{array}[]{c|c}*&*\\\
\hline\cr\begin{matrix}\sum_{i=1}^{k}a_{ii}&\cdots&\cdots&\cdots\\\
1&\ddots&\ddots&\vdots\\\ &\ddots&\ddots&\vdots\\\
&&1&\sum_{i=n-k+1}^{n}a_{ii}\end{matrix}&*\end{array}\right),$

where the lower left block has size $(n-k+1)\times(n-k+1)$. So the
$(k,1)$-entry of $A^{k}$ is $a_{11}+\cdots+a_{kk}$ for $k=1,\ldots,n$; in
particular, the $(n,1)$-entry of $A^{n}$ is $a_{11}+\dots+a_{nn}$.

###### Proof.

The claim can be proved by induction on $k$. ∎

######  Lemma 3.5.

Let $D$ be a division ring and $n\geq 2$ an integer. If
$X=(x_{ij})\in{\mathbb{M}}_{n}(D)$ is a matrix such that the
$(n-1)$-dimensional column vector $\beta=\left[\begin{array}[]{c}x_{21}\\\
\vdots\\\ x_{n1}\\\ \end{array}\right]$ is not zero, then there exists an
invertible matrix $U=\begin{pmatrix}1&\bf 0\\\ \bf
0&V\end{pmatrix}\in{\mathbb{M}}_{n}(D)$ with $V\in{\mathbb{M}}_{n-1}(D)$, and
$1<k\leq n$ such that

$UXU^{-1}=\left(\begin{array}[]{c|c}\begin{matrix}w_{11}&\cdots&\cdots&w_{1k}\\\
1&\ddots&\ddots&\vdots\\\ &\ddots&\ddots&\vdots\\\ &&1&w_{kk}\end{matrix}&Y\\\
\hline\cr\bf 0&Z\end{array}\right).$

###### Proof.

Write $X=\begin{pmatrix}x_{11}&Y_{1}\\\ W_{1}&Z_{1}\end{pmatrix}$ in block
form where $Z_{1}$ is an $(n-1)\times(n-1)$ matrix. Since $W_{1}=\beta\not=0$
there exists an invertible matrix $V_{1}$ in ${\mathbb{M}}_{n-1}(D)$ such that
$V_{1}W_{1}=\left[\begin{array}[]{c}1\\\ 0\\\ \vdots\\\ 0\\\
\end{array}\right]$. Then $U_{1}=\begin{pmatrix}1&\bf 0\\\ \bf
0&V_{1}\end{pmatrix}$ is invertible in ${\mathbb{M}}_{n}(D)$ and
$U_{1}XU_{1}^{-1}=\begin{pmatrix}x_{11}&\gamma_{1}V_{1}^{-1}\\\
V_{1}W_{1}&V_{1}X_{1}V_{1}^{-1}\end{pmatrix}$, so the first column of
$U_{1}XU_{1}^{-1}$ is $\left[\begin{array}[]{c}x_{11}\\\ 1\\\ 0\\\ \vdots\\\
0\\\ \end{array}\right]$.

Write
$U_{1}XU_{1}^{-1}=\left(\begin{array}[]{c|c}\begin{matrix}y_{11}&y_{12}\\\
1&y_{22}\end{matrix}&Y_{2}\\\ \hline\cr W_{2}&Z_{2}\end{array}\right)$, where
$Z_{2}$ is an $(n-2)\times(n-2)$ matrix and the first column of $W_{2}$ is
$\bf 0$. If $W_{2}=\bf 0$ we can simply take $U=U_{1}$. If $W_{2}\not=\bf 0$,
there exists an invertible matrix $V_{2}$ in ${\mathbb{M}}_{n-2}(D)$ such that
$V_{2}W_{2}=\left[\begin{array}[]{cc}0&1\\\ 0&0\\\ \vdots&\vdots\\\ 0&0\\\
\end{array}\right]$. Observe that $U_{2}=\begin{pmatrix}I_{2}&\bf 0\\\ \bf
0&V_{2}\end{pmatrix}$ is invertible in ${\mathbb{M}}_{n}(D)$ and

$U_{2}U_{1}XU_{1}^{-1}U_{2}^{-1}=\left(\begin{array}[]{c|c}\begin{matrix}z_{11}&z_{12}&z_{13}\\\
1&z_{22}&z_{23}\\\ 0&1&z_{33}\end{matrix}&Y_{3}\\\ \hline\cr
W_{3}&Z_{3}\end{array}\right).$

The block consisting of the first two columns of
$U_{2}U_{1}XU_{1}^{-1}U_{2}^{-1}$ is of the form
$\left[\begin{array}[]{cc}z_{11}&z_{12}\\\ 1&z_{22}\\\ 0&1\\\ 0&0\\\
\vdots&\vdots\\\ 0&0\\\ \end{array}\right]$, so the first two columns of
$W_{3}$ are zero. If $W_{3}=\bf 0$, we can take $U=U_{2}U_{1}$. If
$W_{3}\not=\bf 0$, we continue in a similar way.

This process has to stop after a finite number of steps. Therefore, there
exists an integer $1<k\leq n$, and a family of invertible matrices
$U_{l}=\begin{pmatrix}I_{l}&\bf 0\\\ \bf 0&V_{l}\end{pmatrix}$, $1\leq l\leq
k-1$, such that

  * •

$(U_{l}\cdots U_{1})X(U_{l}\cdots
U_{1})^{-1}=\left(\begin{array}[]{c|c}\begin{matrix}w_{11}&w_{12}&\dots&w_{1l}&w_{1(l+1)}\\\
1&w_{22}&\dots&w_{2l}&w_{2(l+1)}\\\ \vdots&\vdots&&\vdots&\vdots\\\
0&0&\dots&1&w_{(l+1)(l+1)}\end{matrix}&X_{l+1}\\\ \hline\cr
W_{l+1}&Y_{l+1}\end{array}\right),$

  * •

the first $l$ columns of $W_{l+1}$ are zero,

  * •

$k=n$, or $k<n$ and $W_{k}=\bf 0$.

Now take $U=U_{k-1}\cdots U_{1}$. By the constructions of
$U_{1},\dots,U_{k-1}$, one easily sees that $U$ has the desired property. ∎

######  Remark 3.6.

The above lemma will be applied for the case when $X$ is nilpotent. We want to
point out that it was proved in [6] that every nilpotent matrix is similar to
a rational form (as in the commutative case). But we need the weaker property
presented above since we have to use the form of the matrix $U$.

######  Lemma 3.7.

Let $D$ be a division ring and $A=(a_{ij})\in{\mathbb{M}}_{n}(D)$ a matrix
with the property that the $k$-th row of $A$ is the only non-zero row of $A$.
Then $A$ is a sum of two nilpotents if and only if $a_{kk}=0$.

###### Proof.

The sufficienty is obvious. For the necessity, let us observe that $A$ is
similar to $A^{\prime}=\begin{pmatrix}a&\alpha\\\ \bf 0&\bf 0\end{pmatrix}$,
where $a=a_{kk}\in D$ and $\alpha$ is an $(n-1)$-dimensional row vector. Hence
$A^{\prime}=X+Y$, where $X,Y$ are nilpotents. Write
$X=\begin{pmatrix}x_{11}&\gamma\\\ \beta&X_{1}\end{pmatrix}$ in block form
where $X_{1}$ is an $(n-1)\times(n-1)$ matrix. If $\beta=0$, then
$a=x_{11}+(a-x_{11})$ is a sum of two nilpotents in $D$; so $a=0$. Thus, we
can assume that $\beta\not=0$. There exists an invertible matrix
$U=\begin{pmatrix}1&\bf 0\\\ \bf 0&V\end{pmatrix}\in\mathbb{M}_{n}(D)$ and an
integer $k>1$ as in Lemma 3.5 such that

$UXU^{-1}=\left(\begin{array}[]{c|c}\begin{matrix}w_{11}&\cdots&\cdots&w_{1k}\\\
1&\ddots&\ddots&\vdots\\\ &\ddots&\ddots&\vdots\\\ &&1&w_{kk}\end{matrix}&T\\\
\hline\cr\bf 0&Z\end{array}\right).$

Note that
$UA^{\prime}U^{-1}=\begin{pmatrix}a&\begin{pmatrix}a_{2}&\cdots&a_{n}\end{pmatrix}\\\
\bf 0&\bf 0\end{pmatrix}$ for some $a_{2},\ldots,a_{n}$ in $D$, and that
$UXU^{-1}$ and $UYU^{-1}$ are nilpotent matrices. It follows that the matrices

$\left(\begin{matrix}w_{11}&\cdots&\cdots&w_{1k}\\\ 1&\ddots&\ddots&\vdots\\\
&\ddots&\ddots&\vdots\\\ &&1&w_{kk}\end{matrix}\right)\text{ and
}\left(\begin{matrix}w_{11}-a&\cdots&\cdots&w_{1k}-a_{k}\\\
1&\ddots&\ddots&\vdots\\\ &\ddots&\ddots&\vdots\\\
&&1&w_{kk}\end{matrix}\right)$

are both nilpotent matrices. Using Lemma 3.3 and Lemma 3.4, we obtain the
equalities $w_{11}+\dots+w_{kk}=0=w_{11}+\dots+w_{kk}-a$, and hence $a=0$. ∎

######  Theorem 3.8.

Let $R$ be a simple right Goldie ring with the $2$-nil-sum property. Then $R$
is a field.

###### Proof.

Note that we can view $R$ as a right order of ${\mathbb{M}}_{n}(D)$ where
$n\geq 1$ and $D$ is a division ring. If $n=1$, then $R$ is a subring of a
division ring. Since ${\rm nil}(R)=0$, the only non central-unit of $R$ is
$0$, so $R$ is a field.

We next show that $n\geq 2$ yields a contradiction. Assume that $n\geq 2$. For
all $1\leq k\leq n$, we denote by $\mathcal{A}_{k}$ the set of all non-zero
matrices $A=(a_{ij})\in R$ with the property that the $k$-th row of $A$ is the
only non-zero row of $A$.

Since $R$ is a right order of ${\mathbb{M}}_{n}(D)$, for $1\leq k\leq n$ we
can write $E_{kk}=U_{k}S_{k}^{-1}$ where $U_{k},S_{k}\in R$ and where
$U_{k}=(u^{(k)}_{ij})$ and $S_{k}=(s^{(k)}_{ij})$. It follows that
$U_{k}=E_{kk}S_{k}\in\mathcal{A}_{k}$, so all sets $\mathcal{A}_{k}$ are non-
empty.

Moreover, if $1\leq k\leq n$, we denote by $i(k)$ the minimal integer with the
property that there exists $(a_{ij})\in\mathcal{A}_{k}$ such that $a_{kl}=0$
for all $l<i(k)$ and $a_{ki(k)}\neq 0$.

We show that if $k<i(k)$ then $i(k)<i(i(k))$. In order to prove this, let
$A=(a_{ij})\in\mathcal{A}_{k}$ be a matrix such that $a_{kl}=0$ for all
$l<i(k)$ and $a:=a_{ki(k)}\neq 0$. For any $B=(b_{ij})\in\mathcal{A}_{i(k)}$,
the $k$-th row of $AB$ is $(ab_{i(k)1},\dots,ab_{i(k)i(k)},\dots,ab_{i(k)n})$
and this is the only possible non-zero row of $AB$. Since $a\neq 0$ it
follows, by the minimality of $i(k)$, that
$b_{i(k)1}=\dots=b_{i(k)(i(k)-1)}=0$. Moreover, by Lemma 3.7 it follows that
$b_{i(k)i(k)}=0$ as $B\in\mathcal{A}_{i(k)}$. Hence, $i(k)<i(i(k))$.

Now we close the proof by observing that $1<i(1)$, hence we obtain a strictly
increasing infinite sequence bounded above by $n$: $1<i(1)<i(i(1))<\cdots$.
This is a contradiction. ∎

######  Remark 3.9.

By Theorem 3.8, for a division ring $D$, the ring $\mathbb{M}_{n}(D)$
satisfies the $2$-nil-sum property iff $n=1$ and $D$ is a field. Note that
there exists a division ring $D$ such that all matrices in
${\mathbb{M}}_{3}(D)$ are sums of three nilpotent matrices (see [15]).

We close the paper, with some applications of Theorem 2.2 and Theorem 3.8. A
nonzero right ideal $I$ of a ring $R$ is called a uniform right ideal if the
intersection of any two nonzero right ideals contained in $I$ is nonzero. A
ring $R$ is semipotent if every nonzero right ideal not contained in $J(R)$
contains a nonzero idempotent. A ring $R$ is said to be of bounded index (of
nilpotence) if there is a positive integer $n$ such that $a^{n}=0$ for all
nilpotents $a$ of $R$.

######  Corollary 3.10.

A ring $R$ with the $2$-nil-sum property is commutative under any of the
following additional assumptions:

  1. (1)

$R$ contains a uniform ideal;

  2. (2)

$R$ is a semipotent ring of bounded index;

  3. (3)

$R$ is a semilocal ring.

###### Proof.

By Theorem 2.2, suppose that $R$ is a simple ring.

(1) It follows from [9] that $R$ contains no infinite direct sums of right
ideals (i.e., $R_{R}$ is of finite uniform dimension). Moreover, from the last
remark of [12] it follows that the maximal right ring of quotients of $R$ is
simple artinian. Thus, $R$ is right Goldie by [13, Proposition 13.41].

(2) If $R$ is semipotent of bounded index, then $R$ is simple artinian by [1,
Corollary 6].

(3) If $R$ is semilocal, then $R$ is simple artinian.

So the claim follows from Theorem 3.8. ∎

## Acknowledgment

Breaz was supported by a grant of the Ministry of Research, Innovation and
Digitization, CNCS/CCCDI – UEFISCDI, project number PN-III-P4-ID-
PCE-2020-0454, within PNCDI III. Zhou was supported by a Discovery Grant
(RGPIN-2016-04706) from NSERC of Canada.

## References

  * [1] K.I. Beidar, On rings with zero total, Beiträge Algebra Geom. 38(2) (1997), 233–239.
  * [2] F. Barroero, C. Frei and R.F. Tichy, Additive unit representations in global fields – a survey, Publ. Math. Debrecen 79(3-4) (2011), 291–307.
  * [3] S. Breaz and G. Călugăreanu, Sums of nilpotent matrices, Linear Multilinear Alg. 65(1) (2017), 67–78.
  * [4] G. Călugăreanu and T.Y. Lam, Fine rings: A new class of simple rings, J. Alg. Appl. 15(9) (2016), 18 pp.
  * [5] M. Chebotar, P.-H. Lee and E.R. Puczyłowski, On commutators and nilpotent elements in simple rings, Bull. London Math. Soc. 42 (2010) 191–194.
  * [6] P.M. Cohn, The similarity reduction of matrices over a skew field, Math. Z. 132 (1973), 151–163.
  * [7] M. Ferrero, E.R. Puczyłowski and S. Sidki, On the representation of an idempotent as a sum of nilpotent elements, Canad. Math. Bull. 39 (2) (1996), 178–185.
  * [8] K.R. Goodearl, Ring Theory: Nonsingular Rings and Modules, (Marcel Dekker, Inc, 1976).
  * [9] R. Hart, Simple rings with uniform right ideals, Journal of London Math. Soc. 42 (1967), 614-617.
  * [10] B. Harris, Commutators in division rings, Proc. Am. Math. Soc. 4 (1958), 628–630.
  * [11] D. Khurana and A.K. Srivastava, Right self-injective rings in which every element is a sum of two units, J. Algebra Appl.  6 (2) (2007), 281-286.
  * [12] Kwangil Koh, On simple rings with uniform right ideals, Amer. Math. Monthly 74 (1967), 685-687.
  * [13] T.Y. Lam, Lectures on Modules and Rings, Graduate Texts in Mathematics, 189. (Springer-Verlag, New York, 1999).
  * [14] C. Meehan, Sums of automorphisms of free modules and completely decomposable groups, J. Alg. 299 (2006), 467-479.
  * [15] A. Salwa, Simple artinian rings as sums of nilpotent subrings, Arch. Math. 78 (2002) 350–357.
  * [16] A. Salwa, Representing idempotents as a sum of two nilpotents - an approach via matrices over division rings, Colloq. Math. 77(1) (1998), 59–83.
  * [17] A.K. Srivastava, A survey of rings generated by units, Ann. Fac. Sci. Toulouse Math. 19 (2010), 203-213.
  * [18] P. Vámos, $2$-Good rings, Quart. J. Math. 56 (2005), 417-430.
  * [19] K.G. Wolfson, An ideal-theoretic characterization of the ring of all linear transformations, Amer. J. Math. 75 (1953), 358-386.
  * [20] D. Zelinsky, Every linear transformation is a sum of nonsingular ones, Proc. AMS. 5 (1954), 627-630.


The purpose of this section is to explain our proposed method for solving the problem outlined in Section <ref>. We coin this novel algorithm DePAint (Decentralized Policy Gradient considering Peak and Average Constraints). This section is primarily divided into two parts; the first part discussed in Section <ref> outlines how we convert the problem stated in Section <ref> to an equivalent yet simpler form, and in Section <ref> we explain the steps in our algorithm which tries to tackle the aforementioned problem.

§.§ Preliminaries

In order to tackle the problem at hand, we transform it into an equivalent simpler problem. At first, in Section <ref>, we introduce a way to approximate the agent's policy to handle large state and action spaces properly. Secondly, in Section <ref>, we augment the long-term utility function so that it accounts for both the average and peak constraint, leaving us with only one constraint to deal with. Finally, in Section <ref>, using the method of Lagrange multipliers, we convert the constrained maximization problem into a min-max optimization problem.

§.§.§ Deep Policy Gradient

The space of all possible policy functions can be rather large; as a result, it is intractable to exactly define the policy function. As such we use parameterized function approximators such as deep neural networks to represent each agent's policy. Using a deep neural network as a policy function approximator reduces the problem of finding the optimal policy to finding the optimal parameters for the policy function approximator i.e. the weights of the neural network. So, $\pi_i: S \rightarrow \Delta(A)$ is parameterized as $\pi_{\theta_{[i]}}$ by a parameter $\theta_{[i]} \in \R^{d_i}$ with dimension $d_i$. So, we can define the joint policy as shown in Equation <ref> where, $\theta = [\theta_{[1]}^T\ ...\ \theta_{[n]}^T]^T \in \R^d$ and $d = \sum_{i = 1}^n d_i$.

\begin{equation}
\pi_{\theta}(a^t | s^t) := \prod_{i = 1}^n \pi_{\theta_{[i]}}(a_i^t | s^t) \label{eq:param_policy}
\end{equation}

As the agents are required to train independently, each agent $i$ keeps a local version of the parameter $\theta$, named $\theta_i$.
Based on this we can modify Equations <ref>, <ref> and <ref> as,

\begin{align}
p(\tau | \theta) := \rho(s^0) \prod_{t=0}^\infty \pi_\theta(a^t | s^t) P(s^{t+1}|s^t, a^t) \label{eq:traj_prob_theta}\\
J_i^{R}(\theta) := \underset{\tau \sim p(.|\theta)}{\E} \left[\sum_{t = 0}^{\infty} \gamma^t R_i (s^t, a^t) \right]
\label{eq:disc_reward_theta}\\
J_i^{C}(\theta) := \underset{\tau \sim p(.|\theta)}{\E} \left[\sum_{t = 0}^{\infty} \gamma^t C_i (s^t, a^t) \right].
\label{eq:disc_util_theta}
\end{align}

That leads to the problem of finding the optimal policy $\pi$ defined in Section <ref> becoming a problem of finding the optimal parameters $\theta$ for which,

\begin{align}
& \underset{\theta \in \R^d}{\max}\ \frac{1}{n} \sum_{i=0}^n J_i^{R}(\theta) \label{eq:prob_theta}\\
\text{subject to }\ & J_i^{C}(\theta) \ge c_i & \forall i \in \N \label{eq:avg_const_theta}\\
\text{and }\ & K_i(s^t, a^t) \ge k_i & \forall i \in \N, (s^t, a^t) \in p(.|\theta). \label{eq:peak_const_theta}
\end{align}

§.§.§ Dealing with Peak Constraint

Dealing with two types of constraints simultaneously is difficult because peak and average constraints have pretty diverse forms.  We solve this problem by augmenting the average constraint to include the peak constraint.

We define the new long-term utility function as Equation <ref> where $C_i(s, a)$ is the long-term utility function we defined in Section <ref>, $M$ is a large number (greater than $max\ J_i^C - c_i$), and $I_i$ is an identity function defined in Equation <ref>.

\begin{equation}
\widehat{C}_i(s, a) := C_i(s, a) - M I_i(s, a) \label{eq:new_peak}.
\end{equation}

\begin{equation}
I_i(s, a) :=
\begin{cases}
\mathbf{1}_{m_1},& \text{if } K_i(s, a) < k_i\\
0,              & \text{otherwise} \label{eq:new_peak_identity}
\end{cases}
\end{equation}

For a finite horizon, $max\ J_i^C - c_i$ is bounded. As such, we can always define a suitable $M$ for a given environment. This formulation ensures the fact that when the peak constraint is violated i.e. $K_i(s, a) < k_i$, the new average constraint is also violated, $\widehat{C}_i(s, a) < c_i$. As a result, we no longer need to handle the peak constraint explicitly. We can also define a new $J_i^{\widehat{C}}(\pi)$ based on the newly defined $\widehat{C}$.

\begin{equation}
J_i^{\widehat{C}}(\pi) := \underset{\tau \sim p(.|\pi)}{\E} \left[\sum_{t = 0}^{\infty} \gamma^t \widehat{C}_i (s^t, a^t) \right]
\label{eq:new_lt_util}
\end{equation}

From this point onward, for the sake of simplicity, we are going to use $C$ and $J_i^{C}(\pi)$ to refer to $\widehat{C}$ and $J_i^{\widehat{C}}(\pi)$ respectively.

§.§.§ Primal Dual Formulation

Now that we only have to deal with the average constraint, the original problem can be written as,

\begin{align}
\underset{\theta \in \Theta}{\text{max}}\ \frac{1}{n} \sum_{i=1}^{n} J_i^{R}(\theta) \ \  \text{subject to} \ \ J_i^{C}(\theta) \ge c_i \ \ \forall i \in \mathcal{N}. \label{eq:no_peak_cmdp}
\end{align}

Maximizing the objective while dealing with the constraints is not a trivial task. Therefore as discussed in Section <ref> using the method of Lagrange multipliers, we can formulate this constrained maximization problem as defined in Expression <ref> into an equivalent unconstrained min-max problem for the associated Lagrangian $J^{L}(\theta, \lambda_1, ..., \lambda_n)$ in Expression <ref> where $\theta$ is the primal variable and $\lambda_1, ..., \lambda_n$ are the non-negative Lagrange multipliers or dual variables.

\begin{align}
\underset{\theta \in \Theta}{\text{max}}\
\underset{\{\lambda_i \ge 0\}}{\text{min}}\
J^{L}(\theta, \lambda_1, ..., \lambda_n) := \frac{1}{n} \sum_{i=1}^{n} J_i^{R}(\theta) + \sum_{i=1}^{n} \lambda_i(J_i^{C}(\theta) - c_i) \label{eq:multi_primal_dual}
\end{align}

Now the problem is reduced to maximizing on the primal variable $\theta$ and minimizing on the dual variables $\{\lambda_i\}_{i=1}^n$. That enables us to use gradient descent ascent based methods to solve the problem.

§.§ The Algorithm Description

Now that we have transformed the problem into a more suitable form, we explain our proposed algorithm for solving this problem. The steps of the algorithm are outlined in the pseudocode defined in Algorithm <ref>. As our system includes multiple agents, for simplicity, we show only the procedure for a single agent $i$. This is executed by all agents in parallel.

number of iterations $T$, horizon $H$, batch size $B$, primal learning rate $\eta_1$, dual learning rate $\eta_2$, primal momentum parameter $\beta_1$, dual momentum parameter $\beta_2$, initial policy parameter $\theta_i^0 \in \mathbb{R}^d$, initial dual parameter $\lambda_i^0$, weight matrix $W$
optimal policy parameter $\theta_i^{T+1}$

Sample $B$ finite-horizon trajectories $\{\tau_i^b\}_{b=1}^B$ of size $H$ from $p(.|\theta_i^0)$
Update critic networks using $\{\tau_i^b\}_{b=1}^B$
$u_i^0, v_i^0 \gets$ $\{\tau_i^b\}_{b=1}^B$
$x_i^1 \gets$ $W, \{0, u_j^0, 0\}_{j \in \mathcal{N}_i}$
$y_i^1 \gets$ $W, \{0, v_j^0, 0\}_{j \in \mathcal{N}_i}$
$t\leftarrow 1$ $T$
Sample $B$ finite-horizon trajectories $\{\tau_i^b\}_{b=1}^B$ of size $H$ from $p(.|\theta_i^0)$
Update critic networks using $\{\tau_i^b\}_{b=1}^B$
$u_i^t, v_i^t \gets$ $\{\tau_i^b\}_{b=1}^B, \beta_1, \beta_2, u_i^{t-1}, v_i^{t-1}$
$x_i^{t+1} \gets$ $W, \{x_j^t, u_j^t, u_j^{t-1}\}_{j \in \mathcal{N}_i}$

$y_i^{t+1} \gets$ $W, \{y_j^t, v_j^t, v_j^{t-1}\}_{j \in \mathcal{N}_i}$

$\theta_i^{t+1}, \lambda_i^{t+1} \gets $ $\eta_1, \eta_2, W, \{\theta_j^t, \lambda_j^t, x_j^{t+1}, y_j^{t+1}\}_{j \in \mathcal{N}_i}$

The algorithm primarily consists of the initialization part and the main loop. The main loop consists of four parts. The first part is the exploration defined in line <ref>, which consists of the agent generating some finite trajectories from the given environment. Then, in the second step in line <ref>, we use these trajectories to update the critic network. Thirdly, in line <ref>, we generate gradient estimators for the Lagrangian function with respect to our $\theta$ and $\lambda$. And finally in lines <ref>, <ref> and <ref>, we update the parameters ($\theta$ and $\lambda$) and communicate with neighbouring agents to establish consensus. The initialization process is very similar to the steps inside the main loop, with slight differences.

In the following sections, we explain the gradient estimation process and the parameter update step in more detail.

§.§.§ Gradient Estimation

In light of Equation <ref>, the required gradients $\nabla_{\theta_i}\ J_i^L(\theta_i^t, \lambda_i^t)$ and $\nabla_{\lambda_i}\ J_i^L(\theta_i^t, \lambda_i^t)$ can be derived as follows,

\begin{align}
\nabla_{\theta_i}\ J_i^L(\theta_i, \lambda_i) = \nabla_{\theta_i} J_i^{R}(\theta_i) + \lambda \nabla_{\theta_i} J_i^{C}(\theta_i) \label{eq:lagrange_grad_theta}\\
\nabla_{\lambda_i}\ J_i^L(\theta_i, \lambda_i) = J_i^{C}(\theta_i) - c_i \label{eq:lagrange_grad_lambda}
\end{align}

For estimating $\nabla_{\theta_i}\ J_i^L(\theta_i, \lambda_i)$ we need to first estimate $\nabla_{\theta_i} J_i^{R}(\theta_i)$ and $\nabla_{\theta_i} J_i^{C}(\theta_i)$.

Without having full knowledge about the dynamics of the environment beforehand, we have no way to calculate the exact value of $\nabla_{\theta_i} J_i^{R}(\theta_i)$ and $\nabla_{\theta_i} J_i^{C}(\theta_i)$. To estimate $\nabla_{\theta_i} J_i^{R}(\theta_i)$ and $\nabla_{\theta_i} J_i^{C}(\theta_i)$, we adopt the REINFORCE policy gradient estimator we previously touched upon on Section <ref>. However, REINFORCE is known to be prone to high variance. So we also use baselines in conjunction with REINFORCE similar to an Actor-Critic framework as discussed in Section <ref> to reduce variance and increase sample efficiency. Let's use the baselines $\hat{b}_i^R$ and $\hat{b}_i^C$ for $\nabla_{\theta_i} J_i^{R}(\theta_i)$ and $\nabla_{\theta_i} J_i^{C}(\theta_i)$ respectively. Therefore the gradient estimates will be as defined in Equation <ref> and <ref> ($\widehat{\nabla}$ is used to express a gradient estimate) where $\tau := (s_0, a_0, s_1, ..., )$ represents an infinite trajectory produced under the policy $\pi_{\theta_i}$.

\begin{align}
\widehat{\nabla}_{\theta} J_i^{R}(\theta_i) := \left[ \sum^{\infty}_{h=0} \nabla_{\theta_i} \log \pi_{\theta_i}(a^h|s^h)\right] \cdot \left[  \sum^{\infty}_{h=0}\gamma^h R_i(a^h, s^h) - \hat{b}_i^R\right] \label{eq:obj_grad_est_inf}\\
\widehat{\nabla}_{\theta} J_i^{C}(\theta_i) := \left[ \sum^{\infty}_{h=0} \nabla_{\theta_i} \log \pi_{\theta_i}(a^h|s^h)\right] \cdot \left[  \sum^{\infty}_{h=0}\gamma^h C_i(a^h, s^h) - \hat{b}_i^C\right] \label{eq:util_grad_est_inf}
\end{align}

For the baseline, the state-value function is used. The state-value function for $R_i$ and $C_i$ may be defined as in Equation <ref> and <ref>. $V_{i}^{R}(s)$ and $V_{i}^{C}(s)$ are usually approximated using separate neural networks, which are trained simultaneously with the policy network in lines <ref> and <ref>. Let us term these as critic networks and denote them as $\widehat{V}_{i}^{R}(s)$ and $\widehat{V}_{i}^{C}(s)$.

\begin{align}
V_{i}^{R}(s) := \underset{s^0 = s,\ \tau \sim p(.|\theta_i)}{\mathbb{E}} \left[\sum_{t=0}^\infty \gamma^t R_i(s^t, a^t) \right] \label{eq:state_value_r}\\
V_{i}^{C}(s) := \underset{s^0 = s,\ \tau \sim p(.|\theta_i)}{\mathbb{E}} \left[\sum_{t=0}^\infty \gamma^t C_i(s^t, a^t) \right] \label{eq:state_value_c}
\end{align}

Now, although Equations <ref> and <ref> give us a reasonable estimate of the required gradients, we cannot calculate them in practice due to requiring an infinite trajectory. Therefore, we use finite horizon approximation with horizon $H$. Moreover, to further reduce variance, we can use a minibatch of $B$ trajectories $\{\tau_i^b\}_{b=1}^B$ instead of a single trajectory and take the average gradient over them. As a result, with mini-batch and finite-horizon estimation, Equations <ref> and <ref> become

\begin{align}
\widehat{\nabla}_{\theta} J_i^{R}(\theta_i) := \frac{1}{B} \sum_{b=1}^B\ \left[ \sum^{H-1}_{h=0} \nabla_{\theta_i} \log \pi_{\theta_i}(a^{h, b}|s^{h, b})\right] \cdot
\left[  \sum^{H-1}_{h=0}\gamma^h R_i(a^{h, b}, s^{h, b}) - \widehat{V}_{i}^{R}(s^{h, b})\right] \label{eq:obj_grad_est}\\
\widehat{\nabla}_{\theta} J_i^{C}(\theta_i) := \frac{1}{B} \sum_{b=1}^B\ \left[ \sum^{H-1}_{h=0} \nabla_{\theta_i} \log \pi_{\theta_i}(a^{h, b}|s^{h, b})\right] \cdot
\left[  \sum^{H-1}_{h=0}\gamma^h C_i(a^{h, b}, s^{h, b}) - \widehat{V}_{i}^{C}(s^{h, b})\right] \label{eq:util_grad_est}
\end{align}

Therefore, according to Equation <ref> we can calculate the gradient estimate $\widehat{\nabla}_{\theta_i}\ J_i^L(\theta_i, \lambda_i)$ as,

\begin{align}
\widehat{\nabla}_{\theta_i}\ J_i^L(\theta_i, \lambda_i) = \widehat{\nabla}_{\theta_i} J_i^{R}(\theta_i) + \lambda \widehat{\nabla}_{\theta_i} J_i^{C}(\theta_i). \label{eq:lagrange_grad_theta_est}
\end{align}

On the other hand, we can also calculate an estimate of $\nabla_{\lambda_i}\ J_i^L(\theta_i^t, \lambda_i^t)$ by calculating an approximation of $J_i^{C}(\theta_i)$ using the minibatch and finite-horizon techniques similar to Equation <ref> and <ref>. Given a batch of $B$ finite-horizon trajectories of size $H$ $\{\tau_i^b\}_{b=1}^B$, we can define $\nabla_{\lambda_i}\ J_i^L(\theta_i^t, \lambda_i^t)$ as in Equation <ref>, where $\widehat{J_i^{C}}(\theta_i)$ is used to indicate an estimate of $J_i^{C}(\theta_i)$.

\begin{align}
\widehat{J_i^{C}}(\theta_i) := \frac{1}{B}\sum_{b=1}^{B}\sum_{h = 0}^{H-1} \gamma^h C_i (s^{h, b}, a^{h, b})\\
\widehat{\nabla}_{\lambda_i}\ J_i^L(\theta_i, \lambda_i) := \widehat{J_i^{C}}(\theta_i) - c_i \label{eq:lagrange_grad_lambda_est}
\end{align}

Momentum Based Variance Reduction
If the estimates from Equations <ref> and <ref> are directly applied in the gradient descent process (discussed in the next section) the high variance results in slow convergence. To further reduce the variance, we apply the momentum-based variance reduction technique <cit.>. The momentum technique ensures the stability of the gradient estimates by smoothing its values across multiple learning steps. Let us use $u_i^t$ and $v_i^t$ to denote the REINFORCE based gradient estimates of $\nabla_{\theta_i}\ J_i^L(\theta_i, \lambda_i)$ and $\nabla_{\lambda_i}\ J_i^L(\theta_i, \lambda_i)$ respectively at time $t$. We use $\widehat{u}_i^t$ and $\widehat{v}_i^t$ to represent their momentum-based counterparts. Now, the modified gradient estimator according to the aforementioned technique can be written as:

\begin{align}
\widehat{u}_i^t &:= \beta_1\, u_i^t + (1-\beta_1) (\widehat{u}_i^{t-1} + u_i^t - u_i^{t-1})\\
\widehat{v}_i^t &:= \beta_2\, v_i^t + (1-\beta_2) (\widehat{v}_i^{t-1} + v_i^t - v_i^{t-1})
\end{align}

Here $\beta_1,\beta_2 \in (0, 1]$ represents the momentum parameters. In order for this method to work properly, the gradient estimators need to be unbiased. However, in Equations <ref>, <ref>, the distribution $p(.|\theta_i^t)$ controls the sampled trajectory $\tau_i^t$. It is clear that the estimator $g_i(\tau_i^t|\theta_i^{t-1})$ is biased with respect to $\nabla V(\theta_i^{t-1})$. We can use the importance sampling technique to ensure the unbiased property

\begin{equation}
\mathbb{E}_{\tau_i^t\mathtt{\sim} p(\cdot|\theta_i^t)}\{\omega(\tau_i^t|\theta_i^{t-1},\theta_i^{t})g_i(\tau_i^t|\theta_i^{t-1}) \} = \nabla V_i(\theta_i^{t-1})
\end{equation}

Here, importance weight is denoted as $\omega(\tau_i^t|\theta_i^{t-1},\theta_i^{t-1})$. It's defined as:

\begin{equation}
\omega(\tau_i^t|\theta_i^{t-1},\theta_i^{t}) := \frac{p(\tau_i^t|\theta_i^{t-1})}{p(\tau_i^t|\theta_i^t)} = \prod_{h=1}^{H-1} \frac{\pi_{\theta_i^{t-1}}(a^h|s^h)}{\theta_i^{t}(a^h|s^h)}
\end{equation}

So, the momentum-based variance reduction for the policy gradient of the i-th agent is given by:

\begin{align}
\widehat{u}_i^t &:= \beta_1\,u_i^t + (1-\beta_1) (\widehat{u}_i^{t-1} + u_i^t - \omega(\theta_i^{t-1},\theta_i^{t})\cdot u_i^{t-1} ) \label{eq:momentum_obj_grad}\\
\widehat{v}_i^t &:= \beta_2\,v_i^t + (1-\beta_2)(\widehat{v}_i^{t-1} + v_i^t - \omega(\theta_i^{t-1},\theta_i^{t})\cdot v_i^{t-1}) \label{eq:momentum_util_grad}
\end{align}

In light of the above discussion, the function $EstimateGradients$ used in lines <ref> and <ref> can be defined as in Procedure <ref>.

$B$ finite-horizon trajectories of size $H$ $\{\tau_i^b\}^B_{b=1}$
Gradient estimates of $\nabla_{\theta_i}\ J_i^L(\theta_i^t, \lambda_i^t)$ and $\nabla_{\lambda_i}\ J_i^L(\theta_i^t, \lambda_i^t)$
Calculate REINFORCE gradient estimates $\widehat{\nabla}_{\theta} J_i^{R}(\theta_i)$ and $\widehat{\nabla}_{\theta} J_i^{C}(\theta_i)$ from $\{\tau_i^b\}^B_{b=1}$ using Equations <ref> and <ref> respectively
Calculate $\widehat{\nabla}_{\theta_i}\ J_i^L(\theta_i, \lambda_i)$ and $\widehat{\nabla}_{\lambda_i}\ J_i^L(\theta_i, \lambda_i)$ using Equations <ref> and <ref>, and denote them as $u_i^t$ and $v_i^t$ respectively
Calculate momentum based estimates $\widehat{u}_i^t$ and $\widehat{v}_i^t$ using Equations <ref> and <ref> respectively
$\widehat{u}_i^t, \widehat{v}_i^t$

§.§.§ Decentralized Optimization

The Expression <ref> would be straightforward to solve if it were in a centralized setting. However, our setting is decentralized, where the agents are connected by a communication network.

We can reformulate the problem as a decentralized min-max optimization problem of the form <ref> where $J_i^{L}(\theta_i, \lambda_i) = J_i^{R}(\theta_i) + \lambda_i(J_i^{C}(\theta_i) - c)$ and $\mathcal{N}_i$ denotes the set of neighbors of agent $i$.

\begin{align}
\underset{\{\theta_i \in \Theta\}}{\text{max}}\
\underset{\{\lambda_i \ge 0\}}{\text{min}}\
\frac{1}{n} \sum_{i=1}^{n} J_i^{L}(\theta_i, \lambda_i) \label{eq:multi_primal_dual_short}\\
\text{s.t.}\quad \theta_i = \theta_j\quad j \in \mathcal{N}_i,\ \forall i \nonumber
\end{align}

This is similar to the problem discussed in Section <ref>. So we can use stochastic gradient descent ascent and gradient tracking as discussed there, resulting in the update steps,

\begin{align}
x_i^{t+1} := \sum_{j \in \mathcal{N}_i}\ W_{ij} (x_j^t + \nabla_{\theta_i}\ J_i^L(\theta_i^t, \lambda_i^t) - \nabla_{\theta_i}\ J_i^L(\theta_i^{t-1}, \lambda_i^{t-1})), \label{eq:theta_tracking}\\
y_i^{t+1} := \sum_{j \in \mathcal{N}_i}\ W_{ij} (y_j^t + \nabla_{\lambda_i}\ J_i^L(\theta_i^t, \lambda_i^t) - \nabla_{\lambda_i}\ J_i^L(\theta_i^{t-1}, \lambda_i^{t-1})), \label{eq:lambda_tracking}\\
\theta_i^{t+1} := \sum_{j \in \mathcal{N}_i}\ W_{ij} (\theta_j^t + \eta_1 x_j^{t+1}), \label{eq:theta_update}\\
\lambda_i^{t+\frac{1}{2}} := \sum_{j \in \mathcal{N}_i}\ W_{ij} (\lambda_j^t - \eta_2 y_j^{t+1}),\quad
\lambda_i^{t+1} := \mathcal{P}_\Lambda(\lambda_i^{t+\frac{1}{2}}). \label{eq:lambda_update}
\end{align}

Here, $x_i$, $y_i$ are the gradient tracking variables for $\theta_i$ and $\lambda_i$, respectively. Note that Equation <ref> consists of a projection step to ensure that the updated $\lambda$ stays in the non-negative $\Lambda$ space.

Lines <ref> and <ref> in the main loop, as well as lines <ref> and <ref> in the initialization part constitute the gradient tracking part. While line <ref> refers to the actual parameter update step. The procedures $UpdateTracking$ and $UpdateParams$ may be formally defined as in Procedure <ref>.

weight matrix $W$, {current tracking variable $x_j^t$, current gradient estimate $u_j^t$, previous gradient estimate $u_j^{t-1}$} $\forall j \in \mathcal{N'}$ (where $\mathcal{N'} \subset \mathcal{N}$)
next tracking variable for current agent $x_i^{t+1}$

$x_i^{t+1} \gets \sum_{j \in \mathcal{N'}}\ W_{ij} (x_j^t + u_j^t - u_j^{t-1})$

primal learning parameter $\eta_1$, dual learning parameter $\eta_2$, weight matrix $W$, { current parameter primal parameter $\theta_j^t$, current dual parameter $\lambda_j^t$, next primal tracking variable $x_j^{t+1}$, next dual tracking variable $y_j^{t+1}$ } $\forall j \in \mathcal{N'}$ (where $\mathcal{N'} \subset \mathcal{N}$)
next primal parameter for current agent $\theta_i^{t+1}$, next dual parameter for current agent $\lambda_i^{t+1}$

$\theta_i^{t+1} \gets \sum_{j \in \mathcal{N}_i}\ W_{ij} (\theta_j^t + \eta_1 x_j^{t+1})$
$\lambda_i^{t+\frac{1}{2}} \gets \sum_{j \in \mathcal{N'}}\ W_{ij} (\lambda_j^t - \eta_2 y_j^{t+1})$
$\lambda_i^{t+1} \gets \mathcal{P}_\Lambda(\lambda_i^{t+\frac{1}{2}})$

$\theta_i^{t+1}, \lambda_i^{t+1}$


Das0000-0002-9770-1377, G. Dezoort, P. Elmer0000-0001-6830-3356, A.
Frankenthal0000-0002-2583-5982, B. Greenberg0000-0002-4922-1934, N. Haubrich,
S. Higginbotham, A. Kalogeropoulos0000-0003-3444-0314, G. Kopp, S.
Kwan0000-0002-5308-7707, D. Lange, M.T. Lucchini0000-0002-7497-7450, D.
Marlow0000-0002-6395-1079, K. Mei0000-0003-2057-2025, I. Ojalvo, J.
Olsen0000-0002-9361-5762, C. Palmer0000-0003-0510-141X, D.
Stickland0000-0003-4702-8820, C. Tully0000-0001-6771-2174 University of Puerto
Rico, Mayaguez, Puerto Rico, USA S. Malik0000-0002-6356-2655, S. Norberg
Purdue University, West Lafayette, Indiana, USA A.S. Bakshi, V.E.
Barnes0000-0001-6939-3445, R. Chawla0000-0003-4802-6819, S.
Das0000-0001-6701-9265, L. Gutay, M. Jones0000-0002-9951-4583, A.W.
Jung0000-0003-3068-3212, S. Karmarkar, M. Liu, G. Negro, N.
Neumeister0000-0003-2356-1700, G. Paspalaki, C.C. Peng, S.
Piperov0000-0002-9266-7819, A. Purohit, J.F. Schulte0000-0003-4421-680X, M.
Stojanovic15, J. Thieman0000-0001-7684-6588, F. Wang0000-0002-8313-0809, R.
Xiao0000-0001-7292-8527, W. Xie0000-0003-1430-9191 Purdue University
Northwest, Hammond, Indiana, USA J. Dolen0000-0003-1141-3823, N. Parashar Rice
University, Houston, Texas, USA A. Baty0000-0001-5310-3466, M. Decaro, S.
Dildick0000-0003-0554-4755, K.M. Ecklund0000-0002-6976-4637, S. Freed, P.
Gardner, F.J.M. Geurts0000-0003-2856-9090, A. Kumar0000-0002-5180-6595, W. Li,
B.P. Padley0000-0002-3572-5701, R. Redjimi, W. Shi0000-0002-8102-9002, A.G.
Stahl Leiton0000-0002-5397-252X, S. Yang0000-0002-2075-8631, L. Zhang, Y.
Zhang0000-0002-6812-761X University of Rochester, Rochester, New York, USA A.
Bodek0000-0003-0409-0341, P. de Barbaro, R. Demina0000-0002-7852-167X, J.L.
Dulemba0000-0002-9842-7015, C. Fallon, T. Ferbel0000-0002-6733-131X, M.
Galanti, A. Garcia-Bellido0000-0002-1407-1972, O.
Hindrichs0000-0001-7640-5264, A. Khukhunaishvili, E. Ranken, R. Taus Rutgers,
The State University of New Jersey, Piscataway, New Jersey, USA B. Chiarito,
J.P. Chou0000-0001-6315-905X, A. Gandrakota0000-0003-4860-3233, Y.
Gershtein0000-0002-4871-5449, E. Halkiadakis0000-0002-3584-7856, A. Hart, M.
Heindl0000-0002-2831-463X, E. Hughes, S. Kaplan, O.
Karacheban230000-0002-2785-3762, I. Laflotte, A. Lath0000-0003-0228-9760, R.
Montalvo, K. Nash, M. Osherson, S. Salur0000-0002-4995-9285, S. Schnetzer, S.
Somalwar0000-0002-8856-7401, R. Stone, S.A. Thayil0000-0002-1469-0335, S.
Thomas, H. Wang0000-0002-3027-0752 University of Tennessee, Knoxville,
Tennessee, USA H. Acharya, A.G. Delannoy0000-0003-1252-6213, S.
Spanier0000-0002-8438-3197 Texas A&M University, College Station, Texas, USA
O. Bouhali920000-0001-7139-7322, M. Dalchenko0000-0002-0137-136X, A.
Delgado0000-0003-3453-7204, R. Eusebi, J. Gilmore, T. Huang, T. Kamon93, H.
Kim0000-0003-4986-1728, S. Luo0000-0003-3122-4245, S. Malhotra, R. Mueller, D.
Overton, D. Rathjens0000-0002-8420-1488, A. Safonov0000-0001-9497-5471 Texas
Tech University, Lubbock, Texas, USA N. Akchurin, J. Damgov, V. Hegde, S.
Kunori, K. Lamichhane, S.W. Lee0000-0002-3388-8339, T. Mengke, S.
Muthumuni0000-0003-0432-6895, T. Peltola0000-0002-4732-4008, I. Volobouev, Z.
Wang, A. Whitbeck Vanderbilt University, Nashville, Tennessee, USA E.
Appelt0000-0003-3389-4584, S. Greene, A. Gurrola0000-0002-2793-4052, W. Johns,
A. Melo, H. Ni, K. Padeken0000-0001-7251-9125, F. Romeo0000-0002-1297-6065, P.
Sheldon0000-0003-1550-5223, S. Tuo, J. Velkovska0000-0003-1423-5241 University
of Virginia, Charlottesville, Virginia, USA M.W. Arenton0000-0002-6188-1011,
B. Cox0000-0003-3752-4759, G. Cummings0000-0002-8045-7806, J.
Hakala0000-0001-9586-3316, R. Hirosky0000-0003-0304-6330, M.
Joyce0000-0003-1112-5880, A. Ledovskoy0000-0003-4861-0943, A. Li, C.
Neu0000-0003-3644-8627, B. Tannenwald0000-0002-5570-8095, S.
White0000-0002-6181-4935, E. Wolfe0000-0001-6553-4933 Wayne State University,
Detroit, Michigan, USA N. Poudyal0000-0003-4278-3464 University of Wisconsin -
Madison, Madison, WI, Wisconsin, USA K. Black0000-0001-7320-5080, T.
Bose0000-0001-8026-5380, J. Buchanan0000-0001-8207-5556, C. Caillol, S.
Dasu0000-0001-5993-9045, I. De Bruyn0000-0003-1704-4360, P.
Everaerts0000-0003-3848-324X, F. Fienga0000-0001-5978-4952, C. Galloni, H. He,
M. Herndon0000-0003-3043-1090, A. Hervé, U. Hussain, A. Lanaro, A. Loeliger,
R. Loveless, J. Madhusudanan Sreekala0000-0003-2590-763X, A. Mallampalli, A.
Mohammadi, D. Pinna, A. Savin, V. Shang, V. Sharma0000-0003-1287-1471, W.H.
Smith0000-0003-3195-0909, D. Teague, S. Trembath-Reichert, W.
Vetens0000-0003-1058-1163 †: Deceased
1: Also at TU Wien, Wien, Austria
2: Also at Institute of Basic and Applied Sciences, Faculty of Engineering,
Arab Academy for Science, Technology and Maritime Transport, Alexandria, Egypt
3: Also at Université Libre de Bruxelles, Bruxelles, Belgium
4: Also at Universidade Estadual de Campinas, Campinas, Brazil
5: Also at Federal University of Rio Grande do Sul, Porto Alegre, Brazil
6: Also at University of Chinese Academy of Sciences, Beijing, China
7: Also at Department of Physics, Tsinghua University, Beijing, China
8: Also at UFMS, Nova Andradina, Brazil
9: Also at Nanjing Normal University Department of Physics, Nanjing, China
10: Now at The University of Iowa, Iowa City, Iowa, USA
11: Also at Institute for Theoretical and Experimental Physics named by A.I.
Alikhanov of NRC ‘Kurchatov Institute’, Moscow, Russia
12: Also at Joint Institute for Nuclear Research, Dubna, Russia
13: Also at Cairo University, Cairo, Egypt
14: Also at Ain Shams University, Cairo, Egypt
15: Also at Purdue University, West Lafayette, Indiana, USA
16: Also at Université de Haute Alsace, Mulhouse, France
17: Also at Tbilisi State University, Tbilisi, Georgia
18: Also at Erzincan Binali Yildirim University, Erzincan, Turkey
19: Also at CERN, European Organization for Nuclear Research, Geneva,
Switzerland
20: Also at RWTH Aachen University, III. Physikalisches Institut A, Aachen,
Germany
21: Also at University of Hamburg, Hamburg, Germany
22: Also at Isfahan University of Technology, Isfahan, Iran
23: Also at Brandenburg University of Technology, Cottbus, Germany
24: Also at Skobeltsyn Institute of Nuclear Physics, Lomonosov Moscow State
University, Moscow, Russia
25: Also at Physics Department, Faculty of Science, Assiut University, Assiut,
Egypt
26: Also at Karoly Robert Campus, MATE Institute of Technology, Gyongyos,
Hungary
27: Also at Institute of Physics, University of Debrecen, Debrecen, Hungary
28: Also at Institute of Nuclear Research ATOMKI, Debrecen, Hungary
29: Also at MTA-ELTE Lendület CMS Particle and Nuclear Physics Group, Eötvös
Loránd University, Budapest, Hungary
30: Also at Wigner Research Centre for Physics, Budapest, Hungary
31: Also at IIT Bhubaneswar, Bhubaneswar, India
32: Also at Institute of Physics, Bhubaneswar, India
33: Also at G.H.G. Khalsa College, Punjab, India
34: Also at Shoolini University, Solan, India
35: Also at University of Hyderabad, Hyderabad, India
36: Also at University of Visva-Bharati, Santiniketan, India
37: Also at Indian Institute of Technology (IIT), Mumbai, India
38: Also at Deutsches Elektronen-Synchrotron, Hamburg, Germany
39: Also at Sharif University of Technology, Tehran, Iran
40: Also at Department of Physics, University of Science and Technology of
Mazandaran, Behshahr, Iran
41: Now at INFN Sezione di Bari, Università di Bari, Politecnico di Bari,
Bari, Italy
42: Also at Italian National Agency for New Technologies, Energy and
Sustainable Economic Development, Bologna, Italy
43: Also at Centro Siciliano di Fisica Nucleare e di Struttura Della Materia,
Catania, Italy
44: Also at Università di Napoli ’Federico II’, Napoli, Italy
45: Also at Riga Technical University, Riga, Latvia
46: Also at Consejo Nacional de Ciencia y Tecnología, Mexico City, Mexico
47: Also at IRFU, CEA, Université Paris-Saclay, Gif-sur-Yvette, France
48: Also at Institute for Nuclear Research, Moscow, Russia
49: Now at National Research Nuclear University ’Moscow Engineering Physics
Institute’ (MEPhI), Moscow, Russia
50: Also at Institute of Nuclear Physics of the Uzbekistan Academy of
Sciences, Tashkent, Uzbekistan
51: Also at St. Petersburg Polytechnic University, St. Petersburg, Russia
52: Also at University of Florida, Gainesville, Florida, USA
53: Also at Imperial College, London, United Kingdom
54: Also at Moscow Institute of Physics and Technology, Moscow, Russia
55: Also at P.N. Lebedev Physical Institute, Moscow, Russia
56: Also at California Institute of Technology, Pasadena, California, USA
57: Also at Budker Institute of Nuclear Physics, Novosibirsk, Russia
58: Also at Faculty of Physics, University of Belgrade, Belgrade, Serbia
59: Also at Trincomalee Campus, Eastern University, Sri Lanka, Nilaveli, Sri
Lanka
60: Also at INFN Sezione di Pavia, Università di Pavia, Pavia, Italy
61: Also at National and Kapodistrian University of Athens, Athens, Greece
62: Also at Ecole Polytechnique Fédérale Lausanne, Lausanne, Switzerland
63: Also at Universität Zürich, Zurich, Switzerland
64: Also at Stefan Meyer Institute for Subatomic Physics, Vienna, Austria
65: Also at Laboratoire d’Annecy-le-Vieux de Physique des Particules,
IN2P3-CNRS, Annecy-le-Vieux, France
66: Also at Şırnak University, Sirnak, Turkey
67: Also at Near East University, Research Center of Experimental Health
Science, Nicosia, Turkey
68: Also at Konya Technical University, Konya, Turkey
69: Also at Istanbul University - Cerrahpasa, Faculty of Engineering,
Istanbul, Turkey
70: Also at Piri Reis University, Istanbul, Turkey
71: Also at Adiyaman University, Adiyaman, Turkey
72: Also at Ozyegin University, Istanbul, Turkey
73: Also at Izmir Institute of Technology, Izmir, Turkey
74: Also at Necmettin Erbakan University, Konya, Turkey
75: Also at Bozok Universitetesi Rektörlügü, Yozgat, Turkey
76: Also at Marmara University, Istanbul, Turkey
77: Also at Milli Savunma University, Istanbul, Turkey
78: Also at Kafkas University, Kars, Turkey
79: Also at Istanbul Bilgi University, Istanbul, Turkey
80: Also at Hacettepe University, Ankara, Turkey
81: Also at Vrije Universiteit Brussel, Brussel, Belgium
82: Also at School of Physics and Astronomy, University of Southampton,
Southampton, United Kingdom
83: Also at IPPP Durham University, Durham, United Kingdom
84: Also at Monash University, Faculty of Science, Clayton, Australia
85: Also at Università di Torino, Torino, Italy
86: Also at Bethel University, St. Paul, Minneapolis, USA
87: Also at Karamanoğlu Mehmetbey University, Karaman, Turkey
88: Also at Bingol University, Bingol, Turkey
89: Also at Georgian Technical University, Tbilisi, Georgia
90: Also at Sinop University, Sinop, Turkey
91: Also at Erciyes University, Kayseri, Turkey
92: Also at Texas A&M University at Qatar, Doha, Qatar
93: Also at Kyungpook National University, Daegu, Korea