# Remarks on potential mean field games

P. Jameson Graber  J. Graber: Baylor University, Department of Mathematics;
Sid Richardson Building
1410 S. 4th Street
Waco, TX 76706
Tel.: +1-254-710-
Fax: +1-254-710-3569<EMAIL_ADDRESS>
###### Abstract.

In this expository article, we give an overview of the concept of potential
mean field games of first order. We give a new proof that minimizers of the
potential are equilibria by using a Lagrangian formulation. We also provide
criteria to determine whether or not a game has a potential. Finally, we
discuss in some depth the selection problem in mean field games, which
consists in choosing one out of multiple Nash equilibria.

###### Key words and phrases:

mean field games, mean field type control, selection problem, Nash
equilibrium, potential games, PDE constrained optimization, Lagrangian
formulation, entropy solutions

###### 2020 Mathematics Subject Classification:

35Q89, 49N80, 91A16

The author is grateful to be supported by National Science Foundation through
NSF Grant DMS-2045027.

##  1\. Introduction

Mean field game theory was introduced around 20 years ago as a framework for
analyzing the behavior of large numbers of agents playing strategically, with
the earliest references generally attributed to Lasry and Lions [74, 75, 76]
and Caines, Huang, and Malhamé [69]. Since then there has been a wealth of
literature published on both mean field games and its companion theory of mean
field type control, which can be defined as the control of McKean-Vlasov type
equations or, alternatively, of transport type partial differential equations
(PDE). The interested reader can examine the lecture notes [1], the monographs
[33, 34, 9], the lectures of Lions at the Collège de France [78] (see the
notes by Cardaliaguet [23]), and the survey article [55]. In this article, we
are going to focus on the case in which the two theories overlap, i.e. a mean
field game that is also a mean field type control problem; such games are
called _potential games_. It is intended to be expository. Though we will
prove one new result, the emphasis will be on explaining what is known about
the topic using a wide variety of sources. It is nevertheless inevitable that
many relevant sources will be missing, and for this the author apologizes in
advance.

Let us briefly state the idea of a mean field game. Assume there is a large
number of agents, each of which chooses a trajectory $\gamma$ in order to
minimize some cost, which may have the form

(1.1)
$J(m,\gamma)=\int_{0}^{T}\mathinner{\left(L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}+f\mathinner{\left(m(t),\gamma(t)\right)}\right)}\operatorname{d\\!}t+g\mathinner{\left(m(T),\gamma(T)\right)}.$

Here $m(t)$ represents a distribution of states (or positions) at time $t$.
The idea is that any given player will have knowledge of the distribution of
the states of all agents, at least through the functions $f$ and $g$. For any
initial state $x$, we say that a curve $\gamma_{x}(t)$ is a _best response_ to
the flow $m(t)$ provided that $\gamma=\gamma_{x}(\cdot)$ minimizes
$J(m,\gamma)$. Now fix an initial measure $m_{0}$, representing the
distribution of the players’ starting positions. If every player chooses a
best response to the flow $m(t)$, then there is a resulting flow of measure
$\mu(t)$, giving the distribution of players’ positions at time $t$ (for
instance, if the best response $\gamma_{x}(t)$ is unique for each $x$, then
$\mu(t)$ is the push-forward of $m_{0}$ through the map
$x\mapsto\gamma_{x}(t)$). We call $\mu(t)$ a best response to $m(t)$. A _Nash
equilibrium_ is defined to be a flow of measures $m(t)$ that is a best
response to itself.

Using principles from optimal control theory and continuum mechanics, we can
derive a system of PDE to characterize Nash equilibrium. Introducing a value
function

(1.2)
$u(t,x)=\inf_{\gamma(t)=x}\mathinner{\left\\{\int_{t}^{T}\mathinner{\left(L\mathinner{\left(\gamma(\tau),\dot{\gamma}(\tau)\right)}+f\mathinner{\left(m(\tau),\gamma(\tau)\right)}\right)}\operatorname{d\\!}t+g\mathinner{\left(m(T),\gamma(T)\right)}\right\\}},$

we see that formally $u$ should be the solution of a Hamilton-Jacobi equation

$-\partial_{t}u(t,x)+H\mathinner{\left(x,D_{x}u(t,x)\right)}=f\mathinner{\left(m(t),x\right)}$

with Hamiltonian given by taking the Legendre transform of $L$ with respect to
the velocity:

(1.3)
$H(x,p)\mathrel{\mathop{\ordinarycolon}}=L^{*}(x,-p)=\sup_{v}\mathinner{\left(-p\cdot
v-L(x,v)\right)}.$

Then the optimal feedback control is (formally) given by
$-D_{p}H\mathinner{\left(x,D_{x}u(x,t)\right)}$, i.e. best response curves are
trajectories of the flow

$\dot{\gamma}(t)=-D_{p}H\mathinner{\left(\gamma(t),D_{x}u\mathinner{\left(\gamma(t),t\right)}\right)}.$

From continuum mechanics, this implies that the best response $\mu(t)$ solves
the continuity equation

$\partial_{t}\mu-\nabla\cdot\mathinner{\left(D_{p}H(x,D_{x}u)\mu\right)}=0.$

In Nash equilibrium, $\mu=m$. The resulting system of PDE for the pair $(u,m)$
is

(1.4) $\displaystyle-\partial_{t}u+H(x,D_{x}u)=f(m,x),$ $\displaystyle
x\in\Omega,t\in(0,T),$
$\displaystyle\partial_{t}m-\nabla\cdot\mathinner{\left(D_{p}H(x,D_{x}u)m\right)}=0,$
$\displaystyle x\in\Omega,t\in(0,T)$ $\displaystyle m(0)=m_{0},\quad
u(x,T)=g(m(T),x).$

The spatial domain $\Omega$ could be all of Euclidean space
${\mathbb{R}}^{d}$, the torus
${\mathbb{T}}^{d}\mathrel{\mathop{\ordinarycolon}}={\mathbb{R}}^{d}/{\mathbb{Z}}^{d}$
in the case of periodic boundary conditions, or a bounded domain with smooth
boundary, on which we impose a no-flux boundary condition, formally written
${\bf n}\cdot D_{p}H(x,D_{x}u)m=0$ for $x\in\partial\Omega$. Here and in the
entire article we treat only _first order_ mean field games, meaning that the
control problem is purely deterministic, and thus the resulting PDE system is
of first order.

In [76, Section 2.6] Lasry and Lions make the observation that (1.4) has an
interpretation as a first-order condition of optimality for the optimal
control of the flow of measures. Formally, solutions of (1.4) can be derived
from finding critical points of the following cost functional:

(1.5)
${\mathcal{J}}(m,v)=\int_{0}^{T}\mathinner{\left(\int_{\Omega}L\mathinner{\left(x,v(t,x)\right)}m(t,\operatorname{d\\!}x)+F\mathinner{\left(m(t)\right)}\right)}\operatorname{d\\!}t+G\mathinner{\left(m(T)\right)}.$

Here $v=v(t,x)$ is the vector field that generates the flow of measures, i.e.
$m$ and $v$ are related through the continuity equation

(1.6) $\partial_{t}m+\nabla\cdot(mv)=0,$

which holds in the sense of distributions. The cost appearing in (1.5) can be
thought of as a _potential_ of the individual cost appearing in (1.1), i.e. if
we take the derivative of (1.5) with respect to the measure $m$, then we will
get (1.1). We will make this claim more precise below in Sections 2.1 and 2.2.

Lasry and Lions’ observation about the mean field game proposed in [76]
implies that it is a _potential game_ , a term which has long been used in the
literature on game theory [82]. In the mathematics literature on mean field
games, one does not always see the term “potential game” explicitly used.
However, many of the first results on weak solutions for first-order models,
or second-order models with degenerate diffusion, were based on the fact that
those games are potential [24, 27, 28, 32]. The methods in these papers are
largely inspired by optimal transport theory, especially the Benamou-Brenier
dynamic formulation from [6] (see also [25]). We will outline these methods
below in Section 1.1.

The purpose of this article is to provide an overview of known results and
open problems about potential mean field games. It will be useful to draw on a
wide range of sources, ranging from PDE literature to theoretical economics.
There are at least two reasons to take a special interest in potential games.
One is computational: there is a rich collection of mathematical techniques
that can be used to find minimizers of a potential, and if we can prove such a
minimizer exists, then we have proved the existence of a Nash equilibrium as
well. Another reason to study potential games is out of consideration for the
_selection problem_ , which we can briefly summarize as follows: if there are
multiple Nash equilibria, how can we rationally choose one as preferable to
another? In a potential game, all minimizers of the potential are Nash
equilibria, but not all equilibria need to be minimizers of the potential.
Thus, to give a preview of things to come, we can say the potential provides a
_selection principle_ , i.e. a justification for choosing one Nash equilibrium
over others. Selection will be a major theme of this article and will be
covered in Section 3.

In the remainder of this introduction, we will lay out the history of
potential games from two different points of view. First, we will explain how
the literature on mean field games has taken the perspective of PDE
constrained optimization in order to produce both theoretical and
computational results. Then we will take a brief look at some economics
literature on potential games and see how it relates to our mean field
setting. Although the results from this literature cannot be immediately
applied to differential games, they are nevertheless illuminating and suggest
certain avenues of future research in mean field games. Following this
historical background, we will shift points of view from the Eulerian (i.e.
using the PDE system) to the Lagrangian (tracking the distribution of paths),
which will provide the setting for this paper’s main result.

###  1.1. PDE constrained optimization

Assume the potential ${\mathcal{J}}$ from (1.5) is convex in the variables
$(m,mv)$. One can construct a unique Nash equilibrium using the following
steps (cf. [8]).

  1. (1)

The constraint (1.6), i.e. the continuity equation, can be written in weak
form as follows:

$\int_{0}^{T}\int_{\Omega}\mathinner{\left(\partial_{t}u(t,x)+D_{x}u(t,x)\cdot
v(t,x)\right)}m(t,\operatorname{d\\!}x)-\int_{\Omega}u(T,x)m(T,\operatorname{d\\!}x)+\int_{\Omega}u(0,x)m_{0}(\operatorname{d\\!}x)=0$

for all smooth test functions $u$. Using $u$ as a Lagrange multiplier, we can
rewrite the constrained problem in Lagrangian form as

$\displaystyle\inf_{(m,v)}{\mathcal{J}}(m,v)$
$\displaystyle=\inf_{(m,v)}\sup_{u}{\mathcal{J}}(m,v)+\int_{0}^{T}\int_{\Omega}\mathinner{\left(\partial_{t}u(t,x)+D_{x}u(t,x)\cdot
v(t,x)\right)}m(t,\operatorname{d\\!}x)$
$\displaystyle\quad-\int_{\Omega}u(T,x)m(T,\operatorname{d\\!}x)+\int_{\Omega}u(0,x)m_{0}(\operatorname{d\\!}x).$

  2. (2)

Use a formal min-max exchange to identify a dual problem ${\mathcal{J}}^{*}$
for the adjoint state $u$:

$\displaystyle\inf_{(m,v)}{\mathcal{J}}(m,v)$
$\displaystyle=\sup_{u}\inf_{(m,v)}\int_{0}^{T}\mathinner{\left(\int_{\Omega}\mathinner{\left(L\mathinner{\left(v(t,x)\right)}+\partial_{t}u(t,x)+D_{x}u(t,x)\cdot
v(t,x)\right)}m(t,\operatorname{d\\!}x)+F\mathinner{\left(m(t)\right)}\right)}\operatorname{d\\!}t$
$\displaystyle\quad+G\mathinner{\left(m(T)\right)}-\int_{\Omega}u(T,x)m(T,\operatorname{d\\!}x)+\int_{\Omega}u(0,x)m_{0}(\operatorname{d\\!}x)$
$\displaystyle=\sup_{u}\inf_{m}\int_{0}^{T}\mathinner{\left(\int_{\Omega}\mathinner{\left(\partial_{t}u(t,x)-H\mathinner{\left(x,D_{x}u(t,x)\right)}\right)}m(t,\operatorname{d\\!}x)+F\mathinner{\left(m(t)\right)}\right)}\operatorname{d\\!}t$
$\displaystyle\quad+G\mathinner{\left(m(T)\right)}-\int_{\Omega}u(T,x)m(T,\operatorname{d\\!}x)+\int_{\Omega}u(0,x)m_{0}(\operatorname{d\\!}x)$
$\displaystyle=\sup_{u}\int_{0}^{T}-F^{*}\mathinner{\left(-\partial_{t}u(t,\cdot)+H\mathinner{\left(\cdot,D_{x}u(t,\cdot)\right)}\right)}\operatorname{d\\!}t-G^{*}\mathinner{\left(u(T,\cdot)\right)}=\mathrel{\mathop{\ordinarycolon}}-\inf_{u}{\mathcal{J}}^{*}(u).$

  3. (3)

Using the Fenchel-Rockafellar Theorem [47], one can establish the above
identity rigorously, and at the same time one proves that ${\mathcal{J}}$ has
a minimizer $(m,v)$.

  4. (4)

To prove the existence of an adjoint state, one must suitably relax the dual
problem ${\mathcal{J}}^{*}(u)$. With a slight abuse of notation, we define

${\mathcal{J}}^{*}(u,\alpha)=\int_{0}^{T}F^{*}\mathinner{\left(\alpha(t,\cdot)\right)}\operatorname{d\\!}t+G^{*}\mathinner{\left(u(T,\cdot)\right)}$

for pairs $(u,\alpha)$ satisfying

$-\partial_{t}u+H\mathinner{\left(x,D_{x}u\right)}\leq\alpha.$

Under suitable conditions on $F$ and $G$, it is possible to prove the
existence of a minimizing pair $(u,\alpha)$. Another technical step is to
prove that $\min{\mathcal{J}}^{*}(u,\alpha)=\inf{\mathcal{J}}^{*}(u)$. See for
instance [27, Section 2.2]. See also [61].

  5. (5)

Finally, one can show that the minimizers of primal and dual problems satisfy
the PDE system (1.4) in some suitable weak sense. For first-order and
degenerate second-order problems in which $m$ is required to be a density (and
$F$ and $G$ depend on pointwise values of that density), one does not
necessarily have that $u$ satisfies the Hamilton-Jacobi equation in the
viscosity sense or even in the sense of distributions; this is replaced by an
even weaker criterion that essentially says $u$ behaves like the value
function on the support of $m$. See [27, 28] for a precise definition of weak
solutions.

It is not necessary to have convexity (or uniqueness of minimizers) in order
to prove that minimizers of the potential are equilibria. If one takes any
minimizer $(\bar{m},\bar{v})$ of ${\mathcal{J}}$, one can show that it also
minimizes the same functional with $F(m)$ and $G(m)$ replaced by linear
approximations near $\bar{m}$; the rest of the argument proceeds by duality as
above. See [17, Proposition 3.1] for details.

In addition to proving the existence of equilibria, the above outline also
motivates numerical algorithms to compute solutions, since it implies that a
solution is equivalent to a saddle point for the Lagrangian

${\mathscr{L}}(m,v,u)\mathrel{\mathop{\ordinarycolon}}={\mathcal{J}}(m,v)+{\mathcal{J}}^{*}(u)+\int_{\Omega}u(0)m_{0}-\int_{\Omega}u(T)m(T)+\int_{0}^{T}\int_{\Omega}\mathinner{\left(\partial_{t}u+D_{x}u\cdot
v\right)}m.$

See, for instance, [7, 8]. This method hearkens back to Benamou and Brenier’s
computational approach to the Monge-Kantorovitch problem in optimal
transportation, which they reformulate as the control of a continuity equation
[6].

###  1.2. Connection to the theoretical economics literature

The concept of a potential game goes back a long way in the literature on game
theory and economics; see, for instance, the overview given by Monderer and
Shapley [82]. For finite games, a potential game is one for which a change in
any player’s cost function is matched precisely by a corresponding change in
the potential function. To be precise, suppose player $i\in\\{1,\ldots,N\\}$
choose a strategy $a_{i}$ from a finite set $S_{i}$, and for a vector
$a=(a_{1},\ldots,a_{n})$ denote by $f_{i}(a)=f_{i}(a_{i},a_{-i})$ the cost to
player $i$ (the notation $(a_{i},a_{-i})$ is standard in game theory for
singling out the component $a_{i}$, with $a_{-i}$ representing all the other
components). (NB: in the economics literature it is more common to think of
maximizing utility instead of minimizing cost; as mathematicians, it seems, we
insist on multiplying all functions by $-1$.) A Nash equilibrium is a vector
$a$ such that

$f_{i}(\tilde{a}_{i},a_{-i})\geq
f_{i}(a_{i},a_{-i})\quad\forall\tilde{a}_{i},\ \forall i\in\\{1,\ldots,N\\}.$

The function $P$ is a potential for the game provided

$f_{i}(\tilde{a}_{i},a_{-i})-f_{i}(a_{i},a_{-i})\equiv
P(\tilde{a}_{i},a_{-i})-P(a_{i},a_{-i})$

for each $i$. In this setting, it is immediate that Nash equilibria correspond
to local minimizers of $P$. Indeed, if $a$ is a minimizer of $P$, then for
player $i$ to change strategies from $a_{i}$ to $\tilde{a}_{i}$ can only mean
an increase in $P$, which corresponds to an increase in $f_{i}$; since this is
true for every $i$, $a$ is an equilibrium.

The concept of a potential game seems to have been first introduced in [85],
which defined a particular class known as _congestion games_. Roughly
speaking, a congestion game is one for which each player’s utility depends on
the total number of players sharing the same spaces (e.g. road segments) that
appear in the player’s strategy. It can be proved that every finite congestion
game has a potential and that any minimizer of the potential is also a Nash
equilibrium. In [82] the authors show that every finite potential game is
isomorphic to a congestion game. Heuristically, one might posit that the
equivalent of congestion games for mean field games is one in which $f$ and
$g$ depend on $m$ through its density, i.e. $f(m,x)=f(\rho(x),x)$ whenever
$\operatorname{d\\!}m(x)=\rho(x)\operatorname{d\\!}x$. It is a remarkable
coincidence that much of the early literature on the PDE system (1.4) deals
with precisely this case, or with a related case in which
$f(m,x)=\int\xi(x-y)f\mathinner{\left(\int\xi(y-z)\operatorname{d\\!}m(z),y\right)}\operatorname{d\\!}y$
where $\xi$ is a standard convolution kernel. One may even be tempted to
conjecture that all potential mean field games are in some sense isomorphic to
an appropriately defined “congestion game;” however, this is beyond the scope
of the present article.

Monderer and Shapley also prove that potential games have the _fictitious
play_ property [81]. This means players can progressively “learn” a Nash
equilibrium through an iterative process in which the next strategy chosen
will be the best response to the average of all previous distributions of
strategies [18]. If every limit point of this sequence is a Nash equilibrium,
the game is said to have the fictitious play property. Such a property
provides additional justification for the concept of Nash equilibrium: in
addition to players having rationally consistent strategies, there is a
plausible path through which these strategies can gradually evolve. The
fictitious play property has also been proved for potential mean field games
by Cardaliaguet and Hadikhanloo [29]; see also [17]. More general, Sandholm
has observed that for a potential game, there exists a general criterion for
population dynamics that lead to Nash equilibrium, namely the familiar concept
of Lyapunov stability [86]. That is, if the potential is a Lyapunov function
for a dynamical system on the space of population distributions, then one can
deduce that through such dynamics the population will converge to a rest
point; if the dynamics are such that rest points are all Nash equilibria, then
we know that each equilibrium is stable, hence we expect them to persist. To
state and prove similar results for mean field games would be considerably
more technical, since the space of population distributions in this case would
be probability measures on the space of continuous curves. We note, however,
that particular evolutionary games have already been studied in a mean field
setting; see for instance [4] and references therein. We will not pursue this
direction in the present paper, but it would make for interesting future
research.

An analog of finite potential games for a continuum of players has been given
by Sandholm [86]. (This does not quite constitute a mean field game because it
is not a differential game; however, the spirit is the same, in the sense that
one can view it as the limit of $N$-player games as $N\to\infty$.) Let us give
the simplest version of such a game. Assume there are $n$ strategies, labeled
$1,\ldots,n$. Instead of individual strategies, we will consider
_distributions_ of strategies, which are vectors in the simplex
$\Sigma=\\{x\in{\mathbb{R}}^{n}\mathrel{\mathop{\ordinarycolon}}x_{j}\geq
0,x_{1}+\cdots+x_{n}=1\\}$. For $x\in\Sigma$, the _support_ of $x$ is the set
of all $j$ such that $x_{j}>0$, i.e. the set of strategies that are being used
by someone. The cost function for the $j$th strategy is denoted $f_{j}(x)$. We
say that $a\in\Sigma$ is a Nash equilibrium provided that, for every $j$ in
the support of $a$, we have $f_{j}(a)\leq f_{i}(a)$ for all $i$. In other
words, if there are any players using strategy $j$, it is because $j$ is an
optimal strategy to use given the population distribution $a$. We say the game
has a potential if there is some
$F\mathrel{\mathop{\ordinarycolon}}{\mathbb{R}}^{n}\to{\mathbb{R}}$ for which
$f=(f_{1},\ldots,f_{n})=\nabla F$.

In this setting, it is straightforward, though not quite immediate, to see
that minimizers of $F$ are also equilibria. Again, we have only to consider
what would happen if some people changed their current strategy. We have the
following simple result.

######  Proposition 1.1.

Suppose $f=(f_{1},\ldots,f_{n})=\nabla F$ for some potential $F$. Then the
minimizers of $F$ are also equilibria.

###### Proof.

Let $a$ minimize $F$, let $j$ be such that $a_{j}>0$, and let $i\neq j$.
Define $e_{i}$ to be the $i$th vector in ${\mathbb{R}}^{n}$ For sufficiently
small $h>0$, $a+he_{i}-he_{j}\in\Sigma$ and represents the strategy
distribution that results from some players switching from $j$ to $i$. By
minimality, $F(a+he_{i}-he_{j})\geq F(a)$ for all $h>0$. We deduce $\nabla
F(a)\cdot(e_{i}-e_{j})\geq 0$, which is the same as $f_{i}(a)-f_{j}(a)\geq 0$,
as desired. ∎

######  Remark 1.2.

Notice that in the proof of Proposition 1.1, we do not need the full gradient
of $F$, but only its derivative in the simplex, i.e. $\lim_{h\to
0}\frac{F\mathinner{\left(a+h(b-a)\right)}-F(a)}{h}$ for vectors
$a,b\in\Sigma$. Cf. [87]. This is useful to keep in mind when we discuss
derivatives over the space of measures below.

Proposition 1.1 has an immediate corollary: if $F$ has at least one minimizer,
then the game has at least one equilibrium in pure strategies. The proof has
the virtue of being transparent, connecting the switching of strategies
directly to the increase or decrease in the potential. The central goal of
this article is to provide an equally transparent account of potential mean
field (differential) games. To do this, we will change point of view from
Eulerian (i.e. using the PDE system (1.4)) to Lagrangian, in which we view the
equilibrium as a measure over the space of curves [8, 14, 19, 32, 29, 50, 80].
This will allow us to generalize the proof of Proposition 1.1 in a direct way,
though the proof is more technical.

###  1.3. Lagrangian formulation

The cost $J$ given in (1.1) is a function of an individual trajectory $\gamma$
and a path of measures $m=m(t)$. Nash equilibrium is then determined from an
Eulerian point of view: we first derive the optimal velocity (feedback
control) at each point $(t,x)$ and then use that velocity field to determine
the evolution of $m$. To switch to the Lagrangian point of view, we instead
consider a probability measure $\eta$ on the space of continuous curves that
stay in $\Omega$. We then define the cost as

(1.7)
$J(\eta,\gamma)=\int_{0}^{T}\mathinner{\left(L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}+f\mathinner{\left(m^{\eta}(t),\gamma(t)\right)}\right)}\operatorname{d\\!}t+g\mathinner{\left(m^{\eta}(T),\gamma(T)\right)},$

where $m^{\eta}(t)\mathrel{\mathop{\ordinarycolon}}=e_{t}\sharp\eta$ is the
push-forward of $\eta$ through the evaluation map
$e_{t}(\gamma)\mathrel{\mathop{\ordinarycolon}}=\gamma(t)$. Nash equilibrium
is then defined by the fact that $\eta$ is concentrated on curves $\gamma$
that are optimal with respect to the cost $J(\cdot,\eta)$.

The Lagrangian point of view has been employed in several works on mean field
games, e.g. [8, 29, 19, 80, 50, 14]. It is especially useful as a first
approach to studying mean field games with state constraints, for which
deriving the PDE solution can be quite technical [20, 21]. Fischer and Silva
used the Lagrangian point of view to prove that $N$-player Nash equilibria
converge to mean field equilibria as $N\to\infty$, for a first-order game
[50]. This may be compared with the result of Sandholm for games with finite
strategy sets [86].

In this article, the Lagrangian point of view will allow us to generalize the
simple proof of Proposition 1.1 showing that minimizers of the potential are
also Nash equilibria. The potential is defined by

(1.8)
${\mathcal{J}}(\eta)=\int_{\Gamma}\int_{0}^{T}L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)+\int_{0}^{T}F\mathinner{\left(m^{\eta}(t)\right)}\operatorname{d\\!}t+G\mathinner{\left(m^{\eta}(T)\right)}.$

Indeed, from the Lagrangian point of view it is now straightforward to see why
${\mathcal{J}}$ is a potential. Formally, one can see that the first variation
of ${\mathcal{J}}$ with respect to the measure variable $\eta$ is given by
$J(\eta,\gamma)$, provided that $f$ and $g$ are the first variation of $F$ and
$G$, respectively. (See Section 2.1 for precise definitions.) The intuition
that minimizers are equilibria is that, if one starts with a particular
$\gamma$ in the support of $\eta$ and exchanges it with a different curve
$\tilde{\gamma}$, the cost can only increase. To make this rigorous, we will
have to exchange _neighborhoods_ and not only individual trajectories;
however, this adds only some minor technical details to an otherwise
straightforward argument.

The rest of this article is organized as follows. In Section 2, we prove that
minimizers of (1.8) are Nash equilibria for the game given by (1.7). To do
this, we start by giving some technical definitions, including derivatives on
the space of probability measures. The main result is proved following the
same form as for Proposition 1.1, while keeping in mind we have a continuous
(and indeed infinite dimensional) set of controls. After proving the main
result, we briefly address two other questions of interest: whether the
Lagrangian formulation is equivalent to the original Eulerian formulation, and
whether we can identify which mean field games have a potential. Then in
Section 3, we make some remarks on the selection problem, especially as it
pertains to potential mean field games. We will draw a strong connection
between selecting mean field equilibria and solving nonlinear transport
equations for which characteristics may cross. We will see that the concept of
entropy solutions from the theory of nonlinear balance laws can be plausibly
used to resolve the selection problem, though not without difficulty.

##  2\. Minimizers of a potential are equilibria

###  2.1. Preliminaries

For a separable metric space $X$, denote by ${\mathscr{P}}(X)$ the set of all
Borel probability measures on $X$. The support of a measure
$m\in{\mathscr{P}}(X)$, denoted $\operatorname{supp}m$, is defined to be the
set of all $x\in X$ such that $m\mathinner{\left(B_{\varepsilon}(x)\right)}>0$
for every $\varepsilon>0$, where $B_{\varepsilon}(x)$ is the ball of radius
$\varepsilon$ around $x$ in $X$. If $Y$ is another separable metric space and
$f\mathrel{\mathop{\ordinarycolon}}X\to Y$, then for $m\in{\mathscr{P}}(X)$ we
define the push-forward $f\sharp m\in{\mathscr{P}}(Y)$ by $f\sharp
m(B)=m\mathinner{\left(f^{-1}(B)\right)}$. If $d$ is the metric on $X$ and
$p\geq 1$, we denote by ${\mathscr{P}}_{p}(X)$ the set of all
$m\in{\mathscr{P}}(X)$ such that

$\int_{X}d(x,x_{0})^{p}\operatorname{d\\!}m(x)<\infty$

for some (hence for all) fixed $x_{0}\in X$. Note that if $X$ is compact, then
${\mathscr{P}}_{p}(X)={\mathscr{P}}(X)$. The Wasserstein metric on
${\mathscr{P}}_{p}(X)$ is given by

(2.1) $W_{p}(\mu,\nu)^{p}=\inf\mathinner{\left\\{\int_{X\times
X}d(x,y)^{p}\operatorname{d\\!}\pi(x,y)\mathrel{\mathop{\ordinarycolon}}\pi\in\Pi(\mu,\nu)\right\\}}$

where $\Pi(\mu,\nu)$ is the set of all couplings between $\mu$ and $\nu$, i.e.
the set of all $\pi\in{\mathscr{P}}(X\times X)$ such that $\pi(B\times
X)=\mu(B)$ and $\pi(X\times B)=\nu(B)$ for all Borel sets $B\subset X$. It is
well-known that an optimal coupling exists, i.e. the infimum is a minimum [91,
Theorem 4.1].

###  2.2. Assumptions

Let us the consider a Lagrangian formulation of a certain class of mean field
games. We fix our spatial domain $\Omega$ to be either ${\mathbb{R}}^{d}$,
${\mathbb{T}}^{d}$, or a compact domain that has a ${\mathcal{C}}^{2}$
boundary. In the last case, we denote by
$d_{\Omega}(x)=\inf\mathinner{\left\\{\mathinner{\\!\left\lvert
x-y\right\rvert}\mathrel{\mathop{\ordinarycolon}}y\in\Omega\right\\}}$ the
distance from $x$ to $\Omega$, and
$b_{\Omega}(x)=d_{\Omega}(x)-d_{\Omega^{c}}(x)$ is called the oriented
distance function. By assumption on the regularity of the boundary, there
exists a constant $\rho_{0}>0$ such that, on the collar domain
$\mathinner{\left\\{x\mathrel{\mathop{\ordinarycolon}}d_{\partial\Omega}(x)<\rho_{0}\right\\}}$,
$b_{\Omega}$ is twice continuously differentiable with bounded derivatives of
first and second order.

The space $\Gamma$ will be defined as the set of all absolutely continuous
curves $\gamma\mathrel{\mathop{\ordinarycolon}}[0,T]\to\Omega$ such that
$\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(t)\right\rvert}^{2}\operatorname{d\\!}t<\infty$;
this is the set of all admissible curves. We endow $\Gamma$ with the topology
of uniform convergence, i.e. it is a subspace of the Banach space
$C\mathinner{\left([0,T];\Omega\right)}$. For a given initial condition
$x\in\Omega$ we set
$\Gamma_{x}=\\{\gamma\in\Gamma\mathrel{\mathop{\ordinarycolon}}\gamma(0)=x\\}$.
For a given time $t\in[0,T]$ we define the evaluation map
$e_{t}\mathrel{\mathop{\ordinarycolon}}\Gamma\to\Omega$ by
$e_{t}(\gamma)=\gamma(t)$.

If $\eta\in{\mathscr{P}}(\Gamma)$, then for $t\in[0,T]$ we define
$m^{\eta}(t)\in{\mathscr{P}}(\Omega)$ by $m^{\eta}(t)=e_{t}\sharp\eta$. Fix an
initial measure $m_{0}\in{\mathscr{P}}(\Omega)$. Define
${\mathscr{P}}_{m_{0}}(\Gamma)$ to be the set of all
$\eta\in{\mathscr{P}}(\Gamma)$ such that $e_{0}\sharp\eta=m_{0}$. For any
$\gamma\in\Gamma$ and $\eta\in{\mathscr{P}}_{m_{0}}(\Gamma)$, define
$J(\eta,\gamma)$ as in (1.7), which we restate here:

(2.2)
$J(\eta,\gamma)=\int_{0}^{T}\mathinner{\left(L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}+f\mathinner{\left(m^{\eta}(t),\gamma(t)\right)}\right)}\operatorname{d\\!}t+g\mathinner{\left(m^{\eta}(T),\gamma(T)\right)}.$

We will need some assumptions on the Lagrangian $L$ and the couplings $f$ and
$g$.

######  Assumption 2.1.

$L\mathrel{\mathop{\ordinarycolon}}\Omega\times{\mathbb{R}}^{d}\to{\mathbb{R}}$
is continuously differentiable, and it is convex with respect to the second
variable. There exists a constant $C>0$ such that

(2.3) $\frac{1}{C}\mathinner{\\!\left\lvert v\right\rvert}^{2}-C\leq
L(x,v)\leq C\mathinner{\\!\left\lvert v\right\rvert}^{2}+C\quad\forall
v\in{\mathbb{R}}^{d},x\in\Omega.$

######  Assumption 2.2.

$f$ and $g$ are continuous real-valued functions on
${\mathscr{P}}_{2}(\Omega)\times\Omega$. Moreover, there exist functions $F$
and $G$ on ${\mathscr{P}}(\Omega)$ such that $f$ and $g$ are linear
derivatives of $F$ and $G$, respectively, and $F$ and $G$ are bounded below.

We will define the set of best responses $\Gamma_{x}(\eta)$ to be the set of
all $\gamma^{*}\in\Gamma_{x}$ such that

(2.4) $J(\gamma^{*},\eta)\leq J(\eta,\gamma)\quad\forall\gamma\in\Gamma_{x}.$

We say that $\eta\in{\mathscr{P}}_{m_{0}}(\Gamma)$ is a _Nash equilibrium_
with respect to $J$ provided that

(2.5) $\operatorname{supp}\eta\subset\bigcup_{x\in\Omega}\Gamma_{x}(\eta).$

In other words, for $\eta$-a.e. $\gamma^{*}$, $\gamma^{*}$ is a best response
to $\eta$ over all curves starting at $x=\gamma^{*}(0)$.

The potential is given by Equation (1.5), which we restate here:

(2.6)
${\mathcal{J}}(\eta)=\int_{\Gamma}\int_{0}^{T}L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)+\int_{0}^{T}F\mathinner{\left(m^{\eta}(t)\right)}\operatorname{d\\!}t+G\mathinner{\left(m^{\eta}(T)\right)}.$

Our goal is to show that minimizers of ${\mathcal{J}}$ must be Nash
equilibria, i.e. they must satisfy (2.5).

It follows from Assumptions 2.1 and 2.2 that any minimizer $\eta$ of
${\mathcal{J}}$ in ${\mathscr{P}}_{m_{0}}(\Gamma)$ must satisfy an estimate of
the form

(2.7)
$\int_{\Gamma}\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(t)\right\rvert}^{2}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)<\infty.$

We will denote by ${\mathscr{P}}^{*}_{m_{0}}(\Gamma)$ the set of all
$\eta\in{\mathscr{P}}_{m_{0}}(\Gamma)$ that satisfy (2.7).

######  Lemma 2.3.

Let $\eta\in{\mathscr{P}}^{*}_{m_{0}}(\Gamma)$ for some
$m_{0}\in{\mathscr{P}}_{2}(\Omega)$. Then $[0,T]\ni t\mapsto
m^{\eta}(t)\in{\mathscr{P}}_{2}(\Omega)$ is continuous.

###### Proof.

The proof is similar to [19, Lemma 3.2], but in general we could be on an
unbounded domain ${\mathbb{R}}^{d}$, so we must take this into account.

Let $t_{n}\to t$ in $[0,T]$. It is enough to show that $m^{\eta}(t_{n})\to
m^{\eta}(t)$ narrowly. That is, let $\varphi(x)$ be a continuous function
satisfying quadratic growth, i.e.

(2.8) $\mathinner{\\!\left\lvert\varphi(x)\right\rvert}\leq
C\mathinner{\left(1+\mathinner{\\!\left\lvert
x\right\rvert}^{2}\right)}\forall x\in\Omega,$

for some constant $C$. We must show that
$\int_{\Omega}\varphi(x)m^{\eta}(t_{n},\operatorname{d\\!}x)\to\int_{\Omega}\varphi(x)m^{\eta}(t,\operatorname{d\\!}x)$,
which is equivalent to showing that
$\int_{\Gamma}\varphi\mathinner{\left(\gamma(t_{n})\right)}\operatorname{d\\!}\eta(\gamma)\to\int_{\Gamma}\varphi\mathinner{\left(\gamma(t)\right)}\operatorname{d\\!}\eta(\gamma)$.
Since $\varphi$ is continuous, for any continuous path $\gamma\in\Gamma$ we
have
$\varphi\mathinner{\left(\gamma(t_{n})\right)}\to\varphi\mathinner{\left(\gamma(t)\right)}$.
Let $B_{R}$ be the set of all $\gamma\in\Gamma$ such that
$\max_{t}\mathinner{\\!\left\lvert\gamma(t)\right\rvert}<R$. On $B_{R}$, we
have that
$\mathinner{\\!\left\lvert\varphi\mathinner{\left(\gamma(t_{n})\right)}\right\rvert}\leq\max_{\mathinner{\\!\left\lvert
x\right\rvert}\leq R}\mathinner{\\!\left\lvert\varphi(x)\right\rvert}$. By the
dominated convergence theorem, it follows that
$\int_{B_{R}}\varphi\mathinner{\left(\gamma(t_{n})\right)}\operatorname{d\\!}\eta(\gamma)\to\int_{B_{R}}\varphi\mathinner{\left(\gamma(t)\right)}\operatorname{d\\!}\eta(\gamma)$.
To finish the proof, it is enough to show that

(2.9) $\lim_{R\to\infty}\sup_{n}\mathinner{\\!\left\lvert\int_{\Gamma\setminus
B_{R}}\mathinner{\left(\varphi\mathinner{\left(\gamma(t_{n})\right)}-\varphi\mathinner{\left(\gamma(t)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\right\rvert}=0.$

We start by observing that

(2.10) $\mathinner{\\!\left\lvert\int_{\Gamma\setminus
B_{R}}\mathinner{\left(\varphi\mathinner{\left(\gamma(t_{n})\right)}-\varphi\mathinner{\left(\gamma(t)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\right\rvert}\leq
C\int_{\Gamma\setminus
B_{R}}\mathinner{\left(2+\mathinner{\\!\left\lvert\gamma(t_{n})\right\rvert}^{2}+\mathinner{\\!\left\lvert\gamma(t)\right\rvert}^{2}\right)}\operatorname{d\\!}\eta(\gamma).$

Using the fundamental theorem of calculus and the Cauchy-Schwarz inequality,

(2.11)
$\mathinner{\\!\left\lvert\gamma(t)\right\rvert}^{2}=\mathinner{\\!\left\lvert\gamma(0)+\int_{0}^{t}\dot{\gamma}(s)\operatorname{d\\!}s\right\rvert}^{2}\leq
2\mathinner{\\!\left\lvert\gamma(0)\right\rvert}^{2}+2T\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(s)\right\rvert}^{2}\operatorname{d\\!}s,$

and so

(2.12) $\mathinner{\\!\left\lvert\int_{\Gamma\setminus
B_{R}}\mathinner{\left(\varphi\mathinner{\left(\gamma(t_{n})\right)}-\varphi\mathinner{\left(\gamma(t)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\right\rvert}\leq
C\int_{\Gamma\setminus
B_{R}}\mathinner{\left(2+4\mathinner{\\!\left\lvert\gamma(0)\right\rvert}^{2}+4T\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(s)\right\rvert}^{2}\operatorname{d\\!}s\right)}\operatorname{d\\!}\eta(\gamma).$

Observe that
$\int_{\Gamma}\mathinner{\\!\left\lvert\gamma(0)\right\rvert}^{2}\operatorname{d\\!}\eta(\gamma)=\int_{\Omega}\mathinner{\\!\left\lvert
x\right\rvert}^{2}\operatorname{d\\!}m_{0}(x)<\infty$. Since we also have
(2.7), it follows that $\gamma\mapsto
2+4\mathinner{\\!\left\lvert\gamma(0)\right\rvert}^{2}+4T\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(s)\right\rvert}^{2}\operatorname{d\\!}s$
is an integrable function with respect to $\eta$. Letting $R\to\infty$ and
using the dominated convergence theorem, the claim follows. ∎

In light of Assumptions 2.1 and 2.2 and Lemma 2.3, $J(\eta,\gamma)$ is well-
defined and finite for any $\gamma\in\Gamma$ and
$\eta\in{\mathscr{P}}_{m_{0}}^{*}(\Gamma)$.

###  2.3. Minimizers are equilibria in the Lagrangian formulation

######  Theorem 2.4.

Suppose $\eta$ minimizes ${\mathcal{J}}$. Then it is a Nash equilibrium with
respect to $J$.

The proof of Theorem 2.4 has the same spirit as that which is found in [86].
In this reference, the objective functional depends on distributions over a
finite control set, and so the problem is finite dimensional. Here the problem
is infinite dimensional, because controls are chosen from an infinite (indeed
infinite dimensional) set of curves.

###### Proof.

We will first prove the theorem in the case where $\Omega$ is translation
invariant, e.g. when $\Omega$ is ${\mathbb{R}}^{d}$ or ${\mathbb{T}}^{d}$.

The economic principle at work is that, because ${\mathcal{J}}$ is a primitive
of $J$, whenever two players currently in an equilibrium exchange strategies,
this causes ${\mathcal{J}}$ to increase. To make this precise, let $\eta$
minimize ${\mathcal{J}}$ and take $\gamma^{*}\in\operatorname{supp}\eta$. Let
$\gamma^{\prime}\in\Gamma_{x}$ where $x=\gamma^{*}(0)$. We want to show that
$J(\gamma^{*},\eta)\leq J(\gamma^{\prime},\eta)$. For $\varepsilon>0$, let
$B_{\varepsilon}=B_{\varepsilon}(\gamma^{*})$ be the set of all
$\gamma\in\Gamma$ such that

(2.13)
$\sup_{t\in[0,T]}\mathinner{\\!\left\lvert\gamma(t)-\gamma^{*}(t)\right\rvert}<\varepsilon,$

i.e. the ball of radius $\varepsilon$ around $\gamma^{*}$ in the uniform norm
topology. Note that $\eta(B_{\varepsilon})>0$. Define
$S_{\varepsilon}\mathrel{\mathop{\ordinarycolon}}\Gamma\to\Gamma$ by

(2.14)
$S_{\varepsilon}(\gamma)=\begin{cases}\gamma(0)-x+\gamma^{\prime},&\text{if}~{}\gamma\in
B_{\varepsilon},\\\ \gamma,&\text{if}~{}\gamma\notin
B_{\varepsilon}.\end{cases}$

Now define

(2.15) $\tilde{\eta}_{\varepsilon}=S_{\varepsilon}\sharp\eta.$

Notice that $\tilde{\eta}_{\varepsilon}\in{\mathscr{P}}(\Gamma)$, and to check
that $e_{0}\sharp\tilde{\eta}_{\varepsilon}=e_{0}\sharp\eta=m_{0}$ it suffices
to observe that $S(\gamma)(0)=\gamma(0)$ for every $\gamma\in\Gamma$. A
straightforward calculation shows that

(2.16)
${\mathcal{J}}\mathinner{\left(\tilde{\eta}_{\varepsilon}\right)}-{\mathcal{J}}(\eta)=\int_{B_{\varepsilon}}\int_{0}^{T}\mathinner{\left(L\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}-L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\\\
+\int_{0}^{T}\int_{0}^{1}\int_{B_{\varepsilon}}\mathinner{\left(f\mathinner{\left(m_{\lambda}^{\eta}(t),\gamma(0)-x+\gamma^{\prime}(t)\right)}-f\mathinner{\left(m_{\lambda}^{\eta}(t),\gamma(t)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\operatorname{d\\!}\lambda\operatorname{d\\!}t\\\
+\int_{0}^{1}\int_{B_{\varepsilon}}\mathinner{\left(g\mathinner{\left(m_{\lambda}^{\eta}(T),\gamma(0)-x+\gamma^{\prime}(T)\right)}-g\mathinner{\left(m_{\lambda}^{\eta}(T),\gamma(T)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\operatorname{d\\!}\lambda,\\\
m_{\lambda}^{\eta}(t)\mathrel{\mathop{\ordinarycolon}}=m^{\eta}(t)+\lambda\mathinner{\left(m^{\tilde{\eta}_{\varepsilon}}(t)-m^{\eta}(t)\right)},$

where we have used the fact that $f$ and $g$ are continuous linear derivatives
of $F$ and $G$, respectively. It is not hard to see that
$m^{\tilde{\eta}_{\varepsilon}}(t)\to m^{\eta}(t)$ in
${\mathscr{P}}_{2}(\Omega)$. Indeed, for any continuous function $\phi$ with
quadratic growth, we have

(2.17)
$\int_{\Omega}\phi(x)m^{\tilde{\eta}_{\varepsilon}}(t,\operatorname{d\\!}x)=\int_{\Gamma\setminus
B_{\varepsilon}}\phi\mathinner{\left(\gamma(t)\right)}\operatorname{d\\!}\eta(\gamma)+\int_{B_{\varepsilon}}\phi\mathinner{\left(S_{\varepsilon}(\gamma)(t)\right)}\operatorname{d\\!}\eta(\gamma)\\\
\to\int_{\Gamma}\phi\mathinner{\left(\gamma(t)\right)}\operatorname{d\\!}\eta(\gamma)=\int_{\Omega}\phi(x)m^{\eta}(t,\operatorname{d\\!}x),$

since $S_{\varepsilon}(\gamma)$ is uniformly close to $\gamma^{\prime}$ in the
set $B_{\varepsilon}$. Moreover, we have a uniform bound on
$m^{\tilde{\eta}_{\varepsilon}}(t)$ in ${\mathscr{P}}_{2}(\Omega)$, i.e. there
is a bounded set containing $m^{\tilde{\eta}_{\varepsilon}}(t)$ for all
$\varepsilon$ small and for all $t\in[0,T]$, since

(2.18) $\int_{\Omega}\mathinner{\\!\left\lvert
x\right\rvert}^{2}m^{\tilde{\eta}_{\varepsilon}}(t,\operatorname{d\\!}x)=\int_{\Gamma\setminus
B_{\varepsilon}}\mathinner{\\!\left\lvert\gamma(t)\right\rvert}^{2}\operatorname{d\\!}\eta(\gamma)+\int_{B_{\varepsilon}}\mathinner{\\!\left\lvert
S_{\varepsilon}(\gamma)(t)\right\rvert}^{2}\operatorname{d\\!}\eta(\gamma)\leq\int_{\Gamma\setminus
B_{\varepsilon}}\mathinner{\\!\left\lvert\gamma(t)\right\rvert}^{2}\operatorname{d\\!}\eta(\gamma)+2\eta(B_{\varepsilon})\mathinner{\left(\varepsilon^{2}+\mathinner{\\!\left\lvert\gamma^{\prime}(t)\right\rvert}^{2}\right)}.$

Now the left-hand side of (2.16) is non-negative, because $\eta$ is a
minimizer. We will divide both sides by
$\eta\mathinner{\left(B_{\varepsilon}\right)}$ and let $\varepsilon\to 0$.
From the continuity of $f$ and $g$, we deduce that

(2.19)
$\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{0}^{T}\int_{0}^{1}\int_{B_{\varepsilon}}\mathinner{\left(f\mathinner{\left(m_{\lambda}^{\eta}(t),\gamma(0)-x+\gamma^{\prime}(t)\right)}-f\mathinner{\left(m_{\lambda}^{\eta}(t),\gamma(t)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\operatorname{d\\!}\lambda\operatorname{d\\!}t\\\
+\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{0}^{1}\int_{B_{\varepsilon}}\mathinner{\left(g\mathinner{\left(m_{\lambda}^{\eta}(T),\gamma(0)-x+\gamma^{\prime}(T)\right)}-g\mathinner{\left(m_{\lambda}^{\eta}(T),\gamma(T)\right)}\right)}\operatorname{d\\!}\eta(\gamma)\operatorname{d\\!}\lambda\\\
\to\int_{0}^{T}\mathinner{\left(f\mathinner{\left(m^{\eta}(t),\gamma^{\prime}(t)\right)}-f\mathinner{\left(m^{\eta}(t),\gamma^{*}(t)\right)}\right)}\operatorname{d\\!}t+\mathinner{\left(g\mathinner{\left(m^{\eta}(T),\gamma^{\prime}(T)\right)}-g\mathinner{\left(m^{\eta}(T),\gamma^{*}(T)\right)}\right)}.$

Similarly, by the continuity of $L$ in the first variable, we get

(2.20)
$\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\int_{0}^{T}L\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\to\int_{0}^{T}L\mathinner{\left(\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\operatorname{d\\!}t.$

For the remaining term, we want to show that

(2.21)
$\int_{0}^{T}L\mathinner{\left(\gamma^{*}(t),\dot{\gamma^{*}}(t)\right)}\operatorname{d\\!}t\leq\liminf_{\varepsilon\to
0}\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\int_{0}^{T}L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma).$

Define

(2.22)
$\Gamma^{K}=\\{\gamma\in\Gamma\mathrel{\mathop{\ordinarycolon}}\int_{0}^{T}L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t\leq
K\\}.$

By Assumption 2.1, there is a constant $C_{K}$ such that
$\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(t)\right\rvert}^{2}\operatorname{d\\!}t\leq
C_{K}$ for all $\gamma\in\Gamma^{K}$. Thanks to the convexity of $L(x,v)$ with
respect to $v$, it follows from standard arguments (cf. [48, Theorem 1 from
Section 8.2]) that
$\gamma\mapsto\int_{0}^{T}L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t$
is lower semi-continuous on $\Gamma^{K}$. We deduce that, for any
$K<\int_{0}^{T}L\mathinner{\left(\gamma^{*}(t),\dot{\gamma}^{*}(t)\right)}\operatorname{d\\!}t$,
there is an $\varepsilon>0$ small enough such that $\Gamma^{K}\cap
B_{\varepsilon}=\emptyset$, and therefore

(2.23) $\liminf_{\varepsilon\to
0}\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\int_{0}^{T}L\mathinner{\left(\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\geq
K.$

Letting
$K\uparrow\int_{0}^{T}L\mathinner{\left(\gamma^{*}(t),\dot{\gamma}^{*}(t)\right)}\operatorname{d\\!}t$
we deduce (2.21). Combining (2.19), (2.20), and (2.21), we see from (2.16)
that

(2.24)
$\int_{0}^{T}L\mathinner{\left(\gamma^{*}(t),\dot{\gamma^{*}}(t)\right)}\operatorname{d\\!}t+\int_{0}^{T}f\mathinner{\left(m^{\eta}(t),\gamma^{*}(t)\right)}\operatorname{d\\!}t+g\mathinner{\left(m^{\eta}(T),\gamma^{*}(T)\right)}\\\
\leq\int_{0}^{T}L\mathinner{\left(\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\operatorname{d\\!}t+\int_{0}^{T}f\mathinner{\left(m^{\eta}(t),\gamma^{\prime}(t)\right)}\operatorname{d\\!}t+g\mathinner{\left(m^{\eta}(T),\gamma^{\prime}(T)\right)},$

i.e. $J(\gamma^{*},\eta)\leq J(\gamma^{\prime},\eta)$, as desired.

Now we deal with the case where there are state constraints, i.e. when
$\Omega$ is not translation invariant. Here we use some ideas from [19]. In
this case the curve $S_{\varepsilon}(\gamma)$ has to be constructed in such a
way as to not leave $\Omega$. Namely, if $\gamma\in B_{\varepsilon}$, we take
the projection of $\gamma(0)-x+\gamma^{\prime}$ onto $\Omega$. We recall that
$b_{\Omega}$ is $C^{2}_{b}$ smooth in a neighborhood of radius $\rho_{0}>0$
around $\partial\Omega$; we may assume that $\varepsilon<\rho_{0}$. Thus

(2.25)
$S_{\varepsilon}(\gamma)(t)=\begin{cases}\gamma(0)-x+\gamma^{\prime}(t)-d_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}Db_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)},&\text{if}~{}\gamma\in
B_{\varepsilon},\\\ \gamma(t),&\text{if}~{}\gamma\notin
B_{\varepsilon}\end{cases}$

where the first formula reduces to $\gamma(0)-x+\gamma^{\prime}(t)$ if it is
already in $\Omega$. Note that we still have $\mathinner{\\!\left\lvert
S_{\varepsilon}(\gamma)(t)-\gamma^{\prime}(t)\right\rvert}\leq\varepsilon$ for
every $\gamma\in B_{\varepsilon}$. Therefore, every step in the prove above
goes through except one: we need to prove that

(2.26)
$\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\int_{0}^{T}L\mathinner{\left(S_{\varepsilon}(\gamma)(t),\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\to\int_{0}^{T}L\mathinner{\left(\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\operatorname{d\\!}t$

as $\varepsilon\to 0$.

Let $\gamma\in B_{\varepsilon}$. By [19, Lemma 3.1] we have that
$S_{\varepsilon}(\gamma)$ is indeed absolutely continuous with

(2.27)
$\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)=\dot{\gamma}^{\prime}(t)-\left\langle
Db_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)},\dot{\gamma}^{\prime}(t)\right\rangle
Db_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}{\bf
1}_{\Omega^{c}}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}\\\
-d_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}D^{2}b_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}\dot{\gamma}^{\prime}(t),\quad\text{a.e.
}t\in[0,T],$

and as in [19, Proposition 3.1] there exists a constant $C>0$ (independent of
both $\varepsilon$ and $t$) such that

(2.28)
$\mathinner{\\!\left\lvert\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right\rvert}\leq
C\mathinner{\\!\left\lvert\dot{\gamma}^{\prime}(t)\right\rvert}\quad\forall
t\in[0,T].$

For constants $\delta>0$ (small) and $M>0$ (large), define

(2.29)
$\begin{split}I_{0}&\mathrel{\mathop{\ordinarycolon}}=\mathinner{\left\\{t\in[0,T]\mathrel{\mathop{\ordinarycolon}}\gamma^{\prime}(t)\in\partial\Omega\right\\}},\\\
I_{\delta}^{\prime}&\mathrel{\mathop{\ordinarycolon}}=\mathinner{\left\\{t\in[0,T]\mathrel{\mathop{\ordinarycolon}}0<d_{\Omega}\mathinner{\left(\gamma^{\prime}(t)\right)}<\delta\right\\}},\\\
I_{\delta}&\mathrel{\mathop{\ordinarycolon}}=\mathinner{\left\\{t\in[0,T]\mathrel{\mathop{\ordinarycolon}}d_{\Omega}\mathinner{\left(\gamma^{\prime}(t)\right)}\geq\delta\right\\}},\\\
J_{M}&\mathrel{\mathop{\ordinarycolon}}=\mathinner{\left\\{t\in[0,T]\mathrel{\mathop{\ordinarycolon}}\mathinner{\\!\left\lvert\dot{\gamma}^{\prime}(t)\right\rvert}\leq
M\right\\}}.\end{split}$

To prove (2.26), we will analyze
$L\mathinner{\left(S_{\varepsilon}(\gamma)(t),\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right)}$
on each of these sets. If $\varepsilon<\delta$, then it follows that
$S_{\varepsilon}(\gamma)(t)=\gamma(0)-x+\gamma^{\prime}(t)$ for every $t\in
I_{\delta}$, for every $\gamma\in B_{\varepsilon}$. We deduce that

(2.30)
$\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\int_{I_{\delta}}L\mathinner{\left(S_{\varepsilon}(\gamma)(t),\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\to\int_{I_{\delta}}L\mathinner{\left(\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\operatorname{d\\!}t.$

Next, by [19, Lemma 3.1] we have that for a.e. $t\in I_{0}$, $\left\langle
Db_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)},\dot{\gamma}^{\prime}(t)\right\rangle=0$,
and therefore we can write

(2.31)
$\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)=\dot{\gamma}^{\prime}(t)\\\
-\left\langle
Db_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}-Db_{\Omega}\mathinner{\left(\gamma^{\prime}(t)\right)},\dot{\gamma}^{\prime}(t)\right\rangle
Db_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}{\bf
1}_{\Omega^{c}}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}\\\
-d_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}D^{2}b_{\Omega}\mathinner{\left(\gamma(0)-x+\gamma^{\prime}(t)\right)}\dot{\gamma}^{\prime}(t),\quad\text{a.e.
}t\in I_{0},$

for any $\gamma\in B_{\varepsilon}$. Let $C_{\Omega}$ be a bound on
$D^{2}b_{\Omega}$ in $\partial\Omega+B_{\rho_{0}}$. Since
$\mathinner{\\!\left\lvert\gamma(0)-x\right\rvert}\leq\varepsilon$, we deduce
that

(2.32)
$\mathinner{\\!\left\lvert\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)-\dot{\gamma}^{\prime}(t)\right\rvert}\leq
2C_{\Omega}\varepsilon\mathinner{\\!\left\lvert\dot{\gamma}^{\prime}(t)\right\rvert},\quad\text{a.e.
}t\in I_{0}.$

Using the fact that $L$ is locally Lipschitz (in fact continuously
differentiable), there exists a constant $C=C(\Omega,M)$ such that for every
$\varepsilon>0$ small enough,

(2.33) $\mathinner{\\!\left\lvert
L\mathinner{\left(S_{\varepsilon}(\gamma)(t),\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right)}-L\mathinner{\left(\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\right\rvert}\leq
C\varepsilon\mathinner{\left(1+\mathinner{\\!\left\lvert\dot{\gamma}^{\prime}(t)\right\rvert}\right)}\quad\forall
t\in I_{0}\cap J_{M},\ \forall\gamma\in B_{\varepsilon}.$

We deduce that

(2.34)
$\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\int_{I_{0}\cap
J_{M}}L\mathinner{\left(S_{\varepsilon}(\gamma)(t),\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\to\int_{I_{0}\cap
J_{M}}L\mathinner{\left(\gamma^{\prime}(t),\dot{\gamma}^{\prime}(t)\right)}\operatorname{d\\!}t.$

Finally, combining the estimate in Assumption 2.1 with (2.28), we get

(2.35)
$\frac{1}{\eta\mathinner{\left(B_{\varepsilon}\right)}}\int_{B_{\varepsilon}}\mathinner{\\!\left\lvert\int_{I_{\delta}^{\prime}\cup
J_{M}^{c}}L\mathinner{\left(S_{\varepsilon}(\gamma)(t),\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{t}}S_{\varepsilon}(\gamma)(t)\right)}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)\right\rvert}\leq
C\int_{I_{\delta}^{\prime}\cup
J_{M}^{c}}\mathinner{\left(1+\mathinner{\\!\left\lvert\dot{\gamma}^{\prime}(t)\right\rvert}^{2}\right)}\operatorname{d\\!}t,$

for some constant $C$. Combining (2.30), (2.34), and (2.35), then letting
$\delta\to 0$ and $M\to\infty$, we deduce (2.26). This completes the proof. ∎

###  2.4. Equivalence of Lagrangian and original formulations

At least in certain cases, the minimization of (1.8) over measures
${\mathscr{P}}_{m_{0}}(\Gamma)$ (i.e. the Lagrangian formulation) can be
proved to be equivalent to the optimal control of the continuity equation,
namely, the minimization of (1.5) subject to the constraint (1.6) and initial
condition $m(0)=m_{0}$ (i.e. the Eulerian formulation). This is proved in [8]
for quadratic Lagrangians $L(x,v)=\frac{1}{2}\mathinner{\\!\left\lvert
v\right\rvert}^{2}$ on compact domains $\Omega$. The key is [8, Proposition
3.1], which proves a one-to-one correspondence between solutions to the
continuity equation and admissible probability measures $\eta$ on the space of
curves $\Gamma$. A similar result is found in [5, Theorem 8.2.1] and seems to
allow us to prove the that equivalence between Lagrangian and Eulerian
formulations holds also when $\Omega={\mathbb{R}}^{d}$, at least when
$L(x,v)=\frac{1}{2}\mathinner{\\!\left\lvert v\right\rvert}^{2}$. In this
brief article, we will not pursue this issue further, since in the example
studied in the following section, the Lagrangian formulation is sufficient.

###  2.5. When can a mean field game be a potential game?

In this subsection, we will consider whether or not it is possible to find a
potential for a given mean field game. All of our results will remain at the
formal level; that is, we will not state precise assumptions on the data but
assume they are as smooth as necessary for our conclusions to hold. Let us
consider a more general cost than (1.7) given by

(2.36)
$J(\eta,\gamma)=\int_{0}^{T}L\mathinner{\left(\mu^{\eta}(t),\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t+g\mathinner{\left(m^{\eta}(T),\gamma(T)\right)}$

where $\mu^{\eta}(t)$ is defined as the distribution of states and velocities
at time $t$ for curves distributed according to $\eta$. To be precise, let us
restrict to measures $\eta$ on the space
$\Gamma^{\prime}=C^{1}\mathinner{\left([0,T];\Omega\right)}$ of continuously
differentiable functions taking values in $\Omega$. Then let
$\dot{e}_{t}(\gamma)\mathrel{\mathop{\ordinarycolon}}=\dot{\gamma}(t)$, and
set $\mu^{\eta}(t)=(e_{t},\dot{e}_{t})\sharp\eta$. We define Nash equilibrium
in the same way as before. At this level of generality, the game is often
referred to as a _mean field game of controls_ , reflecting the fact that the
distribution of controls also plays a role [31]; the terminology “extended
mean field games” has also been used [56, 60]. Potential mean field games of
controls have been considered in [64, 15, 65].

If $J$ defined in (2.36) has a potential in the same way (1.7) has potential
given by (1.8), it must have the form

(2.37)
${\mathcal{J}}(\eta)=\int_{0}^{T}{\mathscr{L}}\mathinner{\left(\mu^{\eta}(t)\right)}\operatorname{d\\!}t+G\mathinner{\left(m^{\eta}(T)\right)}.$

Formally,

(2.38)
$\dfrac{\delta{\mathcal{J}}}{\delta\eta}(\eta,\gamma)=\int_{0}^{T}\dfrac{\delta{\mathscr{L}}}{\delta\mu}\mathinner{\left(\mu^{\eta}(t),\gamma(t),\dot{\gamma}(t)\right)}\operatorname{d\\!}t+\dfrac{\delta
G}{\delta m}\mathinner{\left(m^{\eta}(T),\gamma(T)\right)},$

where the notation $\dfrac{\delta{\mathcal{J}}}{\delta\eta}$ refers to “the”
linear derivative of ${\mathcal{J}}$ with respect to the measure variable. If
${\mathcal{J}}$ is a potential, it follows that $L(\mu,x,v)$ must be a linear
derivative of ${\mathscr{L}}(\mu)$, so we will write

(2.39)
$\dfrac{\delta{\mathscr{L}}}{\delta\mu}\mathinner{\left(\mu,x,v\right)}=L(\mu,x,v)+c(\mu),$

where $c(\mu)$ represents an arbitrary constant depending only on $\mu$.
Taking another derivative, we have

(2.40)
$\dfrac{\delta}{\delta\mu}\dfrac{\delta{\mathscr{L}}}{\delta\mu}\mathinner{\left(\mu,x,v,\tilde{x},\tilde{v}\right)}=\dfrac{\delta
L}{\delta\mu}(\mu,x,v,\tilde{x},\tilde{v})+c(\mu).$

By the usual arguments, if the left-hand side is continuous in all variables,
then it is also symmetric in the sense

(2.41)
$\dfrac{\delta}{\delta\mu}\dfrac{\delta{\mathscr{L}}}{\delta\mu}\mathinner{\left(\mu,x,v,\tilde{x},\tilde{v}\right)}=\dfrac{\delta}{\delta\mu}\dfrac{\delta{\mathscr{L}}}{\delta\mu}\mathinner{\left(\mu,\tilde{x},\tilde{v},x,v\right)},$

which implies

(2.42) $\dfrac{\delta L}{\delta\mu}(\mu,x,v,\tilde{x},\tilde{v})=\dfrac{\delta
L}{\delta\mu}(\mu,\tilde{x},\tilde{v},x,v).$

This symmetry consideration can be used to provide many examples of
Lagrangians $L$ for which there cannot be a potential. For example, suppose
that $L$ depends only on the distribution of states and not controls, i.e.
$L(\mu,x,v)=\ell(m,x,v)$ where $m=p^{1}\sharp\mu$ and $p^{1}(x,v)=x$. We have

(2.43) $\dfrac{\delta
L}{\delta\mu}(\mu,x,v,\tilde{x},\tilde{v})=\dfrac{\delta\ell}{\delta
m}(m,x,v,\tilde{x}).$

By the symmetry criterion (2.42), it follows that $\dfrac{\delta\ell}{\delta
m}(m,x,v,\tilde{x})$ does not depend on $v$. We now deduce that $\ell(m,x,v)$
can be written in the separated form

(2.44) $\ell(m,x,v)=l(x,v)+f(m,x)$

for some functions $l$ and $f$. Hence if the particular case we are
considering is _not_ a mean field game of controls, then it is a potential
game if _and only if_ the Lagrangian can be written in separated form.

##  3\. Remarks on the selection problem

The problem of equilibrium selection is well-known in game theory. In the
Foreword to Harsanyi and Selten’s 1988 text [67], Aumann discusses the
justification of the equilibrium concept as follows:

> “One of the oldest rationales for this concept, advanced already by von
> Neumann and Morgenstern (1944), is that any normative theory that advises
> players how to play games must pick and equilibrium in each game. A theory
> recommending anything other than an equilibrium would be self-defeating, in
> the sense that a player who believes that the others are following the
> theory will sometimes be motivated to deviate from it.”

However, he then goes on to remark how this justification relates to
selection:

> “In general, a given game may have several equilibria. Yet uniqueness is
> crucial to the foregoing argument. Nash equilibrium makes sense only if each
> player knows which strategies the others are playing; if the equilibrium
> recommended by the theory is not unique, the players will not have this
> knowledge. Thus it is essential that for each game, the theory selects one
> unique equilibrium from the set of all Nash equilibrium.

Selten introduced what is known as “trembling hand perfect” equilibrium [88].
The idea is to keep only those equilibria that are robust with respect to
perturbation of the game by noise. In the case where the set of available
controls is finite, one can do this simply by assigning a uniformly small
probability $\varepsilon$ that any given player might switch to a different
strategy, and then see whether, for $\varepsilon$ small enough, the given
strategy distribution is still optimal for each player. A series of further
refinements in this vein have been proposed in [83, 71, 68, 66]; see [90] for
an overview.

The basic idea of selecting solutions that are robust with respect to
perturbations by noise is well-known in the study of PDE. Indeed, the concept
of an entropy solution to a nonlinear transport equation can be seen as the
solution selected by the vanishing viscosity limit (e.g. [73]), which can also
be thought of as a vanishing perturbation by Brownian noise. In the discussion
below, we will see that the same type of selection principle can be used for
mean field games under certain conditions. Note that the strategy space is
infinite, so a uniform perturbation is impossible; instead, a Gaussian
perturbation can be seen as a natural choice. In the context of potential mean
field games, the “vanishing noise limit” will be seen to correspond to
strategies that minimize the potential.

Throughout this section we will consider only a very particular case of the
mean field game studied above. This will allow us to develop the main ideas
without getting caught up in too many technical details, most of which are
still poorly understood. The main ideas are as follows. Corresponding to the
problem of minimizing the potential, there exists a Hamilton-Jacobi equation
on the space of probability measures. Formally taking the derivative of this
equation leads to the so-called _master equation_ in mean field games [26, 45,
38], which corresponds to finding a Nash equilibrium. When the Nash
equilibrium is unique, there is a unique classical solution to the master
equation (see [51] for a result that applies directly to our present context).
When it is not unique, a selection principle must be introduced, which is
deeply connected to defining a notion of weak solution to the master equation.
We will see that the notion of entropy solution is an attractive candidate,
but it is not without difficulties.

###  3.1. Preliminaries

Let $B$ be the ball of unit volume around the origin in ${\mathbb{R}}^{d}$,
let ${\mathscr{B}}$ be the Borel $\sigma$-algebra on $B$, and let
${\mathcal{L}}^{d}$ be $d$-dimensional Lebesgue measure; note that
$(B,{\mathscr{B}},{\mathcal{L}}^{d})$ is a probability space. Define
${\mathbb{H}}\mathrel{\mathop{\ordinarycolon}}=L^{2}(B,{\mathscr{B}},{\mathcal{L}}^{d})$,
and observe that ${\mathbb{H}}$ is a separable Hilbert space with inner
product given by $\left\langle
X,Y\right\rangle={\mathbb{E}}\mathinner{\left[X\cdot Y\right]}$ and norm
$\mathinner{\\!\left\lVert X\right\rVert}=\left\langle
X,X\right\rangle^{1/2}={\mathbb{E}}[\mathinner{\\!\left\lvert
X\right\rvert}^{2}]^{1/2}$. For $X\in{\mathbb{H}}$ we define
$\sharp(X)=X\sharp{\mathcal{L}}^{d}$, which is the law of the random variable
$X$. By Brenier’s polar factorization result [16], we have that
$\sharp\mathrel{\mathop{\ordinarycolon}}{\mathbb{H}}\to{\mathscr{P}}_{2}({\mathbb{R}}^{d})$
is surjective [54]. Moreover, we have

(3.1)
$W_{2}(\mu,\nu)^{2}=\min\mathinner{\left\\{{\mathbb{E}}\mathinner{\\!\left\lvert
X-Y\right\rvert}^{2}\mathrel{\mathop{\ordinarycolon}}X,Y\in{\mathbb{H}},\sharp(X)=\mu,\
\sharp(Y)=\nu\right\\}}.$

Let
$F\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{2}({\mathbb{R}}^{d})\to{\mathbb{R}}$.
We define
$\tilde{F}\mathrel{\mathop{\ordinarycolon}}{\mathbb{H}}\to{\mathbb{R}}$ by
$\tilde{F}(X)=F\mathinner{\left(\sharp(X)\right)}$, and we refer to
$\tilde{F}$ as the “lifted version” of $F$, because it lifts the domain from
${\mathscr{P}}_{2}({\mathbb{R}}^{d})$, which is a curved space, to
${\mathbb{H}}$, which is a Hilbert space. We say that $F$ has a Wasserstein
gradient
$D_{\mu}F\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{2}({\mathbb{R}}^{d})\times{\mathbb{R}}^{d}\to{\mathbb{R}}$
provided $\tilde{F}$ is continuously differentiable with Fréchet gradient
$D\tilde{F}\mathrel{\mathop{\ordinarycolon}}{\mathbb{H}}\to{\mathbb{H}}$, such
that

(3.2) $D\tilde{F}(X)=D_{\mu}F(\sharp(X),X).$

The following result, which is [33, Proposition 5.48] and [33, Proposition
5.51], relates the Wasserstein gradient to the linear derivative.

######  Proposition 3.1.

Suppose $F$ has a linear derivative $f(m,x)$ that is differentiable with
respect to $x$ such that both $f(m,x)$ and $D_{x}f(m,x)$ are continuous in
both variables. Assume $D_{x}f(m,x)$ has at most linear growth in $x$, and in
particular there exists for each $R>0$ a constant $C(R)$ such that

(3.3) $\mathinner{\\!\left\lvert D_{x}f(m,x)\right\rvert}\leq
C(R)\mathinner{\left(1+\mathinner{\\!\left\lvert
x\right\rvert}\right)}\quad\forall x\in{\mathbb{R}}^{d},\quad\forall
m\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})\ \text{s.t.}\
\int\mathinner{\\!\left\lvert\xi\right\rvert}^{2}\operatorname{d\\!}m(\xi)\leq
R.$

Then $F$ has a continuous Wasserstein gradient given by

(3.4) $D_{\mu}F(m,x)=D_{x}f(m,x).$

Conversely, if $F$ has a continuous Wasserstein gradient, and if the Fréchet
gradient $D\tilde{F}$ is Lipschitz continuous, then $F$ has a linear
derivative $f(m,x)$ that is continuous in both variables, and the relation
(3.4) holds.

###### Proof.

See [33, Propositions 5.48 and 5.51], where the Wasserstein gradient is
referred to as the L-derivative. ∎

For a further comparison of different notions of differentiability on the
Wasserstein space, see [33, Chapter 5] and [54].

We say that a measure $m$ is _empirical_ (or _discrete_) if it is the weighted
sum of Dirac masses. We will often deal with the particular case of an
empirical measure with $n$ points equally weighted: for each vector ${\bf
x}=(x_{1},\ldots,x_{n})\in({\mathbb{R}}^{d})^{n}$, we write $m_{\bf
x}^{n}=\frac{1}{n}\sum_{j=1}^{n}\delta_{x_{j}}$. We have the following
“isometry” relation:

(3.5) $W_{2}(m_{\bf x}^{n},m_{\bf
y}^{n})^{2}=\min\mathinner{\left\\{\frac{1}{n}\sum_{j=1}^{n}\mathinner{\\!\left\lvert
x_{j}-y_{\sigma(j)}\right\rvert}^{2}\mathrel{\mathop{\ordinarycolon}}\sigma\in
S_{n}\right\\}}$

where $S_{n}$ is the permutation group on $\\{1,\ldots,n\\}$. If we can
relabel the indices of ${\bf y}$ so that they are in the optimal order with
respect to ${\bf x}$, we can then write $W_{2}(m_{\bf x}^{n},m_{\bf
y}^{n})=n^{-1/2}\mathinner{\\!\left\lvert{\bf x}-{\bf y}\right\rvert}$.

For a function
$F\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{2}({\mathbb{R}}^{d})\to{\mathbb{R}}$,
we will define
$F_{n}\mathrel{\mathop{\ordinarycolon}}({\mathbb{R}}^{d})^{n}\to{\mathbb{R}}$
by $F_{n}({\bf x})=F(m_{\bf x}^{n})$. It is straightforward to show that

(3.6) $D_{x_{j}}F_{n}({\bf x})=\frac{1}{n}D_{\mu}F(m_{\bf x}^{n},x_{j}).$

See [33, Proposition 5.35].

We say that $F$ is _displacement convex_ provided the lifted version
$\tilde{F}(X)$ is convex; in this case we also have that $F_{n}({\bf x})$ is a
convex function on $({\mathbb{R}}^{d})^{n}$. We say that a continuous function
$f\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{2}({\mathbb{R}}^{d})\times{\mathbb{R}}^{d}\to{\mathbb{R}}$
with continuous derivative $D_{x}f$ is _displacement monotone_ provided that
for every $X,Y\in{\mathbb{H}}$ we have

(3.7)
${\mathbb{E}}\mathinner{\left[\mathinner{\left(D_{x}f(\sharp(X),X)-D_{x}f(\sharp{Y},Y)\right)}(X-Y)\right]}\geq
0.$

If $D_{x}f=D_{\mu}F$, then $F$ is displacement convex if and only if $f$ is
displacement monotone [52]. Essentially, this follows from the fact that
$\tilde{F}$ is convex if and only if $D\tilde{F}$ is monotone; cf. Proposition
3.1.

Another type of monotonicity, which is more common in mean field game theory,
is called the Lasry-Lions (LL) condition [76]. We say that
$f\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{2}({\mathbb{R}}^{d})\times{\mathbb{R}}^{d}\to{\mathbb{R}}$
is LL monotone provided that

(3.8)
$\int_{{\mathbb{R}}^{d}}\mathinner{\left(f(m_{1},x)-f(m_{2},x)\right)}\operatorname{d\\!}(m_{1}-m_{2})(x)\geq
0\quad\forall m_{1},m_{2}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d}),$

and we say it is _strictly_ LL monotone provided that equality implies
$f(m_{1},x)=f(m_{2},x)$ for all $x$. See [62] for a further discussion of
monotonicity in mean field games. Note that if $f$ is a linear derivative of
$F\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{2}({\mathbb{R}}^{d})\to{\mathbb{R}}$,
then $f$ is LL monotone if and only if $F$ is convex in the usual sense, i.e.
$F(tm_{1}+(1-t)m_{2})\leq tF(m_{1})+(1-t)F(m_{2})$ for all
$m_{1},m_{2}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})$ and $t\in[0,1]$.

###  3.2. A reduced problem

Throughout the remainder of this section, let us assume that the running cost
$f\equiv 0$ and $L(x,v)=\frac{1}{2}\mathinner{\\!\left\lvert
v\right\rvert}^{2}$, that is,

(3.9)
$J(\eta,\gamma)=\frac{1}{2}\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(t)\right\rvert}^{2}\operatorname{d\\!}t+g\mathinner{\left(m^{\eta}(T),\gamma(T)\right)}.$

The potential reduces to

(3.10)
${\mathcal{J}}(\eta)=\frac{1}{2}\int_{\Gamma}\int_{0}^{T}\mathinner{\\!\left\lvert\dot{\gamma}(t)\right\rvert}^{2}\operatorname{d\\!}t\operatorname{d\\!}\eta(\gamma)+G\mathinner{\left(m^{\eta}(T)\right)}.$

We will assume for simplicity that $G$ is has a continuous Wasserstein
gradient $D_{\mu}G(m,x)$ and that the gradient $D\tilde{G}(X)$ is Lipschitz
continuous. It follows from Proposition 3.1 that $g(m,x)$ and $D_{x}g(m,x)$
are continuous and that $D_{\mu}G=D_{x}g$. We will also assume that the state
space is $\Omega={\mathbb{R}}^{d}$.

To minimize ${\mathcal{J}}$ defined in (3.9), it is sufficient to travel in
straight lines from the initial measure $m_{0}$, as the following lemma makes
clear. For $m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})$ we define

(3.11)
${\mathcal{K}}(m_{T})=\frac{1}{2T}W_{2}\mathinner{\left(m_{0},m_{T}\right)}^{2}+G\mathinner{\left(m_{T}\right)}.$

######  Lemma 3.2.

$\inf\mathinner{\left\\{{\mathcal{J}}(\eta)\mathrel{\mathop{\ordinarycolon}}\eta\in{\mathscr{P}}_{m_{0}}(\Gamma)\right\\}}=\inf\mathinner{\left\\{{\mathcal{K}}(m_{T})\mathrel{\mathop{\ordinarycolon}}m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})\right\\}}$,
i.e. to minimize ${\mathcal{J}}(\eta)$ it suffices to consider $\eta$
concentrated on straight lines.

###### Proof.

First, for $\pi\in{\mathscr{P}}_{2}({\mathbb{R}}^{d}\times{\mathbb{R}}^{d})$
we define

(3.12)
${\mathcal{K}}^{\prime}(\pi)=\frac{1}{2T}\int_{{\mathbb{R}}^{d}\times{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x-y\right\rvert}^{2}\operatorname{d\\!}\pi(x,y)+G\mathinner{\left(p^{2}\sharp\pi\right)}$

where $p^{2}$ is the projection $p^{2}(x,y)=y$. We claim

(3.13)
$\inf\mathinner{\left\\{{\mathcal{J}}(\eta)\mathrel{\mathop{\ordinarycolon}}\eta\in{\mathscr{P}}_{m_{0}}(\Gamma)\right\\}}=\inf\mathinner{\left\\{{\mathcal{K}}^{\prime}(\pi)\mathrel{\mathop{\ordinarycolon}}\pi\in{\mathscr{P}}({\mathbb{R}}^{d}\times{\mathbb{R}}^{d})\mathrel{\mathop{\ordinarycolon}}p^{1}\sharp\pi=m_{0}\right\\}}$

where $p^{1}(x,y)=x$. For any
$(x,y)\in{\mathbb{R}}^{d}\times{\mathbb{R}}^{d}$, we set

(3.14) $\gamma_{x,y}(t)=x+\frac{t}{T}(y-x).$

The map
$S\mathrel{\mathop{\ordinarycolon}}{\mathbb{R}}^{d}\times{\mathbb{R}}^{d}\to\Gamma$
given by $S(x,y)=\gamma_{x,y}$ is continuous. For any
$\pi\in{\mathscr{P}}_{2}({\mathbb{R}}^{d}\times{\mathbb{R}}^{d})$ we see that
${\mathcal{J}}(S\sharp\pi)={\mathcal{K}}^{\prime}(\pi)$, using the fact that
$\dot{\gamma}(t)=1/T$. Moreover, if $p^{1}\sharp\pi=m_{0}$, then
$S\sharp\pi\in{\mathscr{P}}_{m_{0}}(\Gamma)$ since $e_{0}\circ S=p^{1}$. It
follows immediately that (3.13) holds with $\leq$ in place of $=$. To prove
the opposite inequality, let $\eta\in{\mathscr{P}}_{m_{0}}(\Gamma)$. By
Jensen’s inequality and the fundamental theorem of calculus,

(3.15)
${\mathcal{J}}(\eta)\geq\frac{1}{2T}\int_{\Gamma}\mathinner{\\!\left\lvert\gamma(T)-\gamma(0)\right\rvert}^{2}\operatorname{d\\!}\eta(\gamma)+G\mathinner{\left(m^{\eta}(T)\right)}=\frac{1}{2T}\int_{\Gamma}\mathinner{\\!\left\lvert
y-x\right\rvert}^{2}\operatorname{d\\!}\mathinner{\left((e_{0},e_{T})\sharp\eta\right)}(x,y)+G\mathinner{\left(e_{T}\sharp\eta\right)},$

i.e. ${\mathcal{J}}(\eta)\geq{\mathcal{K}}^{\prime}(\pi)$ where
$\pi=(e_{0},e_{T})\sharp\eta$. This proves (3.13).

We now argue that

(3.16)
$\inf\mathinner{\left\\{{\mathcal{K}}^{\prime}(\pi)\mathrel{\mathop{\ordinarycolon}}\pi\in{\mathscr{P}}({\mathbb{R}}^{d}\times{\mathbb{R}}^{d})\mathrel{\mathop{\ordinarycolon}}p^{1}\sharp\pi=m_{0}\right\\}}=\inf\mathinner{\left\\{{\mathcal{K}}(m_{T})\mathrel{\mathop{\ordinarycolon}}m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})\right\\}}.$

If $\pi\in{\mathscr{P}}({\mathbb{R}}^{d}\times{\mathbb{R}}^{d})$ with
$p^{1}\sharp\pi=m_{0}$, then letting $m_{T}=p^{2}\sharp\pi$ yields
$\int_{{\mathbb{R}}^{d}\times{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x-y\right\rvert}^{2}\operatorname{d\\!}\pi(x,y)\geq W_{2}(m_{0},m_{T})^{2}$
and thus ${\mathcal{K}}^{\prime}(\pi)\geq{\mathcal{K}}(m_{T})$. Conversely, if
$m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})$, then we can find an optimal
transport plan $\pi$ with $p^{1}\sharp\pi=m_{0}$ and $p^{2}\sharp\pi=m_{T}$
such that
$\int_{{\mathbb{R}}^{d}\times{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x-y\right\rvert}^{2}\operatorname{d\\!}\pi(x,y)=W_{2}(m_{0},m_{T})^{2}$ and
thus ${\mathcal{K}}^{\prime}(\pi)={\mathcal{K}}(m_{T})$. Equation (3.16)
follows, which is what we needed to show. ∎

Fix an arbitrary $X\in{\mathbb{H}}$ whose law is $m_{0}$, and define for any
$Y\in{\mathbb{H}}$ the functional

(3.17)
$\tilde{{\mathcal{K}}}(Y)=\frac{1}{2T}{\mathbb{E}}\mathinner{\\!\left\lvert
X-Y\right\rvert}^{2}+G\mathinner{\left(\sharp(Y)\right)}.$

The proof of the following lemma is found, for example, in the lectures by
Lions [78], see [23, Remark 6.9].

######  Lemma 3.3.

$\inf\mathinner{\left\\{{\mathcal{K}}(m_{T})\mathrel{\mathop{\ordinarycolon}}m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})\right\\}}=\inf\mathinner{\left\\{\tilde{{\mathcal{K}}}(Y)\mathrel{\mathop{\ordinarycolon}}Y\in{\mathbb{H}}\right\\}}$.

###### Proof.

For the reader’s convenience, we give some details of the proof. If
$Y\in{\mathbb{H}}$, set $m_{T}=\sharp(Y)$; from (3.1) we get
$\tilde{{\mathcal{K}}}(Y)\geq{\mathcal{K}}(m_{T})$. Conversely, let
$m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})$ be given. There exist
$X^{\prime},Y^{\prime}\in{\mathbb{H}}$ such that
${\mathbb{E}}\mathinner{\\!\left\lvert
X^{\prime}-Y^{\prime}\right\rvert}^{2}=W_{2}(m_{0},m_{T})$,
$\sharp(X^{\prime})=m_{0}$ and $\sharp(Y^{\prime})=m_{T}$. Since $X^{\prime}$
and $X$ have the same law, there exists a bijection
$S\mathrel{\mathop{\ordinarycolon}}B\to B$ such that $S$ and $S^{-1}$ are both
measure preserving and $X^{\prime}=X\circ S$. Now let $Y=Y^{\prime}\circ
S^{-1}$, so that $\sharp(Y)=\sharp(Y^{\prime})=m_{T}$ and
${\mathbb{E}}\mathinner{\\!\left\lvert
X-Y\right\rvert}^{2}={\mathbb{E}}\mathinner{\\!\left\lvert
X^{\prime}-Y^{\prime}\right\rvert}^{2}=W_{2}(m_{0},m_{T})^{2}$. It follows
that ${\mathcal{K}}(m_{T})=\tilde{{\mathcal{K}}}(Y)$. This completes the
proof. ∎

Before continuing, we remark that it is not difficult to prove the existence
of minimizers (and therefore equilibria) by assuming $G$ is also continuous in
a slightly stronger sense, e.g.
$G\mathrel{\mathop{\ordinarycolon}}{\mathscr{P}}_{p}({\mathbb{R}}^{d})\to{\mathbb{R}}$
is continuous for some $p<2$. In this case one can appeal to the arguments in
[70, Section 4]. More directly, suppose $m_{T,n}$ is a minimizing sequence for
${\mathcal{K}}$. By the assumptions on $G$ we deduce that

(3.18) $G(m_{T,n})\leq
C\mathinner{\left(1+\mathinner{\left(\int_{{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x\right\rvert}^{2}\operatorname{d\\!}m_{T,n}\right)}^{1/2}\right)}$

for some constant $C$. Using the inequality

(3.19) $\int_{{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x\right\rvert}^{2}\operatorname{d\\!}m_{T,n}\leq
2W_{2}(m_{0},m_{T,n})^{2}+2\int_{{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x\right\rvert}^{2}\operatorname{d\\!}m_{0},$

we deduce that there is a uniform bound on
$\int_{{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
x\right\rvert}^{2}\operatorname{d\\!}m_{T,n}$. Thus $\\{m_{T,n}\\}$ is tight
and has uniformly integrable $p$th moments for $p<2$. By Prokhorov’s Theorem
and [5, Proposition 7.1.5] we can pass to a subsequence that converges in
${\mathscr{P}}_{p}({\mathbb{R}}^{d})$ to some
$m_{T}\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})$. Using the weak lower-
semicontinuity of the $W_{2}$ metric [5, Lemma 7.1.4], we deduce that $m_{T}$
is a minimizer.

###  3.3. Necessary and sufficient conditions for uniqueness of minimizers and
equilibria

We are now ready to state an important result relating uniqueness of
minimizers to the potential to the uniqueness of a Nash equilibrium. The
result we will prove is certainly not as general as possible. However, it
communicates an important point: displacement monotonicity of the coupling is
_necessary_ as well as _sufficient_ for uniqueness of the equilibrium. By
contrast, Lasry-Lions monotonicity is a sufficient condition for uniqueness of
the _value function_ for Nash equilibrium, i.e. the function $u$ in System
(1.4). See e.g. [20] for a proof in the case of “mild” solutions. See also
[2]. On the other hand, by [62, Proposition 3.5], LL monotonicity does not
imply uniqueness of the equilibrium distribution, for the simple reason that
mass can split in different directions. Intuitively, uniqueness of the cost to
players does not imply uniqueness of the paths they take to reach that cost;
this requires convexity of the cost $g(m,x)$ with respect to the state
variable $x$, which holds if it is displacement monotone.

######  Theorem 3.4.

In addition to the regularity assumed above, suppose that $g(m,x)$ is convex
with respect to the $x$-variable. The following are equivalent:

  1. (1)

$G$ is displacement convex.

  2. (2)

$g$ is displacement monotone.

  3. (3)

For every $T>0$, ${\mathcal{J}}$ has a unique minimizer.

  4. (4)

For every $T>0$, there is a unique Nash equilibrium for the cost $J$ given in
(3.9).

Before getting to the proof of Theorem 3.4, we make a few remarks. First, it
is natural to conjecture that the theorem holds even without the added
assumption that $g(m,x)$ is convex with respect to $x$; however, the infinite
dimensional structure of the problem makes it difficult to prove. Second, with
this additional assumption, it is worth noting that LL monotonicity implies
displacement monotonicity and therefore is sufficient to prove uniqueness
[52]. Third, the assumption that $g(m,x)$ is convex with respect to $x$
implies that, for a given $m$, each player has only one optimal decision
available. This has an important consequence for the structure of the game: it
means that if we take an initial measure that is empirical (discrete) with $n$
points, then both the game and its potential become $n$-dimensional problems.
This can be seen as follows. To start with the potential, we see that
minimizers of $\tilde{{\mathcal{K}}}$ satisfy the first order condition

(3.20) $\frac{Y-X}{T}+D\tilde{G}(Y)=0\Leftrightarrow
Y+TD_{x}g(\sharp(Y),Y)=X.$

Since $g(m,x)$ is assumed to be convex with respect to $x$, the function
$I+tD_{x}g(m,\cdot)$ is invertible for every $T\geq 0$ and
$m\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})$. The equilibrium is thus
characterized by

(3.21) $Y=\mathinner{\left(I+TD_{x}g(\sharp(Y),\cdot)\right)}^{-1}(X).$

In particular, if $\sharp(X)$ is an empirical measure of the form $m_{\bf
x}^{n}=\frac{1}{n}\sum_{j=1}^{n}\delta_{x_{j}}$, then $\sharp(Y)=m_{\bf
y}^{n}$ where $y_{j}=\mathinner{\left(I+tD_{x}g(m_{\bf
y}^{n},\cdot)\right)}^{-1}(x_{j})$. In this case, it is sufficient to optimize
over vectors in $({\mathbb{R}}^{d})^{n})$, i.e. we can replace
$\tilde{{\mathcal{K}}}$ with

(3.22) $\tilde{{\mathcal{K}}}_{n}({\bf
y})=\frac{1}{2Tn}\mathinner{\\!\left\lvert{\bf x}-{\bf
y}\right\rvert}^{2}+G_{n}({\bf y}).$

The same considerations apply to the game itself. For a given vector ${\bf
z}$, each individual starting at a point $x\in{\mathbb{R}}^{d}$ seeks to
minimize the cost

(3.23) $\frac{\mathinner{\\!\left\lvert x-y\right\rvert}^{2}}{2T}+g(m_{\bf
z}^{n},y),$

which implies that player will choose

(3.24) $y=\mathinner{\left(I+TD_{x}g(m_{\bf z}^{n},\cdot)\right)}^{-1}(x).$

It follows that, starting from the discrete measure $m_{\bf x}^{n}$, the final
measure will be $m_{\bf y}^{n}$ with $y_{j}=\mathinner{\left(I+TD_{x}g(m_{\bf
z}^{n},\cdot)\right)}^{-1}(x_{j})$. A Nash equilibrium in this case is
equivalent to a vector ${\bf z}$ such that ${\bf y}={\bf z}$, i.e.
$y_{j}=\mathinner{\left(I+TD_{x}g(m_{\bf y}^{n},\cdot)\right)}^{-1}(x_{j})$,
which is the same condition as for minimizers of the potential.

###### Proof of Theorem 3.4.

We have already observed that (1) and (2) are equivalent [51].

To see that (1) implies (3), note that if $G$ is displacement convex, then
$\tilde{K}$ defined in (3.17) is strictly convex, so it has a unique
minimizer. If $m_{T}$ is a minimizer of ${\mathcal{K}}$, then the proof of
Lemma 3.3 implies it is the law of $Y$, hence $m_{T}$ is unique. Similarly, by
the proof of Lemma 3.2, ${\mathcal{J}}$ has a unique minimizer, as desired.

To show that (1) implies (4), see [51]. To see that (4) implies (3), note that
every minimizer of ${\mathcal{J}}$ is an equilibrium, so if there is more than
one minimizer there is more than one equilibrium.

To see that (4) implies (1), note that if ${\mathcal{J}}$ has a unique
minimizer, then $\tilde{{\mathcal{K}}}$ must have a unique minimizer $Y$. We
are going to show that $\tilde{G}$ must be convex. Assume to the contrary that
it is not convex. Then there exist random variables $X,Y$ such that

(3.25)
$\tilde{G}(X)>\frac{1}{2}\mathinner{\left(\tilde{G}(X+Y)+\tilde{G}(X-Y)\right)}.$

Any $X\in{\mathbb{H}}$ can be approximated up to arbitrary precision by a
discrete random variable whose law is $m_{\bf x}^{n}$ for some sufficiently
large $n$. By the continuity of $G$, (3.25) implies that for some sufficiently
large $n$, $G_{n}$ is not convex. It follows that for some ${\bf
x}\in({\mathbb{R}}^{d})^{n}$ and $T>0$, $\tilde{{\mathcal{K}}}_{n}$ defined in
(3.22) has more than one minimizer, using arguments that hold in finite
dimensional settings (see e.g. [62, Theorem A.1]; note that the linear growth
of $\tilde{G}$ assumed here is sufficient to make this argument go through).
This is a contradiction; therefore, $\tilde{G}$ must be convex, i.e. $G$ is
displacement convex. The proof is complete. ∎

Notice that the proof of Theorem 3.4 uses the assumption that $g(m,x)$ is
convex with respect to $x$ only to prove that (4) implies (1). If this
implication could be proved without restricting to finite dimensions, then the
extra assumption could be eliminated.

###  3.4. Hamilton-Jacobi equation and master equation

We now introduce the value function

(3.26)
$U(t,\mu)\mathrel{\mathop{\ordinarycolon}}=\inf\mathinner{\left\\{\frac{1}{2t}W_{2}(\mu,\nu)^{2}+G(\nu)\mathrel{\mathop{\ordinarycolon}}\nu\in{\mathscr{P}}_{2}({\mathbb{R}}^{d})\right\\}}.$

The “lifted” version of it is

(3.27)
$\tilde{U}(t,X)\mathrel{\mathop{\ordinarycolon}}=\inf\mathinner{\left\\{\frac{1}{2t}\mathinner{\\!\left\lVert
X-Y\right\rVert}^{2}+\tilde{G}(Y)\mathrel{\mathop{\ordinarycolon}}Y\in{\mathbb{H}}\right\\}}=U\mathinner{\left(t,\sharp(X)\right)},$

where the second equality follows from Lemma 3.3. By classical results [41,
42], $\tilde{U}$ is the unique viscosity solution of the Hamilton-Jacobi
equation

(3.28) $\partial_{t}\tilde{U}+\frac{1}{2}\mathinner{\\!\left\lVert
D_{X}\tilde{U}\right\rVert}^{2}=0,\quad\tilde{U}(0,X)=\tilde{G}(X).$

By [54], $U$ itself is the unique viscosity solution (see the reference for a
definition) of the corresponding PDE on Wasserstein space:

(3.29)
$\partial_{t}U+\frac{1}{2}\int_{{\mathbb{R}}^{d}}\mathinner{\\!\left\lvert
D_{\mu}U(t,m,x)\right\rvert}^{2}\operatorname{d\\!}m(x)=0,\quad U(0,m)=G(m).$

Let us suppose for the moment that $U$ is smooth. Taking a linear derivative
of (3.29), we deduce that $U$ has a linear derivative $u=u(t,m,x)$ satisfying

(3.30) $\partial_{t}u(t,m,x)+\frac{1}{2}\mathinner{\\!\left\lvert
D_{x}u(t,m,x)\right\rvert}^{2}+\int_{{\mathbb{R}}^{d}}D_{y}u(t,m,y)\cdot
D_{\mu}u(t,m,x,y)\operatorname{d\\!}m(y)=0,$

and such that $u(0,m,x)$ is a linear derivative of $G$. Equation (3.30) is the
_master equation_ for the mean field game. It can be seen as the limit as
$N\to\infty$ of the system of $N$ coupled Hamilton-Jacobi equations for an
$N$-player symmetric game, or it may be seen as the decoupling equation for
the system of PDE given by (1.4); see e.g. [26, 10, 11] for a full exposition.
In the context of potential mean field games, it is not a mere coincidence
that the master equation (3.30) is obtained from taking the linear derivative
of the Hamilton-Jacobi equation (3.29). Indeed, $U$ is the value function for
the minimization of the potential, while $u$ is the value function for an
individual player; more precisely, $u(t,m,x)$ is the minimal cost for a player
starting at position $x$ with time horizon $t$, where $m$ is the initial
distribution of players in the game. Just as the cost to individuals $J$ is
assumed to have a potential ${\mathcal{J}}$, the individual value function
$u=\min J$ in turn has a potential $U=\min{\mathcal{J}}$. Several results on
classical solutions to the master equation are based on precisely this idea:
start with a smooth solution to the Hamilton-Jacobi equation on the space of
measures, and then differentiate to get the master equation [13, 12, 51]; see
also [53].

The master equation is, both in the present context and more generally, a
nonlinear transport equation on the space of measures. The forward-backward
system of PDE, of which (1.4) is a standard class, can be seen as the
characteristics for this transport. The whole problem of finding a Nash
equilibrium can be viewed as a problem of identifying a characteristic that
intersects a given point $(t,m)$. If we feed the system the measure $m_{0}=m$
as an initial condition and the time $T=t$ as a time horizon, the solution
$u(0,x)$ of (1.4), which is the value function over the whole time horizon for
a representative player, can be identified as the solution to the master
equation for inputs $(t,m,x)$. In the context of potential games, one might
expect that the derivative of a Hamilton-Jacobi equation should be a Burgers
type equation, and the characteristics of the one should correspond to the
characteristics of the other. Indeed, this connection can be made clearer by
considering $D_{x}u(t,m,x)=D_{\mu}U(t,m,x)$, which formally satisfies
$D_{x}u(t,\sharp(X),X)=\tilde{\bf u}(t,X)$ where

(3.31) $\partial_{t}\tilde{\bf u}(t,X)+{\mathbb{E}}\mathinner{\left[\tilde{\bf
u}(t,X)\cdot D_{X}\tilde{\bf u}(t,X)\right]}=0,\quad\tilde{\bf
u}(0,X)=D\tilde{G}(X).$

(Cf. [44].) This is often referred to as the _vectorial master equation_.

When we assume that $g(m,x)$ is convex with respect to $x$, we can make a more
direct connection with classical theory. In this case define the restriction
to empirical measures:

(3.32) $U_{n}(t,{\bf x})=U(t,m_{\bf
x}^{n})=\inf\mathinner{\left\\{\frac{1}{2tn}\mathinner{\\!\left\lvert{\bf
x}-{\bf y}\right\rvert}^{2}+G_{n}({\bf y})\right\\}}.$

Then $U_{n}$ is the unique viscosity solution [40, 39] to the Hamilton-Jacobi
equation

(3.33) $\partial_{t}U_{n}+\frac{n}{2}\mathinner{\\!\left\lvert D_{\bf
x}U_{n}\right\rvert}^{2}=0,\quad U_{n}(0,{\bf x})=G_{n}({\bf x}).$

By Krushkov’s results [72], the gradient ${\bf u}_{n}(t,{\bf x})=D_{\bf
x}U_{n}$ is the unique entropy solution of the hyperbolic system

(3.34) $\partial_{t}{\bf u}_{n}+n\mathinner{\left({\bf u}_{n}\cdot D_{\bf
x}\right)}{\bf u}_{n}={\bf 0},\quad{\bf u}_{n}(0,{\bf
x})=\frac{1}{n}D_{x}g(m_{\bf x}^{n},{\bf x}),$

where we have used (3.6). Indeed, the well-posedness of (3.34) is a corollary
of the well-posedness of (3.33); much less is known about hyperbolic systems
that are not the gradient of a Hamilton-Jacobi equation.

###  3.5. Selection

We have seen that a potential game could very well have multiple Nash
equilibria (Theorem 3.4). Indeed, one need only take particular instances of
(3.34) for which it is well-known that characteristics cross, for instance by
taking $d=n=1$ and assuming $D_{x}g(\delta_{x},x)$ is a decreasing function of
$x$. The problem of selecting a particular Nash equilibrium for a mean field
game has been a subject of active research in recent years. First of all,
Delarue has shown that uniqueness in mean field games can be restored by
adding a common noise [44]. This suggests a strategy akin to Selten’s idea in
defining trembling hand perfect equilibrium: if we add a common noise to the
game and allow the intensity to converge to zero, this will select one of
multiple equilibria. In this vein, Delarue and Tchuendom studied a linear-
quadratic mean field game, and showed that the Burgers equation, not
coincidentally played a key role in selection [46]. Cecchin and Delarue showed
that, for a finite state mean field game, the vanishing noise limit would
select the entropy solution to the master equation [36]. Presumably motivated
by these results, Cecchin and Delarue have also introduced a notion of weak
solution for potential mean field games with continuous state space [37], and
it is precisely the kind of “entropy solution” one should expect based on the
following analogy. The derivative of the unique viscosity solution of a
(classical) Hamilton-Jacobi equation is the unique entropy solution of the
hyperbolic system obtained by taking the gradient of the Hamilton-Jacobi
equation; the master equation is the derivative of a Hamilton-Jacobi equation
on the space of measures; therefore, the derivative of the solution to the
Hamilton-Jacobi equation on the space of measures should be the unique entropy
solution to the master equation. Using a highly technical approach, Cecchin
and Delarue introduce a measure ${\mathbb{P}}$ on
${\mathscr{P}}_{2}({\mathbb{T}}^{d})$ (where ${\mathbb{T}}^{d}$ is the torus
in $d$ dimensions) such that uniqueness of the entropy solution holds in a
${\mathbb{P}}$-a.e. sense. Their approach involves approximating measures
using Fourier coefficients, which is suitable in the case where diffusion
terms appear in the dynamics so that the continuity equation becomes a second-
order PDE with a Laplacian. Establishing a general uniqueness theorem for
entropy solutions in the first-order case seems to be an open problem.

Let us now clarify why the concept of entropy solution is a natural choice for
a selection principle. The characteristics for the (vectorial) master equation
(3.31) are curves $X(s)$ satisfying

(3.35) $\dot{X}(s)=\tilde{\bf
u}(s,X(s)),\quad\dfrac{\operatorname{d\\!}{}}{\operatorname{d\\!}{s}}\tilde{\bf
u}(s,X(s))=0,$

hence $\tilde{\bf u}(s,X(s))=D\tilde{G}(X(0))$ for all $s$. Setting $Y=X(0)$,
we deduce that a characteristic hitting $X$ at time $t$ is given by
$X(s)=Y+sD\tilde{G}(Y)$ where $Y$ solves $Y+tD\tilde{G}(Y)=X$. Rewriting this
condition as $Y+tD_{x}g(\sharp(Y),Y)=X$, we see it is exactly the same as the
definition of Nash equilibrium (i.e. given that $Y$ represents the anticipated
final distribution, $Y$ is the optimal response from starting position $X$).
We see that the Nash equilibrium is unique if and only if characteristics do
not cross. The classical theory of entropy solutions was developed precisely
to deal with the case when characteristics do cross; see e.g. [72, 73, 43, 77,
89]. Of course, there are two key differences between the present question of
finding Nash equilibria and the classical problem of studying nonlinear fluid
flow. The first difference is that our present setting is infinite
dimensional. This can be partially alleviated if we take the assumption that
$g(m,x)$ is convex with respect to $x$, and we take our initial measure to be
discrete. Then the equilibrium corresponds to finding characteristics in
$({\mathbb{R}}^{d})^{n}$ of the classical conservation law (3.34).

The second difference is apparently much more fundamental. In the classical
theory of conservation laws, we are interested in fluid flow, and therefore
our criterion for selecting a solution to the PDE should be motivated by
physics. Indeed, the name “entropy solution” comes from the principle that
physical entropy must increase with time. On the other hand, if we are trying
to choose a Nash equilibrium, it is far from clear whether the notion of
increasing entropy is meaningful. Nevertheless, we there is at least one other
physical interpretation that could be useful: by [72], the unique entropy
solution to (3.34) is the limit as $\varepsilon\to 0$ of solutions to the
parabolic equations

(3.36) $\partial_{t}{\bf u}_{n}+n\mathinner{\left({\bf u}_{n}\cdot D_{\bf
x}\right)}{\bf u}_{n}=\varepsilon\Delta{\bf u}_{n},\quad{\bf u}_{n}(0,{\bf
x})=\frac{1}{n}D_{x}g(m_{\bf x}^{n},{\bf x}).$

The Laplacian term $\varepsilon\Delta{\bf u}_{n}$ comes from diffusion, which
in physics arise from viscosity of a fluid but in a game can arise from
uncertainty or “noise.” The interpretation is that players cannot be certain
the crowd will evolve according to the characteristic ${\bf x}(s)$ (or, more
properly speaking, $m_{{\bf x}(s)}^{n}$), so they assume that ${\bf x}(s)$ is
perturbed by a Brownian motion with small variance and that the value ${\bf
u}_{n}(s,{\bf x}(s))$ is a martingale. The perturbation is thus a “common
noise,” which affects the trajectory of the whole crowd, to be distinguished
from idiosyncratic noise, which affects each player independently [35]. The
entropy solution of (3.34) thus gives us the _vanishing common noise limit
equilibrium_ , provided the common noise is Brownian motion.

Here we see the basic idea of Selten [88, 67] coming through quite clearly.
Recall that when the set of strategies is finite, players’ uncertainty can be
expressed by assigning a uniform small probability to every alternative
strategy, and the result will be “trembling hand perfect equilibrium.” Since
the strategy set in this game is infinite, it is not possible to perturb the
set of actions by a uniform distribution, but it is possible to choose a
normal distribution (via Brownian motion), and this, too, will result in a
refinement of Nash equilibrium (the vanishing common noise limit). As for the
fact that we have restricted to finite dimensions, this can be interpreted in
the following way. Given an arbitrary initial distribution, players will all
take the same discrete approximation of it. They will then play the game as if
that discrete approximation were the true distribution. In principle, the
final discrete distribution should be a good approximation of the true
equilibrium.

Notice that the vanishing common noise limit equilibrium, which is the entropy
solution, is also given by the gradient of the value function for the
potential. In fact, we can use the potential to designate a selection
principle without restricting to finite dimensions. Observe that for $t>0$,
the value function $\tilde{U}$ given in (3.27) is Lipschitz and semi-concave,
i.e. there exists a constant $C$ such that

(3.37) $\lambda\tilde{U}(t,X)+(1-\lambda)\tilde{U}(t,Z)-\tilde{U}(t,\lambda
X+(1-\lambda)Z)\leq\frac{C}{t}\lambda(1-\lambda)\mathinner{\\!\left\lVert
X-Z\right\rVert}^{2}$

for all $\lambda\in(0,1)$ and $X,Z\in{\mathbb{H}}$. (In this case we can take
$C=1$.) By a result of Albano and Cannarsa [3], the set of points $\Sigma$ on
which $\tilde{U}(t,X)$ is not differentiable is countably $\infty-1$
rectifiable, meaning that it is contained in the countable union of sets, each
of which is the image under a Lipschitz map of a bounded subset of a subspace
with codimension 1. In other words, the set of $X$ for which
$D_{X}\tilde{U}(t,X)$ exists is “very large” in a certain sense, similar to
the structure of BV functions [49]. For such $X$, we see that
$Y=X-tD_{X}\tilde{U}(t,X)$ is the unique minimizer of the potential; this
follows from standard arguments showing that the optimal control is unique at
points where the value function is differentiable [22]. In the present
context, we can see this by observing that if $Y$ is any minimizer then

(3.38) $\tilde{U}(t,X+Z)-\tilde{U}(t,X)\leq\frac{\mathinner{\\!\left\lVert
X+Z-Y\right\rVert}^{2}}{2t}+\tilde{G}(Y)-\frac{\mathinner{\\!\left\lVert
X-Y\right\rVert}^{2}}{2t}-\tilde{G}(Y)=\left\langle\frac{X-Y}{t},Z\right\rangle+\frac{1}{2t}\mathinner{\\!\left\lVert
Z\right\rVert}^{2}$

for all $Z$, hence $\frac{X-Y}{t}=D_{X}\tilde{U}(t,X)$.

Note that even when the minimizer of the potential is unique, the equilibrium
may not be unique. A simple but illustrative case is when
$G(m)=\int\phi\operatorname{d\\!}m$ for some smooth function $\phi$ such that
$\phi^{\prime}(x)=1$ when $x\leq-1$, $\phi^{\prime}(x)=0$ when $x\geq 0$, and
$\phi^{\prime}(x)=-x$ when $-1\leq x\leq 0$. Take $d=1$ and take the initial
distribution to be a Dirac mass $\delta_{x}$. The Nash equilibrium condition
is to find a $y$ such that $y+t\phi^{\prime}(y)=x$, while the potential to be
minimized is $\frac{\mathinner{\left(x-y\right)}^{2}}{2t}+\phi(y)$. One checks
that, for $t>1$ (the time when the shock occurs) and for $0<x<t$, there are
three equilibria given by $y=x$, $y=x-t$, and $y=x/(1-t)$, but the only
minimizer is $y=x$ when $x>(t-1)/2$ (to the right of the shock) and $y=x-t$
when $x<(t-1)/2$ (to the left of the shock). This corresponds precisely to
solving the Burgers equation with a shock.

Since the minimizer $Y=X+tD_{X}\tilde{U}(t,X)$ selects only one of possibly
many equilibria, we can say that minimizing the potential is itself a sort of
selection principle. It is unclear whether it can be interpreted as a
vanishing noise limit; the difficulty is both in defining the noise on an
infinite dimensional space (but see [44]) and in proving rigorously that the
limit has the desired meaning. If we assume that $g(m,x)$ is convex with
respect to $x$, then for $\sharp(X)=m_{\bf x}^{n}$ we have
$D_{X}\tilde{U}(t,X)={\bf u}_{n}(t,{\bf x})$, which is exists for a.e. ${\bf
x}\in({\mathbb{R}}^{d})^{n}$ (in the sense of Lebesgue measure) and agrees
with the entropy solution of (3.34). For this reason a sort of vanishing noise
limit interpretation is plausible, at least in some cases. However, the
general case seems to be quite open.

Even if we can resolve the infinite dimensionality of the problem, there are
more general difficulties in justifying the selection principle proposed here.
One difficulty, which is highlighted by the work of Lions and Seeger [79], is
that there are infinitely many types of vanishing noise one can add to
characteristics, each resulting in a different selection. Indeed, given _any_
curve $c(t)$ such that $c^{\prime}(t)\in(0,1)$ for all $t\geq 0$, there is a
function $\theta_{c}(t)$ such that the limit of solutions to the PDE

(3.39)
$\partial_{t}u+u\partial_{x}u=\varepsilon\mathinner{\left(\partial_{xx}^{2}u+\theta_{c}(t)\mathinner{\\!\left\lvert\partial_{x}u\right\rvert}^{2}\right)},\quad
u(0,x)=\begin{cases}1,\quad x<0,\\\ 0,\quad x>0\end{cases}$

is the piecewise defined “solution” to the Burgers equation given by

(3.40) $u(t,x)=\begin{cases}1,\quad x<c(t),\\\ 0,\quad x>c(t).\end{cases}$

See [79, Proposition 5.4 and Theorem 5.4]. The meaning of this result is that
the region in which characteristics cross can be divided into two regions
using a more or less arbitrary “shock” curve (which in general does not at all
correspond to the physical entropy criterion), and the selected solution can
still be obtained by a vanishing noise limit. Therefore, the phrase “vanishing
noise limit” does not uniquely specify a selection criterion.

Two possible responses may be given in favor of the entropy solution. First,
for any other vanishing noise limit, the perturbed characteristics are not
martingales. If the notion of martingale is interpreted as an “unbiased”
stochastic perturbation, then that could be seen as an argument in favor of
the entropy solution. On the other hand, if there is reason to suspect that
the noise will be biased in one direction or another, then the entropy
solution no longer provides the appropriate selection principle. The second
possible response is that, as we have seen, the entropy solution also
corresponds to choosing minimizers of the potential. If there is a plausible
evolutionary mechanism by which the population converges only to minimizers of
the potential (cf. [29, 86, 81]), then it is natural to use the entropy
solution to select the appropriate Nash equilibrium. On the other hand, this
justification is contingent on finding and arguing for such an evolutionary
mechanism.

Other problems using entropy solutions as a selection principle can arise when
the mean field game is _not_ potential. See [63] for a discussion of some
examples.

##  4\. Conclusion

In this article, we have briefly surveyed the concept of a potential mean
field game and connected it to what was already known about potential games.
We have only discussed the first-order case. Second-order problems may be more
or less challenging, depending on the structure of the second-order terms and
how they affect the dynamics of the game. For instance, adding an
idiosyncratic noise that affects each individual independently could make
equations in (1.4) strictly parabolic, in which case a number of PDE
techniques can be brought to bear on the analysis [84, 59, 57, 58, 30]. On the
other hand, the addition of a common Brownian motion, affecting all
individuals simultaneously, does not appear to add additional smoothness to
solutions of the standard master equation [26]. Insofar as we have dealt with
the idea of a common noise in this article, it has only been in the case where
the game is essentially finite dimensional; the infinite dimensional analog is
not trivial to define [44].

In any case, restricting to the first-order case has allowed us, first of all,
to provide a transparent proof that minimizers of the potential are also Nash
equilibria, defined according to a Lagrangian framework. This can be done even
in state-constrained problems. Second, we have been able to make a clear
connection between solving a mean field game and analyzing characteristics of
a nonlinear hyperbolic balance law. This has the benefit of motivating
potential approaches to the selection of equilibria, which is a major open
problem in mean field game theory. The recent breakthrough by Cecchin and
Delarue [37] seems to confirm that potential mean field games are the natural
place to start a rigorous analysis of selection principles. It is to be hoped
that the present article helps clarify the main issues, and that it may serve
as a catalyst for new developments.

Conflict of interest. The author declares that he does not have any conflicts
of interests.

Data Availability Statement. Data sharing not applicable to this article as no
datasets were generated or analyzed during the current study.

## References

  * [1] Y. Achdou, P. Cardaliaguet, F. Delarue, A. Porretta, and F. Santambrogio, Mean Field Games: Cetraro, Italy 2019, vol. 2281, Springer Nature, 2021\.
  * [2] Y. Achdou, P. Cardaliaguet, F. Delarue, A. Porretta, F. Santambrogio, P. Cardaliaguet, and A. Porretta, An introduction to mean field game theory, Mean Field Games: Cetraro, Italy 2019, (2020), pp. 1–158.
  * [3] P. Albano and P. Cannarsa, Singularities of Semiconcave Functions in Banach Spaces, Birkhäuser Boston, Boston, MA, 1999, pp. 171–190.
  * [4] L. Ambrosio, M. Fornasier, M. Morandotti, and G. Savaré, Spatially inhomogeneous evolutionary games, Communications on Pure and Applied Mathematics, 74 (2021), pp. 1353–1402.
  * [5] L. Ambrosio, N. Gigli, and G. Savaré, Gradient flows: in metric spaces and in the space of probability measures, Springer Science & Business Media, 2008.
  * [6] J.-D. Benamou and Y. Brenier, A computational fluid mechanics solution to the Monge-Kantorovich mass transfer problem, Numerische Mathematik, 84 (2000), pp. 375–393.
  * [7] J.-D. Benamou and G. Carlier, Augmented lagrangian methods for transport optimization, mean field games and degenerate elliptic equations, Journal of Optimization Theory and Applications, 167 (2015), pp. 1–26.
  * [8] J.-D. Benamou, G. Carlier, and F. Santambrogio, Variational mean field games, in Active Particles, Volume 1, Springer, 2017, pp. 141–171.
  * [9] A. Bensoussan, J. Frehse, and P. Yam, Mean field games and mean field type control theory, Springer, 2013.
  * [10] A. Bensoussan, J. Frehse, and S. C. P. Yam, The master equation in mean field theory, Journal de Mathématiques Pures et Appliquées, 103 (2015), pp. 1441–1474.
  * [11] , On the interpretation of the master equation, Stochastic Processes and their Applications, 127 (2017), pp. 2093–2137.
  * [12] A. Bensoussan, P. J. Graber, and S. C. P. Yam, Control on Hilbert Spaces and Application to Mean Field Type Control Theory, arXiv preprint, (2020).
  * [13] A. Bensoussan and P. Yam, Control problem on space of random variables and master equation, ESAIM: COCV, (2018).  https://doi.org/10.1051/cocv/2018034.
  * [14] J. F. Bonnans, J. Gianatti, and L. Pfeiffer, A lagrangian approach for aggregative mean field games of controls with mixed and final constraints, SIAM Journal on Control and Optimization, 61 (2023), pp. 105–134.
  * [15] J. F. Bonnans, S. Hadikhanloo, and L. Pfeiffer, Schauder estimates for a class of potential mean field games of controls, Applied Mathematics & Optimization, (2019), pp. 1–34.
  * [16] Y. Brenier, Polar factorization and monotone rearrangement of vector-valued functions, Communications on Pure and Applied Mathematics, 44 (1991), pp. 375–417.
  * [17] A. Briani and P. Cardaliaguet, Stable solutions in potential mean field game systems, Nonlinear Differential Equations and Applications NoDEA, 25 (2018), p. 1.
  * [18] G. W. Brown, Iterative solution of games by fictitious play, Activity analysis of production and allocation, 13 (1951), pp. 374–376.
  * [19] P. Cannarsa and R. Capuani, Existence and uniqueness for mean field games with state constraints, in PDE Models for Multi-Agent Phenomena, Springer, 2018, pp. 49–71.
  * [20] P. Cannarsa, R. Capuani, and P. Cardaliaguet, $c^{1,1}$-smoothness of constrained solutions in the calculus of variations with application to mean field games, Mathematics in Engineering, 1 (2019), pp. 174–203.
  * [21] , Mean field games with state constraints: from mild to pointwise solutions of the pde system, Calculus of Variations and Partial Differential Equations, 60 (2021), pp. 1–33.
  * [22] P. Cannarsa and C. Sinestrari, Semiconcave functions, Hamilton-Jacobi equations, and optimal control, vol. 58, Springer Science & Business Media, 2004.
  * [23] P. Cardaliaguet, Notes on mean field games, from P.-L. Lions’ lectures at College de France, (2010).
  * [24] , Weak solutions for first order mean field games with local coupling, in Analysis and Geometry in Control Theory and its Applications, vol. 11 of Springer INdAM Series, Springer, 2015, pp. 111–158.
  * [25] P. Cardaliaguet, G. Carlier, and B. Nazaret, Geodesics for a class of distances in the space of probability measures, Calculus of Variations and Partial Differential Equations, 48 (2013), pp. 395–420.
  * [26] P. Cardaliaguet, F. Delarue, J. Lasry, and P. Lions, The Master Equation and the Convergence Problem in Mean Field Games: (AMS-201), Annals of Mathematics Studies, Princeton University Press, 2019.
  * [27] P. Cardaliaguet and P. J. Graber, Mean field games systems of first order, ESAIM: COCV, 21 (2015), pp. 690–722.
  * [28] P. Cardaliaguet, P. J. Graber, A. Porretta, and D. Tonon, Second order mean field games with degenerate diffusion and local coupling, Nonlinear Differential Equations and Applications NoDEA, 22 (2015), pp. 1287–1317.
  * [29] P. Cardaliaguet and S. Hadikhanloo, Learning in mean field games: The fictitious play, ESAIM: Control, Optimisation and Calculus of Variations, 23 (2017), pp. 569–591.
  * [30] P. Cardaliaguet, J.-M. Lasry, P.-L. Lions, A. Porretta, et al., Long time average of mean field games, Networks and Heterogeneous Media, 7 (2012), pp. 279–301.
  * [31] P. Cardaliaguet and C.-A. Lehalle, Mean field game of controls and an application to trade crowding, Mathematics and Financial Economics, 12 (2018), pp. 335–363.
  * [32] P. Cardaliaguet, A. R. Mészáros, and F. Santambrogio, First order mean field games with density constraints: pressure equals price, SIAM Journal on Control and Optimization, 54 (2016), pp. 2672–2709.
  * [33] R. Carmona and F. Delarue, Probabilistic Theory of Mean Field Games: vol. I, Mean Field FBSDEs, Control, and Games, Stochastic Analysis and Applications, Springer Verlag, (2017).
  * [34] , Probabilistic Theory of Mean Field Games: vol. II, Mean Field Games with Common Noise and Master Equations, Stochastic Analysis and Applications. Springer Verlag, (2017).
  * [35] R. Carmona, F. Delarue, and D. Lacker, Mean field games with common noise, The Annals of Probability, 44 (2016), pp. 3740 – 3803.
  * [36] A. Cecchin and F. Delarue, Selection by vanishing common noise for potential finite state mean field games, Communications in Partial Differential Equations, 47 (2022), pp. 89–168.
  * [37] , Weak solutions to the master equation of potential mean field games, arXiv preprint arXiv:2204.04315, (2022).
  * [38] J.-F. Chassagneux, D. Crisan, and F. Delarue, A probabilistic approach to classical solutions of the master equation for large population equilibria, (2022).
  * [39] M. G. Crandall, H. Ishii, and P.-L. Lions, User’s guide to viscosity solutions of second order partial differential equations, Bulletin of the American Mathematical Society, 27 (1992), pp. 1–67.
  * [40] M. G. Crandall and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Transactions of the American Mathematical Society, 277 (1983), pp. 1–42.
  * [41] , Hamilton-jacobi equations in infinite dimensions i. uniqueness of viscosity solutions, Journal of functional analysis, 62 (1985), pp. 379–396.
  * [42] , Hamilton-jacobi equations in infinite dimensions. ii. existence of viscosity solutions, Journal of Functional Analysis, 65 (1986), pp. 368–405.
  * [43] C. M. Dafermos, Hyperbolic systems of conservation laws, in Systems of nonlinear partial differential equations, Springer, 1983, pp. 25–70.
  * [44] F. Delarue, Restoring uniqueness to mean-field games by randomizing the equilibria, Stochastics and Partial Differential Equations: Analysis and Computations, 7 (2019), pp. 598–678.
  * [45] F. Delarue, D. Lacker, and K. Ramanan, From the master equation to mean field game limit theory: a central limit theorem, Electronic Journal of Probability, 24 (2019), pp. 1 – 54.
  * [46] F. Delarue and R. F. Tchuendom, Selection of equilibria in a linear quadratic mean-field game, Stochastic Processes and their Applications, 130 (2020), pp. 1000–1040.
  * [47] I. Ekeland and R. Temam, Convex analysis and variational problems, vol. 28, SIAM, 1976.
  * [48] L. Evans, Partial Differential Equations, Graduate Studies in Mathematics Series, Amer Mathematical Society, 2010.
  * [49] L. C. Evans and R. F. Gariepy, Measure theory and fine properties of functions, vol. 5, CRC press, 1992.
  * [50] M. Fischer and F. J. Silva, On the asymptotic nature of first order mean field games, Applied Mathematics & Optimization, 84 (2021), pp. 2327–2357.
  * [51] W. Gangbo and A. R. Mészáros, Global well-posedness of master equations for deterministic displacement convex potential mean field games, Communications on Pure and Applied Mathematics, 75 (2022), pp. 2685–2801.
  * [52] W. Gangbo, A. R. Mészáros, C. Mou, and J. Zhang, Mean field games master equations with nonseparable hamiltonians and displacement monotonicity, The Annals of Probability, 50 (2022), pp. 2178–2217.
  * [53] W. Gangbo and A. Święch, Existence of a solution to an equation arising from the theory of mean field games, Journal of Differential Equations, 259 (2015), pp. 6573–6643.
  * [54] W. Gangbo and A. Tudorascu, On differentiability in the wasserstein space and well-posedness for hamilton–jacobi equations, Journal de Mathématiques Pures et Appliquées, 125 (2019), pp. 119–174.
  * [55] D. A. Gomes et al., Mean field games models–a brief survey, Dynamic Games and Applications, 4 (2014), pp. 110–154.
  * [56] D. A. Gomes, S. Patrizi, and V. Voskanyan, On the existence of classical solutions for stationary extended mean field games, Nonlinear Analysis: Theory, Methods & Applications, 99 (2014), pp. 49–79.
  * [57] D. A. Gomes and E. Pimentel, Time-dependent mean-field games with logarithmic nonlinearities, SIAM Journal on Mathematical Analysis, 47 (2015), pp. 3798–3812.
  * [58] D. A. Gomes, E. Pimentel, and H. Sánchez-Morgado, Time-dependent mean-field games in the superquadratic case, ESAIM: Control, Optimisation and Calculus of Variations, 22 (2016), pp. 562–580.
  * [59] D. A. Gomes, E. A. Pimentel, and H. Sánchez-Morgado, Time-dependent mean-field games in the subquadratic case, Communications in Partial Differential Equations, 40 (2015), pp. 40–76.
  * [60] D. A. Gomes and V. K. Voskanyan, Extended deterministic mean-field games, SIAM Journal on Control and Optimization, 54 (2016), pp. 1030–1055.
  * [61] P. J. Graber, Optimal control of first-order Hamilton–Jacobi equations with linearly bounded Hamiltonian, Applied Mathematics & Optimization, 70 (2014), pp. 185–224.
  * [62] P. J. Graber and A. R. Mészáros, On monotonicity conditions for mean field games, Journal of Functional Analysis, 285 (2023), p. 110095.
  * [63] P. J. Graber and A. R. Mészáros, On some mean field games and master equations through the lens of conservation laws, Mathematische Annalen, (2024).
  * [64] P. J. Graber and C. Mouzouni, Variational mean field games for market competition, in PDE Models for Multi-Agent Phenomena, vol. 28 of Springer INdAM Series, 2018.
  * [65] P. J. Graber, A. Mullenix, and L. Pfeiffer, Weak solutions for potential mean field games of controls, Nonlinear Differential Equations and Applications NoDEA, 28 (2021), pp. 1–34.
  * [66] J. C. Harsanyi, A new theory of equilibrium selection for games with complete information, Games and Economic Behavior, 8 (1995), pp. 91–122.
  * [67] J. C. Harsanyi and R. Selten, A general theory of equilibrium selection in games, MIT Press Books, 1 (1988).
  * [68] J. Hillas, On the definition of the strategic stability of equilibria, Econometrica: Journal of the Econometric Society, (1990), pp. 1365–1390.
  * [69] M. Huang, R. P. Malhamé, and P. E. Caines, Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the nash certainty equivalence principle, Communications in Information & Systems, 6 (2006), pp. 221–252.
  * [70] R. Hynd and H. K. Kim, Value functions in the wasserstein spaces: finite time horizons, Journal of Functional Analysis, 269 (2015), pp. 968–997.
  * [71] E. Kalai and D. Samet, Persistent equilibria in strategic games, International Journal of Game Theory, 13 (1984), pp. 129–144.
  * [72] S. Kružkov, Generalized solutions of nonlinear first order equations with several independent variables. ii, Mathematics of the USSR-Sbornik, 1 (1967), p. 93.
  * [73] S. N. Kružkov, First order quasilinear equations in several independent variables, Mathematics of the USSR-Sbornik, 10 (1970), p. 217.
  * [74] J.-M. Lasry and P.-L. Lions, Jeux à champ moyen. I–Le cas stationnaire, Comptes Rendus Mathématique, 343 (2006), pp. 619–625.
  * [75] , Jeux à champ moyen. II–Horizon fini et contrôle optimal, Comptes Rendus Mathématique, 343 (2006), pp. 679–684.
  * [76] , Mean field games, Japanese Journal of Mathematics, 2 (2007), pp. 229–260.
  * [77] P. D. Lax, Hyperbolic systems of conservation laws and the mathematical theory of shock waves, SIAM, 1973.
  * [78] P.-L. Lions, Théorie des jeux de champ moyen et applications (mean field games), Cours du College de France, (2007-2012).
  * [79] P.-L. Lions and B. Seeger, Linear and nonlinear transport equations with coordinate-wise increasing velocity fields, arXiv preprint arXiv:2307.05819, (2023).
  * [80] G. Mazanti and F. Santambrogio, Minimal-time mean field games, Mathematical Models and Methods in Applied Sciences, 29 (2019), pp. 1413–1464.
  * [81] D. Monderer and L. S. Shapley, Fictitious play property for games with identical interests, Journal of economic theory, 68 (1996), pp. 258–265.
  * [82] , Potential games, Games and economic behavior, 14 (1996), pp. 124–143.
  * [83] R. B. Myerson, Refinements of the nash equilibrium concept, International journal of game theory, 7 (1978), pp. 73–80.
  * [84] A. Porretta, Weak solutions to Fokker–Planck equations and mean field games, Archive for Rational Mechanics and Analysis, 216 (2015), pp. 1–62.
  * [85] R. W. Rosenthal, A class of games possessing pure-strategy nash equilibria, International Journal of Game Theory, 2 (1973), pp. 65–67.
  * [86] W. H. Sandholm, Potential games with continuous player sets, Journal of Economic theory, 97 (2001), pp. 81–108.
  * [87] , Large population potential games, Journal of Economic Theory, 144 (2009), pp. 1710–1725.
  * [88] R. Selten, Reexamination of the perfectness concept for equilibrium points in extensive games, International Journal of Game Theory, 4 (1975), pp. 25–55.
  * [89] J. Smoller, Shock waves and reaction—diffusion equations, vol. 258, Springer Science & Business Media, 2012.
  * [90] E. Van Damme, Refinements of the Nash equilibrium concept, vol. 219, Springer Science & Business Media, 2012.
  * [91] C. Villani, Optimal transport: old and new, vol. 338, Springer Science & Business Media, 2008.