# A Measure Theoretical Approach to the Mean-field Maximum Principle for
Training NeurODEs

Benoît Bonnet Email<EMAIL_ADDRESS>Inria Paris and Laboratoire
Jacques-Louis Lions, Sorbonne Université, Université Paris-Diderot SPC, CNRS,
Inria, 75005 Paris, France  Cristina Cipriani Email:
<EMAIL_ADDRESS>Massimo Fornasier Email:
<EMAIL_ADDRESS>Hui Huang Email<EMAIL_ADDRESS>University
of Calgary, Department of Mathematics and Statistics, Calgary, Canada

###### Abstract

In this paper we consider a measure-theoretical formulation of the training of
NeurODEs in the form of a mean-field optimal control with
$L^{2}$-regularization of the control. We derive first order optimality
conditions for the NeurODE training problem in the form of a mean-field
maximum principle, and show that it admits a unique control solution, which is
Lipschitz continuous in time. As a consequence of this uniqueness property,
the mean-field maximum principle also provides a strong quantitative
generalization error for finite sample approximations, yielding a rigorous
justification of the double descent phenomenon. Our derivation of the mean-
field maximum principle is much simpler than the ones currently available in
the literature for mean-field optimal control problems, and is based on a
generalized Lagrange multiplier theorem on convex sets of spaces of measures.
The latter is also new, and can be considered as a result of independent
interest.

Keywords: NeurODEs, Mean-Field Optimal Control, Mean-Field Maximum Principle,
Lagrange Multiplier Theorem

###### Contents

  1. 1 Introduction
    1. 1.1 Deep learning
    2. 1.2 Training of deep nets and residual blocks
    3. 1.3 NeurODEs and stochastic optimal control
    4. 1.4 Measure-theoretical approach to mean-field optimal control
    5. 1.5 Contributions and organization of the paper
  2. 2 Preliminaries and notations
    1. 2.1 Analysis in measure spaces and optimal transport
    2. 2.2 Continuity equations in the space of measures
    3. 2.3 Differential calculus over convex subsets of Banach spaces
  3. 3 Existence of minimizers and stability of solutions
    1. 3.1 Convexity of the reduced cost functional and existence of minimizers
    2. 3.2 Stability of finitely-sampled costs and controls
  4. 4 Mean-Field Maximum Principle
    1. 4.1 Formal derivation of the Lagrangian maximum principle
    2. 4.2 Well-posedness of the maximum principle
    3. 4.3 Rigorous derivation of the mean-field maximum principle
      1. 4.3.1 A Lagrange Multiplier Theorem over convex sets
      2. 4.3.2 Preparation and verification of assumptions
      3. 4.3.3 The mean-field PMP for continuous controls: a Lagrangian approach
      4. 4.3.4 The mean-field PMP for measurable controls: an Hamiltonian approach
  5. 5 Numerical experiments
    1. 5.1 General setting
    2. 5.2 Results
  6. A Well-posedness continuity equations and properties of characteristic flows
  7. B Regularity of ODE flows with respect to the control variables
  8. C Proof of Theorem 4.5

##  1 Introduction

###  1.1 Deep learning

Deep learning is an established computational approach that performs state-of-
the-art on various relevant real-life applications such as speech
[hannun2014deep] and image [NIPS2012_4824, 7780459] recognition, language
translation [NIPS2017_7181], and which also serves as a basis for novel
scientific computing methods [Berner2020, elbraechter2020dnn]. In unsupervised
machine learning, deep neural networks have shown great success as well, for
instance in image and speech generation [pmlr-v48-oord16, oord2016wavenet],
and in reinforcement learning for solving control problems, such as mastering
Atari games [nature15] or beating human champions at playing Go
[silver2017mastering]. Deep learning is about realizing complex tasks as the
ones mentioned above, by means of highly parametrized functions, called deep
artificial neural networks
$\mathcal{N}:\mathbb{R}^{d_{0}}\to\mathbb{R}^{d_{L}}$. A classical
architecture is the one of feed-forward artificial neural networks of the type

$\mathcal{N}(x)=\rho\big{(}W_{L}^{\top}\rho\big{(}W_{L-1}^{\top}\dots\rho\big{(}W_{1}^{\top}x+\tau_{1}\big{)}\dots\big{)}+\tau_{L}\big{)},$
(1.1)

where the matrices $W_{\ell}\in\mathbb{R}^{d_{\ell-1}\times d_{\ell}}$
represent collections of weights, the vectors
$\tau{{}_{\ell}}\in\mathbb{R}^{d_{\ell}}$ are shifts/biases for each layer
$\ell=1,\ldots,L$ and $\rho$ is a scalar activation function acting component-
wisely on vectors. Below, we shall denote by
$\mathcal{F}(X):=\rho(W^{\top}X+\tau)$ a generic layer of the network. In
practical applications, the number $L\geq 1$ of layers – determining the depth
of the network – and the dimensions $d_{\ell-1}\times d_{\ell}$ of the weight
matrices $W_{\ell}$ are typically determined by means of heuristic
considerations, whereas the weight matrices and the shifts are free parameters
which are tuned in various possible ways by using a given training dataset.

Practical evidences towards certified benchmarks confirm that deep-learning
algorithms are able to outperform many of the previously existing methods.
Also, recent mathematical investigations [mhaskar2020function,
shaham2018provable, Berner2020, elbraechter2020dnn, elbrachter2020deep,
daubechies2019, PETERSEN2018296, cloninger2020relu, devore2020neural,
mhaskar2016deep] have proven that deep artificial networks can approximate
high dimensional functions without incurring in the curse of dimensionality,
i.e. without needing a number of parameters (here the weights and shifts of
the network) that is exponential with respect to the input dimension in order
to approximate high-dimensional functions. While the approximation properties
– also called the expressivity – of neural networks are becoming more and more
understood and transparent [gühring2020expressivity], the training phase
itself, based on suitable optimization processes, remains a (black-)box with
some levels of opacity. In fact, the latter procedure features a surprising
and yet mostly unexplained phenomenon, which is in stark contrast with
conventional statistics wisdom: in addition to providing a finer empirical
data fitting, increasing the number of modelling parameters beyond that of
training examples also tends to improve the generalization error, namely the
prediction error on unseen data. This simultaneous decrease of both empirical
and generalization errors is called the double descent phenomenon. Instead,
from classical statistical learning theory [books/daglib/0033642], one would
expect that overfitting should lead to a blow-up of the generalization error,
owing to the wealth of complextiy of the underlying model
[zhang2016understanding]. Hence the prediction of the generalization error
from data remains at large a fundamental open problem in deep learning. As one
of the main results of this paper, we show that for certain classes of neural
networks based on dynamical systems, whose training is reformulated as an
optimal control problem, the double descent phenomenon can be rigorously
explained.

###  1.2 Training of deep nets and residual blocks

In order to understand the context of our results, let us mention how the
neural networks considered in this paper arise. We start by recalling how
training of neural networks is performed and how it is facilitated by
appropriate network architectures. The method that is most frequently used to
train deep neural networks is the so-called backpropagation of error
[werbos74, 10.5555/104279.104293, 7fa6b6a5cde14bcfbd7ab3a8f19d0d56], which is
justified by its tremendous empirical success. Inherently, all the practical
advances recalled above are due to the efficacy of this method. The term
backpropagation111In fact, “backpropagation” refers more precisely to a
recursive way of applying the chain rule needed to compute the gradient of the
loss with respect to weights, but it is often used also to describe any
algorithmic optimization procedure resorting to such gradients. In many cases,
these latter are computed using symbolic calculus. usually refers to the use
of stochastic gradient descent or some of its variants [sun2019optimization]
to minimize a given loss function (e.g. mean-squared distances, Kullback-
Leibler divergences, or Wasserstein distances) over the parameters of the
network (the weights and biases), usually measuring the misfit of input-output
information over a finite number of labeled training samples. On the one hand,
the practical efficiency of deep learning is currently ensured in the so-
called overparametrized regime by fitting a large amount of data with a larger
amount of parameters. On the other hand, solving learning problems with very
large numbers of layers gets increasingly harder with the total depth of the
network, as the resulting non-convex optimization problems become in turn very
high-dimensional.

In their groundbreaking work [7780459], He et al. showed that the training
error of the $56$-layer CNN network remains worse than that of a $20$-layer
network for the same problem, highlighting an issue which could be blamed
either on the optimization function, on initialization of the network, or on
the vanishing/exploding gradient phenomenon. The problem of training very deep
networks has been alleviated with the introduction of a new neural network
layer called the “Residual Block”, see Figure 1.

Figure 1: The layer update reads: $X^{n+1}=X^{n}+{\mathcal{F}}(X^{n})$, see
[7780459].

According to the analysis conveyed in [he2016identity], the use of identity
mappings as skip connections and after-addition activations of the form

$X^{n+1}=X^{n}+\mathcal{F}(X^{n})$ (1.2)

turns out to be beneficial to promote the smoothness of the information
propagation. Therein, the authors present several $1000$-layer deep networks
that can be easily trained and achieves improved accuracy. The use of such
skip connections with identity mappings presupposes a rectangular shape of the
network for which the depths $d_{\ell+1}=d_{\ell}$ of the layers are all
identical.

###  1.3 NeurODEs and stochastic optimal control

While originally the arguments in [he2016identity] that support the use of
residual blocks are based on empirical considerations, a recent line of
research has been devoted to a more mathematical and rigorous formulation of
deep neural networks with residual blocks in terms of dynamical systems. In
this context, the training of the network can be interpreted as a large
optimal control problem, an insight that was proposed independently by E
Weinan [eweinan17] and Haber-Ruthotto [Haber_2017]. Later on, this dynamical
approach has been greatly popularized in the machine learning community under
the name of NeurODE by Chen et al. [10.5555/3327757.3327764], see also
[DBLP:journals/corr/abs-1908-10920]. The formulation starts by reinterpreting
the iteration (1.2) as a step of the discrete-time Euler approximation
[avelin2020neural] of the following dynamical system

$\dot{X}_{t}=\mathcal{F}(t,X_{t},\theta_{t})\,,$ (1.3)

with initial condition $X_{0}\in\mathbb{R}^{d}$. Here, the map
$\mathcal{F}:\mathbb{R}_{+}\times\mathbb{R}^{d}\times\mathbb{R}^{m}\rightarrow\mathbb{R}^{d}$
represents the feed-forwarding dynamics, the parameter
$\theta_{t}\in\mathbb{R}^{m}$ is a general control variable, which encodes the
weights and shifts of the network, i.e. $\theta_{t}:=(W_{t},\tau_{t})$. A
prototypical example is given by

$\mathcal{F}(t,X_{t},\theta_{t})=\rho(W_{t}X_{t}+\tau_{t}),$ (1.4)

for instance with an activation function $\rho:=\tanh$ acting componentwisely
on its entries. In [eweinan17, eweinan19], the authors proposed a stochastic
control formulation of the training of this nonlinear process, with a detailed
analysis of the related optimality conditions. Therein, both the the Hamilton-
Jacobi-Bellman equations [CannarsaS2004] – based on the well-known dynamic
programming principle – and the Pontryagin Maximum Principle [Pontryagin] were
studied in great generality. From another perspective, several recent works
[Agrachev2020, Agrachev2021, Tabuada2021] in geometric control theory have
aimed at explaining the efficiency of NeurODEs in approximating large classes
of mappings in terms of controllability properties of such systems in the
group of diffeomorphisms.

In this paper, we focus on a particular measure theoretical reformulation of
the general approach developed by E Weinan et al. [eweinan19], which allows us
to derive more specific properties of the control problem, such as the
existence, uniqueness, and smoothness of solutions to the Pontryagin Maximum
Principle, and a strong form of generalization error estimates. Most
importantly, our approach encompasses the prototypical model (1.4) as a
possible application. Consider two random variables $X_{0}$ and $Y_{0}$ which
are jointly distributed according to a law
$\mu_{0}\in\mathcal{P}(\mathbb{R}^{2d})$, and let us fix the depth $T>0$ of
the time-continuous neural network (1.3). Training this network then amounts
to learning the control signals $\theta\in L^{2}([0,T];\mathbb{R}^{m})$ in
such a way that the terminal output $X_{T}$ of (1.3) is close to $Y_{0}$, with
respect to some distortion measure $\ell(\cdot,\cdot)\in\mathcal{C}^{2}$. A
typical choice is $\ell(x,y)=:|x-y|^{2}$, which is often called the squared
loss function in the machine learning literature. The stochastic optimal
control problem can hence be posed as

$\inf\limits_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}J(\theta)=\left\\{\begin{aligned}
&\inf\limits_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}\mathbb{E}_{\mu_{0}}\Big{[}\ell(X_{T},Y_{0})\Big{]}+\lambda\int_{0}^{T}|\theta_{t}|^{2}\,{\rm
d}t,\\\ &\hskip 5.69046pt\text{s.t.}~{}\left\\{\begin{aligned}
&\dot{X}_{t}=\mathcal{F}(t,X_{t},\theta_{t}),\\\
&(X_{t},Y_{0})_{|t=0}\sim\mu_{0}.\end{aligned}\right.\end{aligned}\right.$
(1.5)

The use of a regularization term of the type
$\lambda\int_{0}^{T}|\theta_{t}|^{2}\,{\rm d}t$ is very standard in machine
learning, see e.g. [Goodfellow-et-al-2016, Chapter 7] or [Kukacka-et-al-2017,
Section 6]. In the absence of regularization, the resulting trained networks
may have huge Lipschitz constants, rendering them extremely unstable and
susceptible to adversarial attacks [Goodfellow2015ExplainingAH]. Additionally,
the regularization may significantly help the usual training processes, by
making the loss $J$ increasingly more convex. As we shall see more in details
below, such a standard regularization will allow us to establish the existence
and uniqueness of solutions for (1.5), as well as their continuity with
respect to the data, which provides a rigorous explaination to the stability
of trained networks and the double descent phenomenon. Conversely, we shall
also demonstrate numerically in Section 5.2 that the lack of a sufficient
regularization causes significant instabilities in the numerical solution of
the optimal control problem (1.5), see Figure 7, rendering the latter
absolutely essential from a practical standpoint. Other and more general
regularizations are of course possible [Kukacka-et-al-2017], but for the sake
of simplicity and clarity in the exposition, we shall restrict our attention
to this specific one.

###  1.4 Measure-theoretical approach to mean-field optimal control

In this paper, we develop a new point of view that is equivalent to that of
[eweinan19], but which is not based on stochastic control considerations. We
start by providing a measure-theoretic reformulation of (1.5), which can be
interpreted as a generalized optimal transport problem or mean-field optimal
control problem. To the best of our knowledge, the present paper is the first
in the literature to make such a connection. To this end, let us define a new
stochastic process $Z_{t}:=(X_{t},Y_{t})$ satisfying

$\displaystyle\dot{X}_{t}=\mathcal{F}(t,X_{t},\theta_{t})\qquad\text{and}\qquad\dot{Y}_{t}=0,$
(1.6)

with initial data $(X_{0},Y_{0})$ distributed according to $\mu_{0}$, and
denote the law of $(X_{t},Y_{t})$ by $\mu_{t}(x,y)$. It is well-known that
$\mu_{t}$ satisfies the following partial differential equation

$\partial_{t}\mu_{t}+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta_{t})\mu_{t})=0,\qquad\mu_{t}|_{t=0}=\mu_{0}\,,$
(1.7)

understood in the sense of distributions as in Definition 2.2 below. With this
transport equation at hand, we can recast the stochastic optimal control
problem (1.5) as

$\inf_{\theta\in L^{2}([0,T];\mathbb{R}^{m})}J(\theta)=\left\\{\begin{aligned}
&\inf_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}\int_{\mathbb{R}^{2d}}\ell(x,y)\mathrm{d}\mu_{T}(x,y)+\lambda\int_{0}^{T}|\theta_{t}|^{2}\,{\rm
d}t\,,\\\ &\hskip 5.69046pt\text{s.t.}~{}\left\\{\begin{aligned}
&\partial_{t}\mu_{t}+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta_{t})\mu_{t})=0\,,\\\
&\mu_{t}|_{t=0}=\mu_{0}\,.\end{aligned}\right.\end{aligned}\right.$ (1.8)

Therein, the goal is again is to find the control signal $\theta$ for which
$J(\theta)$ is minimal when $\mu$ satisfies the PDE constraint (1.7). Observe
that when the initial measure $\mu_{0}$ is empirical, i.e.

$\mu_{0}:=\mu_{0}^{N}=\frac{1}{N}\sum_{i=1}^{N}\delta_{(X_{0}^{i},Y_{0}^{i})}$

then the optimal control problem (1.8) reduces to a classical finite particle
optimal control problem with ODE constraints.

Optimal control problems over spaces of probability measures of the form (1.8)
have been recently explored, mostly in the absence of final-point constraints
and in the context of multi-agent interactions. The first contributions on
this topic [fornasier2014mean, MR3268059] were concerned with the rigorous
convergence of classical finite particle optimal controls towards their mean-
field counterparts, see also the more recent work [LipReg,
cavagnari2020lagrangian, fornasier_lisini_orrieri_savaré_2019]. The derivation
of first order optimality conditions, i.e., the so-called Pontryagin Maximum
Principle (PMP), has been proposed for the first time in [BFRS15] based on the
leader-follower model studied in [MR3268059]. In this work, the mean-field
Pontryagin Maximum Principle is derived as limit of its classical finite-
particle counterpart. The first general derivation of the PMP for mean-field
optimal control problems was obtained in [PMPWass], and is based on a careful
adaptation of the strategy of needle-variations to the abstract geometric
structure of Wasserstein spaces. These results were further extended in
[PMPWassConst] to problems with general final-point and running state
constraints. In the latter contribution, the proof strategy combines a finite-
dimensional non-smooth multipliers rule and outer-approximations of optimal
trajectories by countable families of curves generated using needle-
variations. Very recently, a simpler approach has been proposed in
[SetValuedPMP], by adapting to the notion of multivalued dynamics in
Wasserstein space introduced in [ContInc] a methodology originally developed
in [Frankowska1990], which relies on suitable linearisations of set-valued
maps that produce admissible inner-perturbed trajectories. From a different
standpoint, we also mention [burger] in which a KKT approach is developed in
Wasserstein spaces for rather general mean-field optimal control problems with
$H^{1}$-controls. Therein, both the first order optimality conditions and
their relationships with finite particle approximations are derived, along
with the corresponding rates of convergence. We finally point out that a
completely different approach to the mean-field PMP was formulated for
stochastic optimal control problems in [10.1214/14-AOP946] inspired by the
theory of mean-field games [lali07] (see also [befrph13, ACFK16]). Similar
methods, based on needle-variations in the space of measures are also
leveraged in [eweinan19] and [Jabir2021] for the derivation of the PMP for
stochastic control problems of the form (1.5).

###  1.5 Contributions and organization of the paper

The contributions of this paper can be summarized as follows. From a global
standpoint, we start by establishing existence and stability results for
(1.8), based on compactness and $\Gamma$-convergence arguments. We then
proceed by deriving general first-order optimality conditions for the measure-
theoretic formulation of the optimal control of NeurODEs. Our modeling
assumptions include the typical forward mappings (1.4) that appear throughout
the literature related to neural networks, with for instance $\rho:=\tanh$. As
a matter of fact, most of the results available in the literature do not fully
encompass this simple model, as they often require global Lipschitz bounds on
the transport velocity field.

Let us now describe with more details the fundamental results of the paper. In
Section 3, we start by showing that the mean-field optimal control problem
(1.8) has solution when the regularization parameter $\lambda>0$ is
sufficiently large, and that the latter is in fact unique. By leveraging
compactness arguments akin to that classically appearing in the theory of
$\Gamma$-convergence, we also establish non-quantitative stability results for
the training problem with respect to finite-samples, both at the level of the
cost and of the controls. We then proceed by investigating first-order
optimality conditions in Section 4. We initiate the discussion by providing in
Section 4.1 a heuristic derivation of the following mean-field Pontryagin
Maximum Principle (“PMP” in the sequel)

$\left\\{\begin{aligned}
\partial_{t}\mu_{t}&+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta_{t})\mu_{t})=0,\hskip
14.22636pt\mu_{t}|_{t=0}=\mu_{0}\,,\\\
\partial_{t}\psi&+\nabla_{x}\psi\cdot\mathcal{F}(t,x,\theta_{t})=0,\hskip
14.22636pt\psi_{t}|_{t=T}=\ell\,,\\\
\theta^{\top}_{t}&=-\frac{1}{2\lambda}\int_{\mathbb{R}^{2d}}\nabla_{x}\psi\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t})\mathrm{d}\mu_{t}(x,y)\,,\end{aligned}\right.$
(1.9)

which characterizes optimal trajectory-control pairs $(\mu,\theta)$ for (1.8).
In Section 4.2, we show that the above optimality system is well-posed, and
prove in Theorem 4.1 that it admits a unique control solution
$\theta^{*}\in\operatorname{Lip}([0,T];\mathbb{R}^{m})$. Consequently, we are
able to show that the function $\mu_{0}\to\theta^{*}$ which maps initial data
distributions to the optimal parameters is single-valued, and to prove that it
is also Lipschitz continuous with respect to the Wasserstein distance. Such a
precise description of how data are encoded in the parameters of the network
is a quite remarkable feature of our results. In particular, it allows us to
establish a quantitative generalization error for finite samples in Corollary
4.4, which writes

$\bigg{|}\int_{\mathbb{R}^{2d}}\ell(x,y)\,\mathrm{d}\mu_{T}(x,y)-\frac{1}{N}\sum_{i=1}^{N}\ell(X_{T}^{i},Y_{T}^{i})\bigg{|}\leq
CW_{1}(\mu_{0}^{N},\mu_{0}).$ (1.10)

In particular, (1.10) provides a rate of convergence that depends exclusively
on the approximability of $\mu_{0}$ by empirical measures $\mu_{0}^{N}$. We
should stress at this point the relevance of (1.10) as it is one of the few
results in the literature that rigorously explains the double descent of both
empirical and generalization error in the training of deep neural networks. In
Section 5.2 we present numerical experiments fully confirming the double
descent phenomenon as theoretically predicted by (1.10), see Figure 5.

######  Remark 1.1 (Comparison with the existing literature on generalization
errors).

We point out that while the generalization errors established in [Jabir2021]
are sharper than those of the present paper (in the sense that they express a
rate of convergence in $N$ which is dimension-independent), this improved
stability comes at the price of considering _relaxed controls_ – i.e.
probability measures over $\mathbb{R}^{m}$ –, that are forced to be non-
deterministic by means of entropic regularization terms (see also
[cavagnari2020lagrangian]). On the contrary, the generalization errors that we
obtain here relate to _deterministic_ optimal controls with values in
$\mathbb{R}^{m}$. A similar bound, yielding (1.10), also appears in a
completely different context in [burger, Theorem 5.1], under the constraint
that the control is in a ball of $H^{1}((0,T),\mathbb{R}^{m})$, which is a
quite restrictive a priori assumption.

After establishing the general form of the optimality system along with some
of its interesting properties and applications, we move on to the rigorous
derivation of the mean-field PMP in Section 4.3. At this stage, let it be
noted that while part of our results may be derived by due adaptations from
other approaches developed, e.g., in [burger, eweinan19] or [PMPWassConst,
PMPWass, SetValuedPMP], we are able to obtain a few stronger properties on the
solutions of the optimal control problem than those generally presented in the
literature. Whereas in [PMPWassConst, PMPWass, SetValuedPMP] the first order
optimality conditions are established in greater generality – but also with
significant technical effort –, we propose in this paper a new and alternative
derivation (very much inspired by the previous work [ACFK16] of the third
author), which is significantly simpler and hopefully more accessible to non-
specialists. The latter can be heuristically explained as follows: under the
technical assumption that the optimal control is continuous in time – which is
motivated by the well-posedness of (1.9) in
$\operatorname{Lip}([0,T];\mathbb{R}^{m})$ discussed in Theorem 4.1 –, we
prove in Theorem 4.6 that the mean-field PMP (1.9) can be obtained by means of
a generalized Lagrange Multiplier Theorem on the convex subset of Radon
measures with unit mass. To this end, we use a new form of calculus recently
introduced in [ambrosio2018spatially], which is simpler than the calculus in
Wasserstein spaces used in [burger]. In contrast to this latter work, our
approach is applied in a slightly simpler setting, as the forward and backward
equations in (1.9) are linear and decoupled, while therein the authors
consider models for which they are non-linear and coupled. This novel
interpretation of the mean-field PMP as result of a Lagrange Multiplier
Theorem in spaces of measures is in our view quite powerful, because it can be
applied in other mean-field optimal control problems and be more easily
understood by a broader community in optimization.

The main theoretical results of the paper can then be summarized as follows.

######  Theorem 1.1 (Main contributions of the article).

Let $T>0$ be given, consider a map ${\mathcal{F}}$ satisfying Assumptions 1
and 2 of Section 3, fix an initial data distribution
$\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$, and suppose that the
regularization parameter $\lambda>0$ is sufficiently large.

Then, the mean-field optimal control problem (1.8) admits solutions, and an
admissible control $\theta^{*}\in L^{2}([0,T],\mathbb{R}^{m})$ fulfills the
mean-field PMP (1.9) _if and only if_ it is optimal. In addition, the optimal
control $\theta^{*}$ is uniquely determined, Lipschitz continuous in time, and
depends continuously on the initial data distribution $\mu_{0}$.

We then close the article by presenting numerical experiments to test the
novel mean-field Pontryagin maximum principle that we propose, in which we
show the training of simple classification models in $\mathbb{R}^{2}$. The
reason for working on simple two-dimensional examples is to provide full
understanding of the properties of the resulting algorithm and a relatively
easy reading and visualization of the results.

The paper is organized as follows. In Section 2 we introduce notations and
recall a series of preliminary results. In Section 3, we derive a general
semiconvexity estimate for the reduced cost functional, and provide sufficient
conditions ensuring the existence and stability of its minimizers. In Section
4 we address investigate the mean-field maximum principle by first studying
its well-posedness and deriving the generalization error estimate (1.10), and
then showing rigorously how it can be derived either by using a Lagrange
multiplier theorem, or via a reduction of the Hamiltonian form. We finally
present instructive numerical experiments in Section 5, where solution of the
mean-field maximum principle are computed by means of a shooting method. The
Appendix contains proofs of auxiliary results, including the proof of a
generalized Lagrange multiplier theorem, Theorem 4.5, for constrained problems
defined over convex subsets of Banach spaces.

##  2 Preliminaries and notations

In this section we list some preliminary notations and results from
[ambrosio2018spatially, Section 2.1 and Appendix A.1], which will be useful
throughout the paper.

###  2.1 Analysis in measure spaces and optimal transport

We denote by $\mathcal{M}(\mathbb{R}^{d})$ the space of signed Borel measures
in $\mathbb{R}^{d}$ with finite total variation. Note that the space
$\mathcal{M}(\mathbb{R}^{d})$ endowed with the total variation norm

$\left\lVert\mu\,\right\rVert_{TV}:=\sup\left\\{\int_{\mathbb{R}^{d}}\varphi\mathrm{d}\mu~{}\big{|}~{}\varphi\in\mathcal{C}_{0}(\mathbb{R}^{d}),~{}\left\lVert\varphi\,\right\rVert_{\infty}\leq
1\right\\}\,,$ (2.1)

is a Banach space, where $\mathcal{C}_{0}(\mathbb{R}^{d})$ represents the set
of continuous functions on $\mathbb{R}^{d}$ which vanish at infinity. By the
Riesz-Markov theorem, it is known that
$\mathcal{M}(\mathbb{R}^{d})=(\mathcal{C}_{0}(\mathbb{R}^{d}))^{\prime}$ can
be identified with the topological dual of $\mathcal{C}_{0}(\mathbb{R}^{d})$
[AmbrosioFuscoPallara, Theorem 1.54]. We further denote
$\mathcal{M}^{+}(\mathbb{R}^{d})$ the space of positive measures and by
$\mathcal{P}(\mathbb{R}^{d})\subset\mathcal{M}^{+}(\mathbb{R}^{d})$ the subset
of probability measures. Furthermore,
$\mathcal{P}_{c}(\mathbb{R}^{d})\subset\mathcal{P}(\mathbb{R}^{d})$ represents
the set of probability measures with compact support, while
$\mathcal{P}^{N}_{c}(\mathbb{R}^{d})\subset\mathcal{P}_{c}(\mathbb{R}^{d})$
denotes the subset of empirical or atomic probability measures. We will also
use the following representation formulas for the subset of measures with zero
mass

$\mathcal{M}_{0}(\mathbb{R}^{d}):=\left\\{\mu\in(\mathcal{C}_{0}(\mathbb{R}^{d}))^{\prime}~{}\big{|}~{}\mu(\mathbb{R}^{d})=\int_{\mathbb{R}^{d}}1\mathrm{d}\mu=0\right\\}=:(\mathcal{C}_{0}(\mathbb{R}^{d}))^{\prime}_{0}\,,$
(2.2)

and the subset of measures with unit mass

$\mathcal{M}_{1}(\mathbb{R}^{d}):=\left\\{\mu\in(\mathcal{C}_{0}(\mathbb{R}^{d}))^{\prime}~{}\big{|}~{}\mu(\mathbb{R}^{d})=\int_{\mathbb{R}^{d}}1\mathrm{d}\mu=1\right\\}=:(\mathcal{C}_{0}(\mathbb{R}^{d}))^{\prime}_{1}\,.$
(2.3)

Moreover, we shall denote by
$\mathcal{M}_{0,c}(\mathbb{R}^{d}),\mathcal{M}_{1,c}(\mathbb{R}^{d})$ the
corresponding subsets of measures whose supports are compact. One can also
note that given $\mu\in\mathcal{M}(\mathbb{R}^{d})$, the Jordan decomposition
theorem tells us that $\mu=\mu^{+}-\mu^{-}$ and
$\|\mu\|_{TV}=\mu^{+}(\mathbb{R}^{d})+\mu^{-}(\mathbb{R}^{d})$, where
$\mu^{+},\mu^{-}\in\mathcal{M}^{+}(\mathbb{R}^{d})$.

For the convenience of the reader, we briefly recall the definition of the
Wasserstein metrics of optimal transport in the following definition, and
refer to [AGS, Chapter 7] for more details.

######  Definition 2.1.

Let $1\leq p<\infty$ and $\mathcal{P}_{p}(\mathbb{R}^{d})$ be the space of
Borel probability measures on $\mathbb{R}^{d}$ with finite $p$-moment. In the
sequel, we endow the latter with the $p$-_Wasserstein metric_

$W_{p}^{p}(\mu,\nu):=\inf\left\\{\int_{\mathbb{R}^{2d}}|z-\widehat{z}|^{p}\
d\pi(z,\widehat{z})\ \big{|}\ \pi\in\Pi(\mu,\nu)\right\\}$ (2.4)

where $\Pi(\mu,\nu)$ denotes the set of _transport plan_ between $\mu$ and
$\nu$, that is the collection of all Borel probability measures on
$\mathbb{R}^{d}\times\mathbb{R}^{d}$ with marginals $\mu$ and $\nu$ in the
first and second component respectively. The Wasserstein distance can also be
expressed as

$W_{p}^{p}(\mu,\nu)=\inf\left\\{\mathbb{E}\big{[}|Z-\widehat{Z}|^{p}\big{]}\right\\}$
(2.5)

where the infimum is taken over all possible joint distributions of random
variables $(Z,\widehat{Z})$ which laws are given by $\mu$ and $\nu$
respectively.

It is a well-known result in optimal transport theory that when $p=1$ and
$\mu,\nu\in\mathcal{P}_{c}(\mathbb{R}^{d})$, the following alternative
representation holds for the Wasserstein distance

$W_{1}(\mu,\nu)=\sup\left\\{\int_{\mathbb{R}^{d}}\varphi(x)\mathrm{d}(\mu-\nu)(x)~{}\big{|}~{}\varphi\in\mbox{Lip}(\mathbb{R}^{d}),~{}\mbox{Lip}(\varphi)\leq
1\right\\}\,,$ (2.6)

by Kantorovich’s duality [AGS, Chapter 6]. Here, $\mbox{Lip}(\mathbb{R}^{d})$
stands for the space of real-valued Lipschitz continuous functions on
$\mathbb{R}^{d}$, and $\mbox{Lip}(\varphi)$ is the Lipschitz constant of a
mapping $\varphi$. In the sequel, we shall also use the signed generalized
Wasserstein distance $\mathbb{W}_{1}^{1,1}$ introduced in
[piccoli2019wasserstein], which coincides with the bounded Lipschitz distance.
Given $\mu,\nu\in\mathcal{M}(\mathbb{R}^{d})$, we set

$\mathbb{W}_{1}^{1,1}(\mu,\nu)=\sup\left\\{\int_{\mathbb{R}^{d}}\varphi(x)\mathrm{d}(\mu-\nu)(x)~{}\big{|}~{}\varphi\in\mbox{Lip}_{b}(\mathbb{R}^{d}),~{}\left\lVert\varphi\,\right\rVert_{\mbox{Lip}_{b}}\leq
1\right\\}\,,$ (2.7)

where

$\left\lVert\varphi\,\right\rVert_{\mbox{Lip}_{b}}:=\sup\limits_{x\in\mathbb{R}^{d}}|\varphi(x)|+\mbox{Lip}(\varphi)\,.$
(2.8)

In this context, we also define the bounded Lipschitz norm of a signed measure
as

$\|\mu\|_{BL}:=\mathbb{W}_{1}^{1,1}(\mu,0)\,.$ (2.9)

###  2.2 Continuity equations in the space of measures

In what follows, we recollect some basic facts about continuity equations in
the space of measures, following [AGS, Section 8.1].

######  Definition 2.2.

For any given $T>0$ and $\theta\in L^{2}([0,T];\mathbb{R}^{m})$, we say that
$\mu\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))$ is a weak solution
of (1.7) on the time interval $[0,T]$ if

$\int_{0}^{T}\int_{\mathbb{R}^{2d}}\Big{(}\partial_{t}\psi(t,x,y)+\nabla_{x}\psi(t,x,y)\cdot\mathcal{F}(t,x,\theta_{t})\Big{)}\mathrm{d}\mu_{t}(x,y)\,{\rm
d}t=0,$ (2.10)

for every $\psi\in\mathcal{C}_{c}^{1}((0,T)\times\mathbb{R}^{2d})$.

######  Remark 2.1.

First, note that (2.10) is equivalent to

$\int_{\mathbb{R}^{2d}}\psi(x,y)\mathrm{d}\mu_{t_{2}}(x,y)-\int_{\mathbb{R}^{2d}}\psi(x,y)\mathrm{d}\mu_{t_{1}}(x,y)=\int_{t_{1}}^{t_{2}}\int_{\mathbb{R}^{2d}}\nabla_{x}\psi(x,y)\cdot\mathcal{F}(s,x,\theta_{s})\mathrm{d}\mu_{s}(x,y)\,{\rm
d}s$ (2.11)

for all $\psi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$ and every
$t_{1},t_{2}\in[0,T]$. This follows from the fact that the linear span of
functions of the form $\psi(t,x,y):=\eta(t)\xi(x,y)$ with
$\eta\in\mathcal{C}_{c}^{1}((0,T))$ and
$\xi\in\mathcal{C}_{c}^{1}(\mathbb{R}^{2d})$ is dense in
$\mathcal{C}_{c}^{1}((0,T)\times\mathbb{R}^{2d})$ (see e.g. [AGS, Remark
8.1.1]). Also, observe that since $\mu$ is a curve of compactly supported
probability measures, we can use the simpler testing space
$\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$ instead of
$\mathcal{C}_{c}^{1}(\mathbb{R}^{2d})$ or
$\mathcal{C}_{0}^{1}(\mathbb{R}^{2d})$ in (2.11).

Classical well-posedness result for (1.7) for arbitrary initial measures is
usually established under the following type of standard Cauchy-Lipschitz
assumptions (or minimal variations thereof).

######  Assumption 1.

For any given $T>0$, the vector field ${\mathcal{F}}$ satisfies the following.

  1. $(i)$

For any fixed $\theta\in\mathbb{R}^{m}$, the map
$(t,x)\mapsto{\mathcal{F}}(t,x,\theta)\in\mathbb{R}^{d}$ is continuous.

  2. $(ii)$

There exists a constant $C_{{\mathcal{F}}}>0$ that may depend on $d,m$ such
that for every $\theta\in\mathbb{R}^{m}$, it holds

$|\mathcal{F}(t,x,\theta)|\leq C_{{\mathcal{F}}}(1+|x|),\quad\mbox{ for a.e.
}t\in[0,T]\mbox{ and every }x\in\mathbb{R}^{d}\,.$

  3. $(iii)$

There exists a constant $L_{{\mathcal{F}}}>0$ independent of $d,m$ such that
for every $\theta\in\mathbb{R}^{m}$, it holds

$|\mathcal{F}(t,x_{1},\theta)-\mathcal{F}(t,x_{2},\theta)|\leq
L_{{\mathcal{F}}}(1+|\theta|)|x_{1}-x_{2}|,\quad\mbox{ for a.e.
}t\in[0,T]\mbox{ and every }x_{1},x_{2}\in\mathbb{R}^{d}\,,$

and we denote
$L_{\mathcal{F},T,\|\theta\|_{1}}:=L_{{\mathcal{F}}}\int_{0}^{T}(1+|\theta_{t}|)\textnormal{d}t$

  4. $(iv)$

For all $(t,x)\in[0,T]\times\mathbb{R}^{d}$, the map
$\theta\mapsto{\mathcal{F}}(t,x,\theta)$ is twice differentiable. Moreover for
each $R>0$, there exists a constant $C(d,m,R)>0$ such that

$\|\nabla_{\theta}{\mathcal{F}}\|_{\mathcal{C}([0,T]\times
B(R)\times\mathbb{R}^{m};\mathbb{R}^{d\times
m})}\,+\,\|\nabla_{\theta}^{2}\mathcal{F}\|_{\mathcal{C}([0,T]\times
B(R)\times\mathbb{R}^{m};\mathbb{R}^{d\times m\times m})}\leq C(d,m,R)\,.$

Under the set of assumptions listed above, we can prove the well-posedness of
(1.7) as stated in the following theorem. The proof of the latter is standard
and deferred to Appendix A.

######  Theorem 2.3 (Classical well-posedness for continuity equation).

Consider a measure $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ with
$\textnormal{supp}(\mu_{0})\subset B(R)$ for some $R>0$, and suppose that
${\mathcal{F}}$ satisfies Assumption 1.

Then for any given $T>0$ and $\theta\in L^{2}([0,T];\mathbb{R}^{m})$, there
exists a unique solution
$\mu\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))$ to (1.7) in the
sense of Definition 2.2. Moreover, there exists a radius $R_{T}>0$ depending
only on $R$ and $C_{{\mathcal{F}}}$ such that

$\textnormal{supp}(\mu_{t})\subset B(R_{T}),$ (2.12)

for all times $t\in[0,T]$, and additionally it holds for any $s,t\in[0,T]$
that

$W_{1}(\mu_{t},\mu_{s})\leq C(R,T,C_{{\mathcal{F}}})|t-s|\,.$ (2.13)

Denoting by $\mu^{i}$ for $i=1,2$ two solutions of (1.7) with initial data
$\mu_{0}^{i}$ satisfying the above assumptions, the following stability
estimate

$W_{1}(\mu_{t}^{1},\mu_{t}^{2})\leq
e^{L_{{\mathcal{F}},T,\|\theta\|_{1}}}W_{1}(\mu_{0}^{1},\mu_{0}^{2}),$ (2.14)

holds for all times $t\in[0,T]$, where $C_{{\mathcal{F}}}$ and
$L_{{\mathcal{F}},T,\|\theta\|_{1}}$ are defined as in Assumption 1.

###  2.3 Differential calculus over convex subsets of Banach spaces

We end this series of preliminaries by introducing a notion of multi-valued
Fréchet differential for functions defined on convex sets. To this end, given
a convex subset $E$ of a normed vector space $X$, we define

$X_{E}:=\mathbb{R}(E-E)=\Big{\\{}x\in
X~{}\big{|}~{}x=\alpha(e_{1}-e_{2})~{}\text{with $\alpha\in\mathbb{R}$ and
$e_{1},e_{2}\in E$}\Big{\\}},$

and given $e\in E$, we denote by $X_{e}:=\mathbb{R}_{+}(E-e)$ the convex cone
of directions at $e$.

######  Definition 2.4.

Let $X$, $Y$ be normed vector spaces, $E\subset X$ be a convex set and $f:E\to
Y$. Then, $f$ is _$F$ -differentiable_ at $e\in E$ if there exists
$L\in\mathcal{L}(X_{E},Y)$ such that

$\lim\limits_{\begin{subarray}{c}e^{\prime}\to e\\\ e^{\prime}\in
E\end{subarray}}\frac{\left\lVert
f(e^{\prime})-f(e)-L(e^{\prime}-e)\,\right\rVert_{Y}}{\left\lVert
e^{\prime}-e\,\right\rVert_{X}}=0\,,$ (2.15)

where $\mathcal{L}(X_{E},Y)$ denotes the space of bounded linear operators
from $X_{E}$ into $Y$.

Following the previous definition, we define the _$F$ -differential_ of $f$ at
$e\in E$ by

$Df(e):=\Big{\\{}L\in\mathcal{L}(X_{E},Y)~{}\big{|}~{}L\mbox{ satisfies
$\eqref{A1}$}\Big{\\}}\,.$ (2.16)

It can be checked that if $X_{e}$ is not dense in $X_{E}$, then the mapping
$D$ is set-valued (similarly to classical convex subdifferentials). However if
$v\in\overline{X}_{e}$, then the evaluation $Df(e)(v)$ is uniquely determined,
namely it does not depend on the choice of $L$ in $Df(e)$, and in this case we
will slightly abuse the notation and write $Df(e)(v)$ to mean $L(v)$ for any
$L\in Df(e)$. By a density argument, each $L\in Df(e)$ can be uniquely
extended to an operator $\overline{L}$ in $\mathcal{L}(\overline{X}_{E},Y)$.
We will then say that $f\in\mathcal{C}^{1}(E;Y)$ if $f$ is $F$-differentiable
at each $e\in E$, and there exists a selection $e\in E\mapsto L_{e}\in Df(e)$
such that

$e\mapsto L_{e}\quad\mbox{ is continuous from }E\mbox{ into
}\mathcal{L}(X_{E},Y)\,,$ (2.17)

where $\mathcal{L}(X_{E},Y)$ is endowed with the distance induced by the
standard operator norm.

######  Definition 2.5.

Let $X$, $Y$ be normed vector spaces, $E\subset X$ be a convex set, and
$f:E\to Y$. Then, $f$ is _$G$ -differentiable_ at $e\in E$ if the directional
right derivatives

$df(e,v):=\lim\limits_{h\to 0^{+}}\frac{f(e+hv)-f(e)}{h},$ (2.18)

exist in $Y$ for all $v\in X_{e}$.

######  Remark 2.2.

Obviously if $f$ is $F$-differentiable at some $e\in E$, then it is
$G$-differentiable as well with $df(e,v)=Df(e)(v)$ for all $v\in X_{e}$.

We shall also use the following lemma as a criterion for $\mathcal{C}^{1}$
regularity, see [ambrosio2018spatially, Lemma A.4].

######  Lemma 2.1.

Let $f:~{}E\to F$ be a continuous map and suppose that there exists a
continuous application

$e\in E\mapsto L_{e}\in\mathcal{L}(X_{E},Y),$ (2.19)

such that $df(e,v)=L_{e}v$ for all $e\in E$ and any $v\in X_{e}$. Then
$f\in\mathcal{C}^{1}(E;Y)$ and $e\mapsto L_{e}\in Df(e)$ is an admissible
selection.

##  3 Existence of minimizers and stability of solutions

In this section, we investigate sufficient conditions ensuring the existence
of optimal solutions to the mean-field optimal control problem (1.8), as well
as stability properties for the minimizers and costs stemming from large
finite-sample training. Throughout the remainder of this article, we will use
Assumption 1 and the following additional hypotheses to establish most of our
results.

######  Assumption 2.

For any given $T>0$ and $R>0$, the vector field ${\mathcal{F}}$ satisfies the
following.

  1. $(i)$

The map $x\in\mathbb{R}^{d}\mapsto{\mathcal{F}}(t,x,\theta)$ is of class
$\mathcal{C}^{2}$ all times $t\in[0,T]$ and any $\theta\in\mathbb{R}^{m}$, and
for each $x\in B(R)$, it holds

$|\nabla_{x}\cdot\nabla_{\theta}{\mathcal{F}}(t,x,\theta)|+|\nabla_{x}{\mathcal{F}}(t,x,\theta)|+|\nabla_{x}^{2}{\mathcal{F}}(t,x,\theta)|\leq
C(d,m,R,|\theta|)\,;$ (3.1)

  2. $(ii)$

For any $\theta^{1},\theta^{2}\in\mathbb{R}^{m}$, every $s,t\in[0,T]$ and all
$x\in B(R)$, it holds

$|\mathcal{F}(t,x,\theta^{1})-\mathcal{F}(s,x,\theta^{2})|\leq
C(d,m,R)\big{(}|t-s|+|\theta^{1}-\theta^{2}|\big{)}\,;$ (3.2)

  3. $(iii)$

For all fixed $\theta$ and $t\in[0,T]$, it holds

$|\nabla_{\theta}\mathcal{F}(t,x,\theta)-\nabla_{\theta}\mathcal{F}(t,y,\theta)|\leq
C(d,m,R,|\theta|)|x-y|,$ (3.3)

for every $x,y\in B(R)$.

Before moving on to the discussion pertaining to the existence and stability
properties for solutions of (1.8), we highlight the adequacy of our working
hypotheses in connection with classical machine learning models.

######  Remark 3.1 (Adequacy of smooth sigmoidal activations).

Assumptions 1 and 2 require smooth activation functions that exhibit also some
boundedness properties with respect to the parameter $\theta$, e.g. as in
Assumption 1-$(ii)$. These latter are needed both to express the PMP and to
establish its well-posedness, as will become apparent in Section 4. Hence,
some popular network models which use for instance ReLu activations are not
covered by our results. However, we check here that the sets of hypotheses
listed in Assumptions 1 and 2 include the popular subclass of feed-forwarding
dynamics (1.4) involving sigmoidal-type activation functions, such as

${\mathcal{F}}(t,x,\theta)={\mathcal{F}}(x,\theta):=\tanh(\theta
x)\in\mathbb{R}^{d}\,,$

where $\theta\in\mathbb{R}^{m}=\mathbb{R}^{d\times d}$ and
$x\in\mathbb{R}^{d}$. In that case, Assumption 1-$(i)$ obviously holds, and
since

${\mathcal{F}}_{k}(x,\theta)=\tanh\Big{(}{\sum}\limits_{l=1}^{d}\theta_{k,l}x_{l}\Big{)}$

for each $k\in\\{1,\dots,d\\}$ and $|\tanh(r)|\leq 1$ for all
$r\in\mathbb{R}$, we have that $|{\mathcal{F}}(x,\theta)|\leq\sqrt{d}$ for all
$(x,\theta)\in\mathbb{R}^{d}\times\mathbb{R}^{d\times d}$, and Assumption
1-$(ii)$ also holds. This uniform boundedness property of the driving field
implies in particular that the radius $R_{T}>0$ given by Theorem 2.3 and
controlling the support sizes of the solutions of (1.7) will scale
polynomially and not exponentially on $d\geq 1$, along with all the relevant
constants depending polynomially thereon. Moreover, observe that

$\partial_{x_{i}}{\mathcal{F}}_{k}(x,\theta)=\tanh^{\prime}\Big{(}{\sum}\limits_{l=1}^{d}\theta_{k,l}\,x_{l}\Big{)}\theta_{k,i}$

for each $i,k\in\\{1,\dots,d\\}$, which implies in particular that
$|\nabla_{x}\mathcal{F}(x,\theta)|\leq|\theta|$ for all
$(x,\theta)\in\mathbb{R}^{d}\times\mathbb{R}^{d\times d}$ by using the fact
that $|\tanh^{\prime}(r)|=|1-\tanh(r)^{2}|\leq 1$ for each $r\in\mathbb{R}$.
By the mean-value theorem, this latter fact directly implies that

$|{\mathcal{F}}(t,x_{1},\theta)-{\mathcal{F}}(t,x_{2},\theta)|\leq|\theta||x_{1}-x_{2}|$

for all $\theta\in\mathbb{R}^{d\times d}$ and $x_{1},x_{2}\in\mathbb{R}^{d}$,
which verifies Assumption 1-$(iii)$. Concerning Assumption 1-$(iv)$, one has
that

$\partial_{\theta_{ij}}{\mathcal{F}}_{k}(x,\theta)=\delta_{k,i}\tanh^{\prime}\Big{(}{\sum}\limits_{l=1}^{d}\theta_{k,l}x_{l}\Big{)}x_{j}$

for each $i,j,k\in\\{,1\dots,d\\}$ – where $\delta_{k,i}$ refers here to the
Kronecker symbol –, which implies that
$|\nabla_{\theta}{\mathcal{F}}(x,\theta)|\leq\sqrt{d}|x|$ for all
$\theta\in\mathbb{R}^{m}$ and $x\in\mathbb{R}^{d}$. Furthermore, one can
easily see that

$\partial^{2}_{\theta_{i,j},\theta_{m,n}}{\mathcal{F}}_{k}(x,\theta)=\delta_{k,m}\delta_{k,i}\tanh^{\prime\prime}\Big{(}{\sum}\limits_{l=1}^{d}\theta_{k,l}x_{l}\Big{)}x_{j}x_{n}$

for each $i,j,k,m,n\in\\{1,\dots,d\\}$, which then yields
$|\nabla_{\theta}^{2}{\mathcal{F}}(x,\theta)|\leq 4\sqrt{d}\,|x|^{2}$ for all
$(x,\theta)\in\mathbb{R}^{d}\times\mathbb{R}^{d\times d}$ since
$|\tanh^{\prime\prime}(r)|=|2\tanh(r)\left(\tanh(r)-1\right)|\leq 4$ for every
$r\in\mathbb{R}^{d}$. Thence, it holds

$\max\limits_{(x,\theta)\in
B(R)\times\mathbb{R}^{m}}|\nabla_{\theta}{\mathcal{F}}(x,\theta)|\leq\sqrt{d}R\qquad\text{and}\qquad\max\limits_{(x,\theta)\in
B(R)\times\mathbb{R}^{m}}|\nabla^{2}_{\theta}{\mathcal{F}}(x,\theta)|\leq
4\sqrt{d}R^{2},$ (3.4)

which completes the verification of Assumption 1.

We now shift our attention to the verification of Assumption 2. First of all,
one has that

$\partial^{2}_{x_{i},x_{j}}{\mathcal{F}}_{k}(x,\theta)=\tanh^{\prime\prime}\Big{(}{\sum}_{l=1}^{d}\theta_{k,l}x_{l}\Big{)}\theta_{k,i}\theta_{k,j}$

for each $i,j,k\in\\{1,\dots,d\\}$, which yields the estimate
$|\nabla_{x}^{2}\mathcal{F}(x,\theta)|\leq 4|\theta|^{2}$ for all
$(x,\theta)\in\mathbb{R}^{d}\times\mathbb{R}^{d\times d}$. Moreover, one can
check that

$\partial_{x_{n}}\partial_{\theta_{i,j}}{\mathcal{F}}_{k}(x,\theta)=\delta_{k,i}\tanh^{\prime\prime}\Big{(}{\sum}\limits_{l=1}^{d}\theta_{k,l}x_{l}\Big{)}x_{j}\theta_{k,n}+\delta_{k,i}\delta_{j,n}\tanh^{\prime}\Big{(}{\sum}\limits_{l=1}^{d}\theta_{k,l}x_{l}\Big{)}$

for each $i,j,k,n\in\\{1,\dots,d\\}$. Thus, we obtain the estimates

$|\nabla_{x}\cdot\nabla_{\theta}{\mathcal{F}}(x,\theta)|\leq\sqrt{d}|\nabla_{x}\nabla_{\theta}{\mathcal{F}}(x,\theta)|\leq\sqrt{2}\sqrt{d}\big{(}4|x||\theta|+d\big{)}$

for all $(x,\theta)\in\mathbb{R}^{d}\times\mathbb{R}^{d\times d}$, which leads
to Assumption 2-$(i)$ being fulfilled. Moreover, we can also deduce from the
previous estimate that Assumption 2-$(iii)$ holds, since

$|\nabla_{\theta}\mathcal{F}(t,x,\theta)-\nabla_{\theta}\mathcal{F}(t,y,\theta)|\leq\sqrt{2}(4R|\theta|+d)|x-y|$

for all $\theta\in\mathbb{R}^{d\times d}$ and $x,y\in B(R)$. Lastly, it
follows from (3.4) that

$|\mathcal{F}(t,x,\theta^{1})-\mathcal{F}(s,x,\theta^{2})|\leq\sqrt{d}R|\theta^{1}-\theta^{2}|$

for all $\theta^{1},\theta^{2}\in\mathbb{R}^{d\times d}$ and
$x\in\mathbb{R}^{d}$, which equivalently means that Assumption 2-$(ii)$ is
satisfied and completes the verification of Assumption 2.

###  3.1 Convexity of the reduced cost functional and existence of minimizers

As already recalled in the introduction, $L^{2}$-regularization of network
parameters is a standard practice in machine learning which helps stabilizing
the training procedure, while promoting the generalization capacities of
networks [Goodfellow-et-al-2016, Kukacka-et-al-2017]. In this section, we show
that for regularization parameters $\lambda>0$ that are sufficiently large,
the reduced cost of the problem is actually strictly convex, which in
particular implies the existence and uniqueness of an optimal control
$\theta^{*}\in L^{2}([0,T];\mathbb{R}^{m})$ for the mean-field optimal control
problem (1.8). Given the smoothness of the forward map $\mathcal{F}$ the
convexity of $J$ is perhaps not surprising, but it has never been noticed
before in the literature in connection to mean-field optimal control problems,
and appears to have far-reaching practical implications that we shall explore
in the remainder of the paper.

For any fixed $\theta\in L^{2}([0,T],\mathbb{R}^{m})$, we denote by
$(\Phi^{\theta}_{(\tau,t)}(\cdot))_{\tau,t\in[0,T]}$ the characteristic flow
generated by the controlled velocity field
$(t,x)\in[0,T]\times\mathbb{R}^{d}\mapsto{\mathcal{F}}(t,x,\theta_{t})\in\mathbb{R}^{d}$,
defined by

$\left\\{\begin{aligned}
\partial_{t}\Phi^{\theta}_{(\tau,t)}(x)&={\mathcal{F}}\big{(}t,\Phi^{\theta}_{(\tau,t)}(x),\theta_{t}\big{)},\\\
\Phi^{\theta}_{(\tau,\tau)}(x)&=x,\end{aligned}\right.$ (3.5)

for every $x\in\mathbb{R}^{d}$. It is a well-known result in the theory of
non-linear dynamical systems (see e.g. [BressanPiccoli, Theorem 2.3.2]) that
under Assumption 1, the flow maps
$\Phi^{\theta}_{(\tau,t)}:\mathbb{R}^{d}\rightarrow\mathbb{R}^{d}$ are
continuously differentiable for every $\tau,t\in[0,T]$, and the application
$t\in[0,T]\mapsto\nabla_{x}\Phi^{\theta}_{(\tau,t)}(x)\in\mathbb{R}^{d\times
d}$ is the unique solution of the forward linearized Cauchy problem

$\left\\{\begin{aligned}
\partial_{t}w(t,x)&=\nabla_{x}{\mathcal{F}}\big{(}t,\Phi^{\theta}_{(\tau,t)}(x),\theta_{t}\big{)}w(t,x)\\\
w(\tau,x)&=\mbox{Id}.\end{aligned}\right.$ (3.6)

This allows us to establish the following semiconvexity result for the reduced
cost of (1.8).

######  Proposition 3.1 (Semiconvexity of the reduced cost functional).

Let $T,R>0$ and $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{d})$ be such that
$\textnormal{supp}(\mu_{0})\subset B(R)$, and suppose that Assumptions 1 and 2
hold. Then, for every ball $\Gamma\subset L^{2}([0,T];\mathbb{R}^{m})$, there
exists a constant $\mathcal{L}(T,R,\Gamma)>0$ such that the reduced cost
functional

$J:\theta\in L^{2}([0,T];\mathbb{R}^{m})\mapsto\left\\{\begin{aligned}
&\int_{\mathbb{R}^{2d}}\ell(x,y)\textnormal{d}\mu_{T}^{\theta}(x,y)+\lambda\int_{0}^{T}|\theta_{t}|^{2}\textnormal{d}t,\\\
&\;\,\textnormal{s.t.}\,\,\left\\{\begin{aligned}
&\partial_{t}\mu_{t}^{\theta}+\nabla_{x}\big{(}\mathcal{F}(t,x,\theta_{t})\mu_{t}^{\theta}\big{)}=0,\\\
&\mu_{0}^{\theta}=\mu_{0},\end{aligned}\right.\end{aligned}\right.$ (3.7)

satisfies the semiconvexity estimate

$J\big{(}(1-\zeta)\theta^{1}+\zeta\theta^{2}\big{)}\leq(1-\zeta)J(\theta^{1})+\zeta
J(\theta^{2})-(2\lambda-\mathcal{L}(T,R,\Gamma))\tfrac{\zeta(1-\zeta)}{2}\|\theta^{1}-\theta^{2}\|_{2}^{2}$
(3.8)

for any $\theta^{1},\theta^{2}\in\Gamma$ and all $\zeta\in[0,1]$. In
particular if $\lambda>\tfrac{1}{2}\mathcal{L}(T,R,\Gamma)$, the reduced cost
functional is then strictly convex over $\Gamma$.

The proof of this convexity estimate is almost entirely contained in the
following regularity result, which itself relies on a series of technical
properties for characteristic flows which are exposed in Appendix B.

######  Lemma 3.1 (Regularity of the reduced final cost).

Let $T,R>0$ and $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ be such that
$\textnormal{supp}(\mu_{0})\subset B(R)$, and suppose that Assumptions 1 and 2
hold. Then, the reduced final cost

$J_{\ell}:\theta\in L^{2}([0,T];\mathbb{R}^{m})\mapsto\left\\{\begin{aligned}
&\int_{\mathbb{R}^{2d}}\ell(x,y)\textnormal{d}\mu_{T}^{\theta}(x,y),\\\
&\;\,\textnormal{s.t.}\,\,\left\\{\begin{aligned}
&\partial_{t}\mu_{t}^{\theta}+\nabla_{x}\big{(}\mathcal{F}(t,x,\theta_{t})\mu_{t}^{\theta}\big{)}=0,\\\
&\mu_{0}^{\theta}=\mu_{0},\end{aligned}\right.\end{aligned}\right.$ (3.9)

is Fréchet-differentiable. Moreover, denoting its gradient by
$\nabla_{\theta}J_{\ell}(\theta)\in L^{2}([0,T];\mathbb{R}^{m})$ and choosing
$\theta^{1},\theta^{2}\in L^{2}([0,T];\mathbb{R}^{m})$, there exists a
constant $\mathcal{L}(T,R,\|\theta^{1}\|_{1},\|\theta^{2}\|_{1})>0$ such that

$\big{\|}\nabla_{\theta}J_{\ell}(\theta^{1})-\nabla_{\theta}J_{\ell}(\theta^{2})\big{\|}_{2}\leq\mathcal{L}(T,R,\|\theta^{1}\|_{1},\|\theta^{2}\|_{1})\,\big{\|}\theta^{1}-\theta^{2}\big{\|}_{2}.$

###### Proof.

We start by fixing a control signal $\theta\in L^{2}([0,T];\mathbb{R}^{m})$.
Following the discussion in Appendix A below, the unique solution
$\mu^{\theta}\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))$ of the
controlled continuity equation can be expressed as
$\mu_{t}^{\theta}=\boldsymbol{\Phi}^{\theta}_{(0,t)}\sharp\mu_{0}$, where

$\boldsymbol{\Phi}_{(0,t)}^{\theta}(x,y)=\big{(}\Phi^{\theta}_{(0,t)}(x),y\big{)}$

for all $(x,y)\in\mathbb{R}^{2d}$, with
$(\Phi^{\theta}_{(0,t)}(\cdot))_{t\in[0,T]}$ being the characteristic flow
defined in (3.5). In particular, this allows us to rewrite the reduced final
cost as

$J_{\ell}(\theta)=\int_{\mathbb{R}^{2d}}\ell\Big{(}\Phi^{\theta}_{(0,T)}(x),y\Big{)}\textnormal{d}\mu_{0}(x,y).$

Given another control signal $\vartheta\in L^{2}([0,T];\mathbb{R}^{m})$ and
some $\varepsilon>0$, we know by Proposition B.2 that the following Taylor
expansion

$\Phi^{\theta+\epsilon\vartheta}_{(0,T)}(x)=\Phi^{\theta}_{(0,T)}(x)+\varepsilon\int_{0}^{T}\mathcal{R}^{\theta}_{(t,T)}(x)\nabla_{\theta}\mathcal{F}\big{(}t,\Phi^{\theta}_{(0,t)}(x),\theta_{t}\big{)}\vartheta_{t}\textnormal{d}t+o_{\theta}(\varepsilon)$
(3.10)

holds for all $(t,x)\in[0,T]\times B(R)$, where
$(\mathcal{R}^{\theta}_{(\tau,t)}(\cdot))_{t\in[0,T]}\subset\mathcal{C}^{1}(\mathbb{R}^{d};\mathbb{R}^{d\times
d})$ are the resolvent maps of the linearized Cauchy problem defined as in
(B.2). Since the small-o in (3.10) is uniform in $x\in B(R)$, it holds by
Lebesgue’s dominated convergence and Fubini’s theorems that

$\displaystyle\int_{\mathbb{R}^{2d}}\ell\Big{(}\Phi^{\theta+\varepsilon\vartheta}_{(0,T)}(x),y\Big{)}\textnormal{d}\mu_{0}(x,y)$
(3.11)
$\displaystyle=\int_{\mathbb{R}^{2d}}\ell\Big{(}\Phi^{\theta}_{(0,T)}(x),y\Big{)}\textnormal{d}\mu_{0}(x,y)$
$\displaystyle\hskip
12.80365pt+\varepsilon\int_{0}^{T}\bigg{\langle}\int_{\mathbb{R}^{2d}}\Big{(}\mathcal{R}^{\theta}_{(t,T)}(x)\nabla_{\theta}\mathcal{F}\big{(}t,\Phi^{\theta}_{(0,t)}(x),\theta_{t}\big{)}\Big{)}^{\top}\nabla_{x}\ell\big{(}\Phi^{\theta}_{(0,T)}(x),y\big{)}\textnormal{d}\mu_{0}(x,y),\vartheta_{t}\bigg{\rangle}\textnormal{d}t+o_{\theta}(\varepsilon),$

for every $\varepsilon>0$ small enough. From the regularity estimates of
Assumption 1, Proposition B.1 and Proposition B.2, we may infer that the
Gateaux derivative expressed in (3.11) is continuous with respect to
$\theta\in L^{2}([0,T];\mathbb{R}^{m})$, so that the reduced final cost is
Fréchet-differentiable, with

$\nabla_{\theta}J_{\ell}(\theta):t\in[0,T]\mapsto\int_{\mathbb{R}^{2d}}\Big{(}\mathcal{R}^{\theta}_{(t,T)}(x)\nabla_{\theta}\mathcal{F}\big{(}t,\Phi^{\theta}_{(0,t)}(x),\theta_{t}\big{)}\Big{)}^{\top}\nabla_{x}\ell\big{(}\Phi^{\theta}_{(0,T)}(x),y\big{)}\textnormal{d}\mu_{0}(x,y).$
(3.12)

At this stage, by resorting again to Assumptions 1 and 2, Proposition B.1 and
Proposition B.2, one can check that the previous expression is a (formal)
product of quantities which are bounded and Lipschitz with respect to $\theta$
on bounded subsets of $L^{1}([0,T];\mathbb{R}^{m})$. Whence, for every pair
$\theta^{1},\theta^{2}\in L^{2}([0,T];\mathbb{R}^{m})$, there exists a
constant $\mathcal{L}(T,R,\|\theta^{1}\|_{1},\|\theta^{2}\|_{1})$ such that

$\big{\|}\nabla_{\theta}J_{\ell}(\theta^{1})-\nabla_{\theta}J_{\ell}(\theta^{2})\big{\|}_{2}\leq\mathcal{L}(T,R,\|\theta^{1}\|_{1},\|\theta^{2}\|_{1})\,\big{\|}\theta^{1}-\theta^{2}\big{\|}_{2},$

which ends the proof of our claim. ∎

We are now ready to move on to the proof of Proposition 3.1.

###### Proof of Proposition 3.1.

First, observe that the reduced cost of the problem can be written as

$J(\theta)=J_{\ell}(\theta)+\lambda\|\theta\|_{2}^{2}$

for all $\theta\in L^{2}([0,T];\mathbb{R}^{m})$, where
$J_{\ell}(\mu_{0},\theta)$ stands for the reduced final cost defined in (3.9).
Whence, it can be easily checked as a consequence of Lemma 3.1 that the
reduced cost is Fréchet-differentiable, with

$\nabla_{\theta}J(\theta)=\nabla_{\theta}J_{\ell}(\theta)+2\lambda\theta.$
(3.13)

Let $\Gamma\subset L^{2}([0,T];\mathbb{R}^{m})$ be a closed ball and
$\theta^{1},\theta^{2}\in\Gamma$. By performing routine computations based on
the integral version of Taylor’s theorem (see e.g. [LipReg, Lemma 6] for a
detailed proof in the finite-dimensional case), one can show that

$\displaystyle J_{\ell}\big{(}(1-\zeta)\theta^{1}+\zeta\theta^{2})\big{)}$
$\displaystyle\leq(1-\zeta)J_{\ell}(\theta^{1})+\zeta
J_{\ell}(\theta^{2})+\textnormal{Lip}(\nabla_{\theta}J_{\ell}\,;\Gamma)\tfrac{\zeta(1-\zeta)}{2}\big{\|}\theta^{1}-\theta^{2}\big{\|}_{2}^{2}$
$\displaystyle\leq(1-\zeta)J_{\ell}(\theta^{1})+\zeta
J_{\ell}(\theta^{2})+\mathcal{L}(T,R,\Gamma)\tfrac{\zeta(1-\zeta)}{2}\big{\|}\theta^{1}-\theta^{2}\big{\|}_{2}^{2},$

for all $\zeta\in[0,1]$, where the constant
$\mathcal{L}(T,R,\Gamma):=\mathcal{L}(T,R,\|\theta^{1}\|_{1},\|\theta^{2}\|_{1})$
is given as in Lemma 3.1. This, together with the standard fact of convex
analysis in Hilbert spaces stating that

$\big{\|}(1-\zeta)\theta^{1}+\zeta\theta^{2}\big{\|}_{2}^{2}\leq(1-\zeta)\|\theta^{1}\|_{2}^{2}+\zeta\|\theta^{2}\|_{2}^{2}-\tfrac{\zeta(1-\zeta)}{2}\big{\|}\theta^{1}-\theta^{2}\big{\|}_{2}^{2}$

allows us to conclude that the reduced cost functional satisfies the
semiconvexity estimate (3.8) over $\Gamma$. ∎

By leveraging the semiconvexity result of Proposition 3.1, we are able to
derive sufficient conditions for the existence of mean-field optimal controls.

######  Theorem 3.2 (Existence of minimizers).

Let $T,R>0$, $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ be such that
$\textnormal{supp}(\mu_{0})\subset B(R)$, and $\Gamma\subset
L^{2}([0,T];\mathbb{R}^{m})$ be the closed ball of radius
$C_{\Gamma}^{1/2}:=\|\ell\|_{\mathcal{C}(B(R))}+1$. If the regularization
parameter is such that $\lambda>\tfrac{1}{2}\mathcal{L}(T,R,\Gamma)$ where the
latter constant is given as in Lemma 3.1, then there exists a unique optimal
control $\theta^{*}\in\Gamma$ for (1.8).

###### Proof.

The result follows from a standard application of the direct method of the
calculus of variations. Given a minimizing sequence $(\theta^{n})\subset
L^{2}([0,T];\mathbb{R}^{m})$ for which

$J(\theta^{n})~{}\underset{n\to+\infty}{\longrightarrow}~{}\inf_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}J(\theta),$ (3.14)

it necessarily holds for $n\geq 1$ sufficiently large that

$J(\theta^{n})\leq J(0)+1\leq\|\ell\|_{\mathcal{C}(B(R))}+1.$

Recalling the expression (3.7) of the reduced cost, this implies in particular
that $\|\theta^{n}\|_{2}\leq C_{\Gamma}^{1/2}$ for each $n\geq 1$, or
equivalently $(\theta^{n})\subset\Gamma$. Remark now that $\Gamma\subset
L^{2}([0,T];\mathbb{R}^{m})$ is weakly compact since it is a closed ball in a
Hilbert space (see e.g. [Brezis, Theorem 3.17]), so that there exists an
element $\theta^{*}\in\Gamma$ for which

$\qquad\theta^{n_{k}}~{}\underset{k\to+\infty}{\rightharpoonup}~{}\theta^{*}\qquad\text{in
$L^{2}([0,T];\mathbb{R}^{m})$},$

along an adequate subsequence. Moreover, it easily follows from Lemma 3.1 that
$\theta\mapsto J(\theta)\in\mathbb{R}$ is continuous in the strong
$L^{2}$-topology, as well as convex since we assumed that
$\lambda>\tfrac{1}{2}\mathcal{L}(T,R,\Gamma)$. As such, it is weakly lower-
semicontinuous (see e.g. [Brezis, Corollary 3.9]), which together with (3.14)
implies that

$J(\theta^{*})\,\leq\,\liminf_{n\to+\infty}J(\theta^{n})\,=\inf_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}J(\theta).$

Hence, we have shown that $\theta^{*}\in\Gamma$ is a solution of the mean-
field optimal control problem (1.8), and its uniqueness follows
straightforwardly from the strict convexity of the reduced cost. ∎

###  3.2 Stability of finitely-sampled costs and controls

In this section, we establish a general stability property for solutions of
the mean-field optimal control problem (1.8) with respect to finite-samples.
More precisely, assume that we are given a sample
$\\{(X_{0}^{i},Y_{0}^{i})\\}_{i=1}^{N}$ of size $N\geq 1$ independently and
identically distributed according to
$\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$, and let us consider the
empirical loss minimization problem

$\inf_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}J^{N}(\theta):=\left\\{\begin{aligned}
&\inf_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}\frac{1}{N}\sum_{i=1}^{N}\ell(X_{T}^{i},Y_{T}^{i})+\lambda\int_{0}^{T}|\theta_{t}|^{2}\textnormal{d}t\\\
&\,\;\textnormal{s.t.}\,\,\left\\{\begin{aligned}
&\dot{X}_{t}^{i}=\mathcal{F}(t,X_{t}^{i},\theta_{t}),\hskip
69.70915pt\dot{Y}_{t}^{i}=0,\\\
&(X_{t}^{i},Y_{t}^{i})_{|t=0}=(X_{0}^{i},Y_{0}^{i}),~{}~{}i\in\\{1,\dots,N\\}.\end{aligned}\right.\end{aligned}\right.$
(3.15)

By introducing the empirical measure
$\mu_{0}^{N}\in\mathcal{P}_{c}^{N}(\mathbb{R}^{2d})$, defined by

$\mu_{0}^{N}:=\frac{1}{N}\sum_{i=1}^{N}\delta_{(X_{0}^{i},Y_{0}^{i})},$ (3.16)

the latter can be rewritten as the mean-field optimal control problem (1.8)
with initial datum $\mu^{N}_{0}$. In the following theorem, we show that when
the regularization parameter $\lambda>0$ is sufficiently large and the
empirical samples satisfy

$W_{1}(\mu_{0}^{N},\mu_{0})~{}\underset{N\to+\infty}{\longrightarrow}~{}0,$
(3.17)

then the minimizers and optimal values of the problems (3.15) converge in a
suitable sense towards those of (1.8). Even though we do not resort explicitly
to this terminology in the sequel, this stability result amounts to showing
that the sequence $(J^{N})$ is $\Gamma$-converging towards $J$ for the weak
topology of $L^{2}([0,T];\mathbb{R}^{m})$ in the sense e.g. of [DM]. Although
it bears some interest and provides insights on the finite data consistency of
the problem, the result that follows is non-quantitative and purely based on
compactness arguments. In order to obtain a quantitative version of this
stability property, it is necessary to establish a smooth relation between
optimal controls the $\theta^{*}$ and the data distributions $\mu_{0}$. Such a
connection will be realized through the fundamental formula (4.8) below, by
leveraging the mean-field PMP studied in Section 4.

######  Theorem 3.3 (Stability of finitely sampled costs and controls).

Let $T,R>0$ be given, $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ be such
that $\textnormal{supp}(\mu_{0})\subset B(R)$, and assume that Assumptions 1
and 2 hold. Moreover, suppose that $\lambda>0$ is sufficiently large in the
sense of Theorem 3.2.

Then for every empirical approximating sequence $(\mu_{0}^{N})$ satisfying
(3.16)-(3.17), the corresponding sequence of optimal controls
$(\theta^{N})\subset L^{2}([0,T];\mathbb{R}^{m})$ is such that

$\theta^{N}~{}\underset{N\to+\infty}{\rightharpoonup}~{}\theta^{*}\qquad\text{in
$L^{2}([0,T];\mathbb{R}^{m})$},$ (3.18)

where $\theta^{*}\in L^{2}([0,T];\mathbb{R}^{m})$ is the unique solution of
(1.8). Moreover, the optimal values converge as well, in the sense that

$J^{N}(\theta^{N})~{}\underset{N\to+\infty}{\longrightarrow}~{}J(\theta^{*})~{}=\min_{\theta\in
L^{2}([0,T];\mathbb{R}^{m})}J(\theta).$ (3.19)

Before proving Theorem 3.3, we state a useful auxiliary lemma exhibiting the
dependence of the reduced empirical cost with respect to the sample size
$N\geq 1$.

######  Lemma 3.2 (Dependence of the reduced cost with respect to $N$).

For every $\theta\in L^{2}([0,T];\mathbb{R}^{m})$, there exists a constant
$C(T,R,\|\theta\|_{1})>0$ such that

$\big{|}J(\theta)-J^{N}(\theta)\big{|}\leq
C(T,R,\|\theta\|_{1})W_{1}(\mu_{0}^{N},\mu_{0})$ (3.20)

and

$\big{\|}\nabla J(\theta)-\nabla J^{N}(\theta)\big{\|}_{2}\leq
C(T,R,\|\theta\|_{1})W_{1}(\mu_{0}^{N},\mu_{0})$ (3.21)

for each $N\geq 1$.

###### Proof.

Let us denote by
$\mu,\mu^{N}\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d})$ the
solutions of (1.7) with control $\theta$ and initial data
$\mu_{0}^{N},\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ respectively. Under
Assumptions 1, it follows from Theorem 2.3 that

$\sup_{t\in[0,T]}W_{1}(\mu_{t}^{N},\mu_{t})\leq
e^{L_{\mathcal{F},T,\|\theta\|_{1}}}W_{1}(\mu_{0}^{N},\mu_{0})$

for some $L_{\mathcal{F},T,\|\theta\|_{1}}>0$. This combined with
Kantorovich’s duality formula (2.6) implies that

$\big{|}J(\theta)-J^{N}(\theta)\big{|}=\bigg{|}\int_{\mathbb{R}^{2d}}\ell(x,y)\textnormal{d}\big{(}\mu_{T}-\mu_{T}^{N}\big{)}(x,y)\bigg{|}\leq\textnormal{Lip}(\ell\,;B(R))\,e^{L_{\mathcal{F},T,\|\theta\|_{1}}}W_{1}(\mu_{0}^{N},\mu_{0}),$

for each $N\geq 1$. Analogously by leveraging the analytical expression (3.12)
of the gradient of the reduced final cost , one also has that

$\displaystyle\big{\|}\nabla J(\theta)-\nabla J^{N}(\theta)\big{\|}_{2}^{2}$
(3.22) $\displaystyle\hskip
35.56593pt\leq\int_{0}^{T}\bigg{(}\int_{\mathbb{R}^{2d}}\Big{(}\mathcal{R}^{\theta}_{(t,T)}(x)\nabla_{\theta}\mathcal{F}\big{(}t,\Phi^{\theta}_{(0,t)}(x),\theta_{t}\big{)}\Big{)}^{\top}\nabla_{x}\ell\big{(}\Phi^{\theta}_{(0,T)}(x),y\big{)}\textnormal{d}\big{(}\mu_{0}-\mu_{0}^{N}\big{)}(x,y)\bigg{)}^{2}\textnormal{d}t.$

At this stage, one can check that as a consequence of Assumptions 1 and 2
along with the definition (B.2) of the resolvent maps that there exists a
constant $C^{\prime}(T,R,\|\theta\|_{1})>0$ such that

$\int_{0}^{T}\Big{\|}\Big{(}\mathcal{R}^{\theta}_{(t,T)}(\cdot)\nabla_{\theta}\mathcal{F}\big{(}t,\Phi^{\theta}_{(0,t)}(\cdot),\theta_{t}\big{)}\Big{)}^{\top}\nabla_{x}\ell\big{(}\Phi^{\theta}_{(0,T)}(\cdot),\cdot\big{)}\Big{\|}_{\mathcal{C}^{1}(B(R))}^{2}\textnormal{d}t\leq
C^{\prime}(T,R,\|\theta\|_{1})^{2}.$ (3.23)

By combining (3.22) and (3.23) with an application of Kantorovich’s duality
formula (2.6), we finally obtain that

$\big{\|}\nabla J(\theta)-\nabla J^{N}(\theta)\big{\|}_{2}\leq
C^{\prime}(T,R,\|\theta\|_{1})W_{1}(\mu_{0}^{N},\mu_{0})$

for each $N\geq 1$, which concludes the proof of Lemma 3.2 by simply setting
$C(T,R,\|\theta\|_{1}):=\max\big{\\{}\textnormal{Lip}(\ell\,;B(R))\,e^{L_{\mathcal{F},T,\|\theta\|_{1}}}\,,\,C^{\prime}(T,R,\|\theta\|_{1})\big{\\}}$.
∎

Building on these a priori estimates, we can move on to the proof of Theorem
3.3.

###### Proof of Theorem 3.3.

Observe first that and because $\textnormal{supp}(\mu_{0}^{N})\subset B(R)$
for each $N\geq 1$ and we assumed $\lambda>0$ to be sufficiently large, there
exists a unique optimal control $\theta^{N}\in L^{2}([0,T];\mathbb{R}^{m})$
solution of (3.15) as a consequence of Theorem 3.2. Noticing again that

$J^{N}(\theta^{N})\leq J^{N}(0)\leq\|\ell\|_{\mathcal{C}^{1}(B(R))}+1$

for each $N\geq 1$, the sequence $(\theta^{N})$ is uniformly contained in the
closed ball $\Gamma\subset L^{2}([0,T];\mathbb{R}^{m})$ whose radius is
defined in Theorem 3.2, and as such it admits a subsequence (that we do not
relabel) which converges weakly to some $\theta^{*}\in
L^{2}([0,T];\mathbb{R}^{m})$.

Our goal is to show that $\theta^{*}$ is the unique minimizer of $J$ and that
the optimal values $(J^{N}(\theta^{N}))$ converge towards $J(\theta^{*})$. To
this end observe first that by Mazur’s lemma (see e.g. [Brezis, Corollary
3.8]), there exists a sequence $(\widetilde{\theta}^{N})$ made of convex
combinations of the elements of $(\theta^{N})$ such that

$\widetilde{\theta}^{N}~{}\underset{N\to+\infty}{\longrightarrow}~{}\theta^{*}\qquad\text{in
$L^{2}([0,T];\mathbb{R}^{m})$}.$

Recalling that $\theta^{N}$ are minimizers of $J^{N}$ and that these latter
are uniformly equi-Lipschitz over $\Gamma$ as a consequence of Lemma 3.1, it
further holds that

$\displaystyle J^{N}(\theta^{N})$ $\displaystyle\leq
J^{N}(\widetilde{\theta}^{N})$ $\displaystyle\leq
J^{N}(\theta^{*})+\big{(}\mathcal{L}(T,R,\Gamma)+2\lambda\big{)}\big{\|}\theta^{*}-\widetilde{\theta}^{N}\big{\|}_{2},$

for each $N\geq 1$. Using the stability estimate (3.20) of Lemma 3.2, we can
pass to the limit in the previous expression and obtain that

$\limsup_{N\to+\infty}J^{N}(\theta^{N})\leq J(\theta^{*}).$ (3.24)

In order to recover a similar inequality for the liminf, notice that the
reduced costs $J^{N}$ are convex by Proposition 3.1, which implies that

$\displaystyle J^{N}(\theta^{N})$ $\displaystyle\geq
J^{N}(\theta^{*})+\big{\langle}\nabla
J^{N}(\theta^{*}),\theta^{N}-\theta^{*}\big{\rangle}_{L^{2}([0,T];\mathbb{R}^{m})}$
(3.25) $\displaystyle\geq J^{N}(\theta^{*})+\big{\langle}\nabla
J(\theta^{*}),\theta^{N}-\theta^{*}\big{\rangle}_{L^{2}([0,T];\mathbb{R}^{m})}+\big{\langle}\nabla
J(\theta^{*})-\nabla
J^{N}(\theta^{*}),\theta^{N}-\theta^{*}\big{\rangle}_{L^{2}([0,T];\mathbb{R}^{m})}$

for each $N\geq 1$. Observe now that by (3.21) in Lemma 3.2, one has that

$\big{\|}\nabla J(\theta^{*})-\nabla
J^{N}(\theta^{*})\big{\|}_{2}~{}\underset{N\to+\infty}{\longrightarrow}~{}0,$

which together with the fact that $(\theta^{N})\subset\Gamma$ is converging
weakly towards $\theta^{*}$ then yields

$\big{\langle}\nabla J(\theta^{*})-\nabla
J^{N}(\theta^{*}),\theta^{*}-\theta^{N}\big{\rangle}_{L^{2}([0,T];\mathbb{R}^{m})}~{}\underset{N\to+\infty}{\longrightarrow}~{}0,$

by standard results on weak-strong convergence (see e.g. [Brezis, Proposition
3.5]). Thus, by passing to the limit as $N\to+\infty$ in (3.25) while using
(3.20) of Lemma 3.2, we recover

$J(\theta^{*})\leq\liminf_{N\to+\infty}J^{N}(\theta^{N}),$ (3.26)

which together with (3.24) finally implies that

$J^{N}(\theta^{N})~{}\underset{N\to+\infty}{\longrightarrow}~{}J(\theta^{*}).$
(3.27)

In order to conclude that $\theta^{*}$ is a minimizer of $J$, it is sufficient
to consider a minimizing sequence $(\theta^{n})\subset\Gamma$ for (1.8) and to
observe that by Lemma 3.2 and (3.27), it holds that

$\displaystyle
J(\theta^{n})=\lim_{N\to+\infty}J^{N}(\theta^{n})\geq\lim_{N\to+\infty}J^{N}(\theta^{N})=J(\theta^{*})$

and to let $n\to+\infty$. The strict convexity of $J$ in turn provides the
uniqueness of $\theta^{*}$, from whence we can deduce that it is the weak
limit of the whole sequence $(\theta^{N})$. ∎

##  4 Mean-Field Maximum Principle

In this section, we investigate first-order optimality conditions for the
mean-field optimal control problem (1.8), which take the form of a mean-field
Pontryagin Maximum Principle (“PMP” for short). Their derivation – which is
based on a Lagrange multiplier rule for the convex calculus introduced in
Section 2 – is heuristically presented in Section 4.1. After studying the
well-posedness of the optimality system in Section 4.2, we proceed to
rigorously establish the PMP throughout Section 4.3.

###  4.1 Formal derivation of the Lagrangian maximum principle

We start this section by providing a formal derivation of the mean-field PMP.
To this end, we first introduce the Lagrangian of the mean-field optimal
control problem (1.8), defined by

$\displaystyle\mathcal{L}(\mu,\theta,\psi)$
$\displaystyle=\int_{\mathbb{R}^{2d}}\ell(x,y)\mathrm{d}\mu_{T}(x,y)+\lambda\int_{0}^{T}|\theta_{t}|^{2}\,{\rm
d}t$ $\displaystyle\hskip
11.66573pt+\int_{\mathbb{R}^{2d}}\psi(0,x,y)\mathrm{d}\mu_{0}(x,y)-\int_{\mathbb{R}^{2d}}\psi(T,x,y)\mathrm{d}\mu_{T}(x,y)$
$\displaystyle\hskip
11.66573pt+\int_{0}^{T}\int_{\mathbb{R}^{2d}}\Big{(}\partial_{t}\psi(t,x,y)+\nabla_{x}\psi(t,x,y)\cdot\mathcal{F}(t,x,\theta_{t})\Big{)}\mathrm{d}\mu_{t}(x,y)\,{\rm
d}t\,.$ (4.1)

Next, we compute its functional derivatives with respect to the curves $\mu$
and $\theta$, namely

$\displaystyle\frac{\delta\mathcal{L}}{\delta\mu_{t}}=\begin{cases}0,&\quad\text{for
$t=0$ (the initial condition is fixed)}\\\
\partial_{t}\psi+\nabla_{x}\psi\cdot{\mathcal{F}},&\quad\text{for
$t\in(0,T)$},\\\ \ell-\psi_{T},&\quad\text{for $t=T$},\end{cases}$

and

$\frac{\delta\mathcal{L}}{\delta\theta_{t}}=2\lambda\theta_{t}^{\top}+\int_{\mathbb{R}^{2d}}\nabla_{x}\psi\cdot\nabla_{\theta}{\mathcal{F}}(t,x,\theta_{t})\textnormal{d}\mu_{t}(x,y)\,.$

for almost every $t\in[0,T]$. Then, given an optimal trajectory-control pair
$(\mu^{*},\theta^{*})$ for the problem (1.8), we will show that there exists a
Lagrange multiplier $\psi^{*}$ such that

$\frac{\delta\mathcal{L}}{\delta\mu}(\mu^{*},\theta^{*},\psi^{*})=0\qquad\mbox{and}\qquad\frac{\delta\mathcal{L}}{\delta\theta}(\mu^{*},\theta^{*},\psi^{*})=0\,.$
(4.2)

These latter will in turn provide us with the following backward adjoint
dynamics

$\partial_{t}\psi^{*}+\nabla_{x}\psi^{*}\cdot\mathcal{F}(t,x,\theta_{t}^{*})=0,$
(4.3)

subject to the terminal condition $\psi^{*}_{T}=\ell$, along with the fixed-
point equation

$2\lambda\theta^{*\top}_{t}+\int_{\mathbb{R}^{2d}}\nabla_{x}\psi^{*}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta^{*}_{t})\mathrm{d}\mu_{t}^{*}(x,y)=0,$
(4.4)

characterizing the optimal controls, where the curve $\mu^{*}$ satisfies the
native forward dynamics

$\partial_{t}\mu_{t}^{*}+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta_{t}^{*})\mu_{t}^{*})=0,\qquad\mu_{t}^{*}|_{t=0}=\mu_{0}\,.$
(4.5)

We will see below that (4.3) is understood in the sense of (4.71), and that
(4.4) is understood in the sense of (4.72).

###  4.2 Well-posedness of the maximum principle

This section is devoted to discussing the existence and uniqueness of a
solution
$(\mu^{*},\theta^{*},\psi^{*})\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))\times\operatorname{Lip}([0,T];\mathbb{R}^{m})\times\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
to the first-order optimality system

$\displaystyle\partial_{t}\mu_{t}^{*}$
$\displaystyle+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta_{t}^{*})\mu_{t}^{*})=0,\hskip
14.22636pt\mu_{t}^{*}|_{t=0}=\mu_{0},$ (4.6)
$\displaystyle\partial_{t}\psi^{*}$
$\displaystyle+\nabla_{x}\psi^{*}\cdot\mathcal{F}(t,x,\theta_{t}^{\ast})=0,\hskip
14.22636pt\psi_{t}^{*}|_{t=T}=\ell,$ (4.7) $\displaystyle\theta^{*\top}_{t}$
$\displaystyle=-\frac{1}{2\lambda}\int_{\mathbb{R}^{2d}}\nabla_{x}\psi^{*}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta^{*}_{t})\mathrm{d}\mu_{t}^{*}(x,y).$
(4.8)

To do so, we consider a compact and convex subset $\Gamma_{M,C}$ of the
subspace
$\mbox{Lip}([0,T];\mathbb{R}^{m})\subset\mathcal{C}([0,T];\mathbb{R}^{m})$,
defined by

$\Gamma_{M,C}:=\Big{\\{}\theta\in\mathcal{C}([0,T];\mathbb{R}^{m})~{}\big{|}~{}|\theta_{t}-\theta_{s}|\leq
M|t-s|,~{}\|\theta\|_{\infty}\leq C_{\Gamma}\Big{\\}}\,.$ (4.9)

for some constants $M,C_{\Gamma}>0$. We will also make use of the following
ball in $L^{2}([0,T];\mathbb{R}^{m})$

$\Gamma_{C}:=\left\\{\theta\in
L^{2}([0,T];\mathbb{R}^{m})~{}|~{}\|\theta\|_{2}\leq
C_{\Gamma}T^{\frac{1}{2}}\right\\}\,.$ (4.10)

One can easily notice that $\Gamma_{M,C}\subset\Gamma_{C}$.

######  Theorem 4.1.

For any given $T>0$, take an initial data
$\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ and a terminal condition
$\psi_{T}$ satisfying (4.14), let ${\mathcal{F}}$ be a map satisfying
Assumptions 1 and 2, and suppose that $\lambda>0$ is large enough.

Then, there exists a triple
$(\mu^{*},\theta^{*},\psi^{*})\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))\times\operatorname{Lip}([0,T];\mathbb{R}^{m})\times\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
solution of (4.6)-(4.8). Moreover, the control solution $\theta^{*}$ is unique
in $\Gamma_{C}\subset L^{2}([0,T];\mathbb{R}^{m})$ defined as in (4.10), and
$\psi^{*}\in\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$ is in
characteristic form.

######  Remark 4.1.

If there exists an optimal control $\theta^{*}\in L^{2}([0,T];\mathbb{R}^{m})$
satisfying the maximum principle (4.6)-(4.8), then the uniqueness result in
Theorem 4.1 ensures that $\theta^{*}$ coincides with a Lipschitz continuous
function almost everywhere. This means that in such a case there exists a
smooth optimal control
$\theta^{*}\in\operatorname{Lip}([0,T];\mathbb{R}^{m})$.

Using arguments that are similar to those of Theorem 2.3, one can show the
following result.

######  Proposition 4.2.

Consider an initial data $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ with
$\textnormal{supp}(\mu_{0})\subset B(R)$ for some $R>0$, and let
${\mathcal{F}}$ satisfy Assumption 1. Then for any $T>0$ and
$\theta\in\Gamma_{M,C}$, there exists a unique solution
$\mu^{\theta}\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))$ to (4.6)
in the sense of Definition 2.2. Moreover, there exists some $R_{T}>0$
depending only on $R$ and $C_{{\mathcal{F}}}$, such that

$\textnormal{supp}(\mu_{t}^{\theta})\subset B(R_{T})\qquad\mbox{for all
}t\in[0,T]\,.$ (4.11)

Additionally, for any $s,t\in[0,T]$, it holds

$W_{1}(\mu_{t}^{\theta},\mu_{s}^{\theta})\leq C(R,C_{{\mathcal{F}}})|t-s|\,.$
(4.12)

If $\mu^{\theta,i}$, $i=1,2$ are two solutions with initial data $\mu_{0}^{i}$
satisfying the above assumptions, we have

$W_{1}(\mu_{t}^{\theta,1},\mu_{t}^{\theta,2})\leq
e^{L_{{\mathcal{F}},T,C_{\Gamma}}}W_{1}(\mu_{0}^{1},\mu_{0}^{2})\quad\mbox{
for all }t\in[0,T]\,.$ (4.13)

Here $C_{{\mathcal{F}}}$ and $L_{{\mathcal{F}},T,C_{\Gamma}}$ are defined as
in Assumption 1 by replacing $\|\theta\|_{1}$ by $C_{\Gamma}T$.

In what follows, we will only be interested in what is happening inside the
supports of $\mu^{\theta}$ for $\theta\in\Gamma_{M,C}$. Therefore, we shall
recast the terminal condition in (4.7) as
$\psi_{T}\in\mathcal{C}_{c}^{2}(\mathbb{R}^{2d})$ with

$\textnormal{supp}(\psi_{T})=B(R_{T})\quad\text{and}\quad\psi_{T}(x,y)=\ell(x,y)~{}~{}\mbox{
for all }x,y\in B(R_{T})\,.$ (4.14)

In this context, we are able to derive the following norm estimate on
$\psi^{\theta}$.

######  Proposition 4.3.

Suppose that ${\mathcal{F}}$ satisfies Assumption 1. Then for any $T>0$ and
$\theta\in\Gamma_{M,C}$, there exists a unique characteristic solution
$\psi^{\theta}\in\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
to the equation (4.7) which terminal condition satisfies (4.14). Moreover it
holds

$\big{\|}\psi_{t}^{\theta}\big{\|}_{\mathcal{C}_{c}^{2}(\mathbb{R}^{2d})}\leq
C(R^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R_{T}))}\,,$
(4.15)

for all times $t\in[0,T]$. Here the supports of $\psi_{t}^{\theta}$ satisfies
the inclusion $\textnormal{supp}(\psi_{t}^{\theta})\subset B(R_{T}^{\prime})$
where $R^{\prime}=R+(R+C_{{\mathcal{F}}}T)e^{C_{{\mathcal{F}}}T}$.

The results of Proposition 4.3 are classical, and we postpone their proof to
Appendix A.

######  Remark 4.2.

Here, the fact that $\psi^{\theta}$ is a _characteristic solution_ means that
it is obtained via the characteristic method, and is of the form
$\psi^{\theta}(t,x,y)=\psi_{T}(\boldsymbol{\Phi}_{(T,t)}^{\theta}(x,y))$.
Therein, we denoted by
$(\boldsymbol{\Phi}_{(\tau,t)}^{\theta})_{\tau,t\in[0,T]}$ the flow maps
defined as in (A.3) with ${\mathcal{F}}(t,x):={\mathcal{F}}(t,x,\theta_{t})$.
Characteristic solutions to (4.8) are unique because of the way they depends
on the terminal condition (4.15). Note here that we do not claim to have
general uniqueness in
$\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$ for (4.8), i.e.
there may exist $\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
solutions that are not in the characteristic form. In what follows however, we
will only consider characteristic solutions.

###### Proof of Theorem 4.1.

The existence of optimal controls $\theta^{*}$ in $\Gamma_{M,C}$ is based on
the Schauder fixed point theorem [gilbarg2015elliptic, Theorem 11.1]. Then,
the uniqueness will be obtained by additionally showing that the underlying
fixed-point map is a contraction in $\Gamma_{C}$.

$\bullet$ (Existence in $\Gamma_{M,C}$) For any $\theta\in\Gamma_{M,C}$,
denote by $\mu^{\theta}\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))$
the corresponding solution of (4.6) and by
$\psi^{\theta}\in\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
the unique characteristic solution of (4.7). In this context, we introduce the
continuous mapping
$\Lambda:~{}\Gamma_{M,C}\to\mathcal{C}([0,T];\mathbb{R}^{m})$, defined by

$\Lambda(\theta)(t)^{\top}=-\frac{1}{2\lambda}\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{t}^{\theta}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t})\mathrm{d}\mu_{t}^{\theta}(x,y),$
(4.16)

for every $\theta\in\Gamma_{M,C}$ and all times $t\in[0,T]$. We start by
checking that $\Lambda(\Gamma_{M,C})\subset\Gamma_{M,C}$ for $\lambda$ large
enough. On the one hand, it follows Assumption 1-$(iii)$ and (4.15) that

$\displaystyle|\Lambda(\theta)(t)|$
$\displaystyle\leq\frac{1}{2\lambda}\int_{B(R_{T})}\big{|}\nabla_{x}\psi_{t}^{\theta}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t})\big{|}\mathrm{d}\mu_{t}^{\theta}(x,y)$
$\displaystyle\leq\frac{1}{2\lambda}C(R_{T},T)\sup_{t\in[0,T]}\big{\|}\psi_{t}^{\theta}\big{\|}_{\mathcal{C}^{1}(B(R_{T}^{\prime}))}$
$\displaystyle\leq\frac{1}{2\lambda}C(R_{T},T)C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{1}(B(R_{T}))},$

for all $t\in[0,T]$, with the explicit constant
$R_{T}^{\prime}:=R+(R+C_{{\mathcal{F}}}T)e^{C_{{\mathcal{F}}}T}$. Hence, upon
choosing a parameter $\lambda>0$ that is large enough, it holds

$\left\lVert\Lambda(\theta)\,\right\rVert_{L^{\infty}([0,T];\mathbb{R}^{m})}\leq
C_{\Gamma}.$ (4.17)

On the other hand, one has for any $s,t\in[0,T]$ that

$\displaystyle|\Lambda(\theta)(t)-\Lambda(\theta)(s)|$
$\displaystyle\leq\frac{1}{2\lambda}\left|\int_{B(R_{T})}\big{(}\nabla_{x}\psi_{t}^{\theta}-\nabla_{x}\psi_{s}^{\theta}\big{)}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t})\mathrm{d}\mu_{t}^{\theta}(x,y)\right|$
$\displaystyle\hskip
12.80365pt+\frac{1}{2\lambda}\left|\int_{B(R_{T})}\nabla_{x}\psi_{s}^{\theta}\cdot(\nabla_{\theta}\mathcal{F}(t,x,\theta_{t})-\nabla_{\theta}\mathcal{F}(s,x,\theta_{s}))\mathrm{d}\mu_{t}^{\theta}(x,y)\right|$
$\displaystyle\hskip
12.80365pt+\frac{1}{2\lambda}\left|\int_{B(R_{T})}\nabla_{x}\psi_{s}^{\theta}\cdot\nabla_{\theta}\mathcal{F}(s,x,\theta_{s})(\mathrm{d}\mu_{t}^{\theta}-\mathrm{d}\mu_{s}^{\theta})(x,y)\right|$
$\displaystyle=:I_{1}+I_{2}+I_{3}.$

Using the fact that
$\psi^{\theta}\in\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
along with Assumption 1-$(iii)$, one can see that

$I_{1}\leq\frac{1}{2\lambda}C(R_{T},T)|t-s|\,,$ (4.18)

for all $s,t\in[0,T]$. Furthermore, it follows from assumption (3.2) and the
estimate (4.15) that

$\displaystyle I_{2}$
$\displaystyle\leq\frac{1}{2\lambda}C(R_{T})\sup_{t\in[0,T]}\big{\|}\psi_{t}^{\theta}\big{\|}_{\mathcal{C}^{1}(B(R_{T}^{\prime}))}\big{(}|t-s|+|\theta_{t}-\theta_{s}|\big{)}$
$\displaystyle\leq\frac{1}{2\lambda}C(R_{T})C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{1}(B(R_{T}))}M|t-s|,$
(4.19)

with $R_{T}^{\prime}:=R+(R+C_{{\mathcal{F}}}T)e^{C_{{\mathcal{F}}}T}$. Lastly
by Kantorovich’s duality formula (2.6), one has

$I_{3}\leq\frac{1}{2\lambda}\mbox{Lip}\big{(}\nabla_{x}\psi_{s}^{\theta}\cdot\nabla_{\theta}\mathcal{F}(s,\cdot,\theta_{s})\,;B(R_{T})\big{)}W_{1}(\mu_{t},\mu_{s}),$
(4.20)

and can further notice that

$\displaystyle\mbox{Lip}\big{(}\nabla_{x}\psi_{s}^{\theta}\cdot\nabla_{\theta}\mathcal{F}(s,\cdot,\theta_{s})\,;B(R_{T})\big{)}$
$\displaystyle\leq
C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R_{T}))}$
$\displaystyle\hskip
14.22636pt\times\Big{(}\|\nabla_{\theta}{\mathcal{F}}(s,\cdot,\theta_{s})\|_{L^{\infty}(B(R_{T}))}+\mbox{Lip}(\nabla_{\theta}{\mathcal{F}}(s,\cdot,\theta_{s})\,;B(R_{T}))\Big{)}$
$\displaystyle\leq
C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}},R_{T})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R_{T}))},$

where we have used (4.15) and Assumption 2-$(iii)$. This combined with (4.12)
thus yields

$I_{3}\leq\frac{1}{2\lambda}C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}},R_{T})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R_{T}))}|t-s|.$
(4.21)

Collecting estimates (4.18), (4.2) and (4.21), we deduce that for $\lambda>0$
large enough, it holds

$|\Lambda(\theta)(t)-\Lambda(\theta)(s)|\leq M|t-s|.$ (4.22)

Thus, we have proven that $\Lambda(\Gamma_{M,C})\subset\Gamma_{M,C}$ when
$\lambda>0$ is taken to be sufficiently large. Hence by Schauder’s fixed point
theorem, the mapping $\Lambda$ has at least a fixed point $\theta^{*}$, namely

$\theta^{*\top}=-\frac{1}{2\lambda}\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{t}^{\theta^{*}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{*})\mathrm{d}\mu_{t}^{\theta^{*}}(x,y).$
(4.23)

This concludes the existence part of the proof.

$\bullet$ (Uniqueness in $\Gamma_{C}$) Our goal now is to prove that $\Lambda$
is a contraction over $\Gamma_{M,C}$ with respect to the $L^{2}$-norm, so that
that the fixed point $\theta^{*}\in\Gamma_{M,C}$ is actually unique in
$\Gamma_{C}$. Indeed assuming that $\Lambda$ had two distinct fixed points
$\theta^{1}$ and $\theta^{2}$, it would hold

$\|\theta^{1}-\theta^{2}\|_{2}=\|\Lambda(\theta^{1})-\Lambda(\theta^{2})\|_{2}\leq\kappa\|\theta^{1}-\theta^{2}\|_{2},$

which leads to a contradiction for contraction constants satisfying
$0\leq\kappa<1$. In order to prove the contractivity of $\Lambda$, we start by
fixing $t\in[0,T]$ and denote by $\mu^{\theta^{1}},\mu^{\theta^{2}}$ two
solutions of (4.6) driven by $\theta^{1},\theta^{2}$ respectively, with the
same initial condition $\mu_{0}$. Similarly, denote by
$\psi^{\theta^{1}},\psi^{\theta^{2}}$ the solutions of (4.7) generated by
$\theta^{1},\theta^{2}$ with the same terminal condition $\psi_{T}$. Then

$\displaystyle|\Lambda(\theta^{1})(t)-\Lambda(\theta^{2})(t)|$
$\displaystyle=\frac{1}{2\lambda}\left|\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{t}^{\theta^{1}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})\mathrm{d}\mu_{t}^{\theta^{1}}(x,y)-\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{t}^{\theta^{2}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{2})\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)\right|$

which can in turn be estimated by inserting suitable crossed terms as

$\displaystyle|\Lambda(\theta^{1})(t)-\Lambda(\theta^{2})(t)|$
$\displaystyle\leq\frac{1}{2\lambda}\left|\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{t}^{\theta^{1}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})(\mathrm{d}\mu_{t}^{\theta^{1}}-\mathrm{d}\mu_{t}^{\theta^{2}})(x,y)\right|$
$\displaystyle\quad+\frac{1}{2\lambda}\left|\int_{\mathbb{R}^{2d}}\Big{(}\nabla_{x}\psi_{t}^{\theta^{1}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})-\nabla_{x}\psi_{t}^{\theta^{2}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{2})\Big{)}\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)\right|$
$\displaystyle=:\frac{1}{2\lambda}(|I_{1}|+|I_{2}|).$

We start by further simplifying the integral term $I_{2}$, which can be recast
as

$\displaystyle|I_{2}|$
$\displaystyle=\bigg{|}\int_{\mathbb{R}^{2d}}\Big{(}\nabla_{x}\psi_{t}^{\theta^{1}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})-\nabla_{x}\psi_{t}^{\theta^{2}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})$
$\displaystyle\qquad\qquad+\nabla_{x}\psi_{t}^{\theta^{2}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})-\nabla_{x}\psi_{t}^{\theta^{2}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{2})\Big{)}\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)\bigg{|}$
$\displaystyle\leq\left|\int_{\mathbb{R}^{2d}}(\nabla_{x}\psi_{t}^{\theta^{1}}-\nabla_{x}\psi_{t}^{\theta^{2}})\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)\right|$
$\displaystyle\quad+\left|\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{t}^{\theta^{2}}\cdot(\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})-\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{2}))\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)\right|$
$\displaystyle=:|I_{3}|+|I_{4}|.$

Hence, the estimate in (4.2) is equivalent to

$|\Lambda(\theta^{1})(t)-\Lambda(\theta^{2})(t)|\leq\frac{1}{2\lambda}(|I_{1}|+|I_{3}|+|I_{4}|)\,.$
(4.24)

Let us focus on each term separately, starting with the integral $I_{1}$.
Henceforth, we only consider the integrals over $B(R_{T})$, in which the
curves $\mu^{\theta^{i}}$ are supported for $i=1,2$. By using the same
reasoning as in (4.21), we have that

$\displaystyle|I_{1}|$
$\displaystyle=\left|\int_{B(R_{T})}\nabla_{x}\psi_{t}^{\theta^{1}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})(\mathrm{d}\mu_{t}^{\theta^{1}}-\mathrm{d}\mu_{t}^{\theta^{2}})(x,y)\right|$
$\displaystyle\leq\operatorname{Lip}\big{(}\nabla_{x}\psi_{t}^{\theta^{1}}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})\,;B(R_{T})\big{)}W_{1}(\mu_{t}^{\theta^{1}},\mu_{t}^{\theta^{2}})$
$\displaystyle\leq
C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}},R_{T})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R^{\prime}_{T}))}W_{1}(\mu_{t}^{\theta^{1}},\mu_{t}^{\theta^{2}}).$
(4.25)

Observe now that following Appendix A, the curves $\mu_{t}^{\theta^{1}}$ and
$\mu_{t}^{\theta^{2}}$ are characteristic solutions of (4.6), in the sense
that

$\mu_{t}^{\theta^{i}}=\boldsymbol{\Phi}_{(0,t)}^{\theta^{i}}\sharp\mu_{0}$
(4.26)

for all times $t\in[0,T]$, where
$\boldsymbol{\Phi}_{(0,t)}^{\theta^{i}}(\cdot)$ are the flow maps of the
underlying ODEs

$\displaystyle\frac{\mathrm{d}X^{i}_{t}}{\,{\rm
d}t}=\mathcal{F}(t,X^{i}_{t},\theta^{i}_{t}),\qquad\frac{\mathrm{d}Y^{i}_{t}}{\,{\rm
d}t}=0,\qquad(X_{0}^{i},Y_{0}^{i})=(x_{0},y_{0})$

for $i=1,2$. Then, it follows from Assumption 1 that

$\displaystyle\big{|}(X_{t}^{1},Y_{t}^{1})-(X_{t}^{2},Y_{t}^{2})\big{|}$
$\displaystyle=\left|\left(x_{0}-x_{0}+\int_{0}^{t}(\mathcal{F}(s,X_{s}^{1},\theta_{s}^{1})-\mathcal{F}(s,X_{s}^{2},\theta_{s}^{2}))\mathrm{d}s,y_{0}-y_{0}\right)\right|$
$\displaystyle\leq\int_{0}^{t}\left|(\mathcal{F}(s,X_{s}^{1},\theta_{s}^{1})-\mathcal{F}(s,X_{s}^{2},\theta_{s}^{2})\right|\mathrm{d}s$
$\displaystyle\leq\int_{0}^{t}\big{|}\mathcal{F}(t,X_{s}^{1},\theta_{s}^{1})-\mathcal{F}(t,X_{s}^{2},\theta_{s}^{1})\big{|}\mathrm{d}s+\int_{0}^{t}\mathcal{F}(t,X_{s}^{2},\theta_{s}^{1})-\mathcal{F}(t,X_{s}^{2},\theta_{s}^{2})\big{|}\mathrm{d}s$
$\displaystyle\leq
L_{{\mathcal{F}},T,C_{\Gamma}}\int_{0}^{t}|X_{s}^{1}-X_{s}^{2}|\mathrm{d}s+C(R_{T},T)\int_{0}^{t}|\theta_{s}^{1}-\theta_{s}^{2}|\mathrm{d}s.$
(4.27)

Then by Gronwall’s lemma and the definition of Wasserstein distance, we obtain

$W_{1}(\mu_{t}^{\theta^{1}},\mu_{t}^{\theta^{2}})\leq
W_{1}\Big{(}\boldsymbol{\Phi}_{(0,t)}^{\theta^{1}}\sharp\mu_{0},\boldsymbol{\Phi}_{(0,t)}^{\theta^{2}}\sharp\mu_{0}\Big{)}\leq
C(R_{T},T)e^{L_{{\mathcal{F}},T,C_{\Gamma}}T}\|\theta^{1}-\theta^{2}\|_{2},$
(4.28)

and by using (4.28) in (4.2), it further holds that

$|I_{1}|\leq
C(R_{T}^{\prime},T,C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}},R_{T})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R_{T}))}\|\theta^{1}-\theta^{2}\|_{2}.$
(4.29)

We now shift our focus to the integral $I_{3}$. By Assumption 2-$(i)$, we have
that

$\displaystyle|I_{3}|$
$\displaystyle=\left|\int_{B(R_{T})}(\psi_{t}^{\theta^{1}}-\psi_{t}^{\theta^{2}})\nabla_{x}\cdot\nabla_{\theta}\mathcal{F}(t,x,\theta_{t}^{1})\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)\right|$
$\displaystyle\leq
C(R_{T},T,C_{\Gamma})\sup_{t\in[0,T]}\big{\|}\psi_{t}^{\theta^{1}}-\psi_{t}^{\theta^{2}}\big{\|}_{\mathcal{C}(B(R_{T}^{\prime}))}.$
(4.30)

Recalling that $\psi^{\theta^{1}},\psi^{\theta^{2}}$ are characteristic
solutions of (4.7) while using (A.16), one further has

$\displaystyle\left\lVert\psi_{t}^{\theta^{1}}-\psi_{t}^{\theta^{2}}\,\right\rVert_{\mathcal{C}(B(R_{T}^{\prime}))}$
$\displaystyle=\left\lVert\psi_{T}\big{(}\boldsymbol{\Phi}^{\theta^{1}}_{(t,T)}\big{)}-\psi_{T}\big{(}\boldsymbol{\Phi}^{\theta^{2}}_{(t,T)}\big{)}\,\right\rVert_{\mathcal{C}(B(R_{T}^{\prime}))}$
$\displaystyle\leq\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{1}(B(R_{T}))}\left\lVert\boldsymbol{\Phi}^{\theta^{1}}_{(t,T)}-\boldsymbol{\Phi}^{\theta^{2}}_{(t,T)}\,\right\rVert_{C(B(R_{T}))}.$
(4.31)

Besides, it simply follows from Proposition B.1 that

$\sup_{t\in[0,T]}\big{\|}\boldsymbol{\Phi}^{\theta^{1}}_{(t,T)}-\boldsymbol{\Phi}^{\theta^{2}}_{(t,T)}\big{\|}_{\mathcal{C}(B(R_{T}))}\leq
C(R_{T},T)e^{L_{{\mathcal{F}},T,C_{\Gamma}}T}\|\theta^{1}-\theta^{2}\|_{2},$
(4.32)

for some given constant $C(R_{T},T)e^{L_{{\mathcal{F}},T,C_{\Gamma}}T}>0$.
Therefore, the term $I_{3}$ can be estimated as

$|I_{3}|\leq
C(T,R_{T},C_{\Gamma},C_{{\mathcal{F}}})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{1}((B(R_{T}))}\|\theta^{1}-\theta^{2}\|_{2}\,.$
(4.33)

Lastly, we focus on the integral quantity $I_{4}$. Using Assumption
(1)-$(iv)$, we can write

$\displaystyle|I_{4}|$
$\displaystyle\leq\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{T}^{\theta^{2}}\cdot\left|\nabla_{\theta}(\mathcal{F}(t,x,\theta_{t}^{1})-\mathcal{F}(t,x,\theta_{t}^{2}))\right|\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)$
$\displaystyle\leq\int_{\mathbb{R}^{2d}}\nabla_{x}\psi_{T}^{\theta^{2}}\cdot\left|\nabla_{\theta}^{2}(\mathcal{F}(t,x,\theta)\right|\left|\theta_{t}^{1}-\theta_{t}^{2}\right|\mathrm{d}\mu_{t}^{\theta^{2}}(x,y)$
$\displaystyle\leq
C(R_{T},T)|\theta_{t}^{1}-\theta_{t}^{2}|\sup_{t\in[0,T]}\left\lVert\psi_{t}^{\theta^{2}}\,\right\rVert_{C^{1}(B(R_{T}^{\prime}))}$
$\displaystyle\leq
C(R_{T},T,R_{T}^{\prime},C_{\Gamma},C_{{\mathcal{F}}},L_{{\mathcal{F}},T,C_{\Gamma}})\big{\|}\psi_{T}\big{\|}_{\mathcal{C}^{1}(B(R_{T}))}|\theta_{t}^{1}-\theta_{t}^{2}|\,.$
(4.34)

Collecting the estimates from (4.29), (4.33) and (4.2), we can conclude

$\displaystyle\|\Lambda(\theta^{1})-\Lambda(\theta^{2})\|_{2}$
$\displaystyle\leq\frac{1}{2\lambda}C(R_{T}^{\prime},R_{T},T,C_{{\mathcal{F}}},C_{\Gamma},L_{{\mathcal{F}},T,C_{\Gamma}})\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(B(R_{T}))}\|\theta^{1}-\theta^{2}\|_{2}$
$\displaystyle=\kappa_{\lambda}\|\theta^{1}-\theta^{2}\|_{2}\,.$

Hence by choosing the parameter $\lambda>0$ to be large enough, we obtain that
$\kappa_{\lambda}<1$, which means that the mapping
$\Lambda:~{}\Gamma_{M,C}\to~{}\Gamma_{M,C}$ is a contraction and thus that its
fixed point $\theta^{*}$ is unique in $\Gamma_{C}$. Thus we have obtained a
solution
$(\mu^{*},\theta^{*},\psi^{*})\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))\times\Gamma_{M,C}\times\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
to equations (4.6)-(4.8), and it is unique in $\Gamma_{C}$. ∎

######  Remark 4.3.

As it was shown in the proof above, the size condition imposed on $\lambda$
depends on some constant
$C(|R_{T}^{\prime}|,R_{T},T,C_{{\mathcal{F}}},C_{\Gamma},L_{{\mathcal{F}},T,C_{\Gamma}})$
and $\left\lVert\psi_{T}\,\right\rVert_{\mathcal{C}^{2}(R_{T})}$. Especially
for the case $\mathcal{F}(t,x,\theta):=\tanh(\theta x)$, we can simplify the
constant as $C(R_{T},T,C_{\Gamma})$, which shows that $\lambda$ depends on the
size of the support of $\mu_{0}$, on the final time $T>0$ and on the constant
$C_{\Gamma}$.

In addition to its usefulness in characterizing and computing optimal
controls, the mean-field maximum principle allows us to derive a quantitative
norm rate of convergence of the latter with respect to the $L^{p}$-norms and a
quantitative generalization error.

######  Corollary 4.4.

For any $T,>0$, let $\mu_{0}\in\mathcal{P}_{c}(\mathbb{R}^{2d})$ be such that
$\textnormal{supp}(\mu_{0})\subset B(R)$ and and $\psi_{T}$ be a terminal
condition satisfying (4.14), and suppose Assumptions 1 and 2 hold. Moreover,
assume that for each $N\geq 1$ we are given an approximating empirical measure
of the form

$\mu_{0}^{N}:=\frac{1}{N}\sum_{i=1}^{N}\delta_{(X^{i}_{0},Y^{i}_{0})}\in\mathcal{P}_{c}^{N}(\mathbb{R}^{2d}),$

such that

$\lim_{N\to\infty}W_{1}(\mu^{N}_{0},\mu_{0})=0.$

Let $\lambda>0$ be sufficiently large so that
$(\mu^{*},\theta^{*},\psi^{*})\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))\times\operatorname{Lip}([0,T];\mathbb{R}^{m})\times\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
and
$(\mu^{N},\theta^{N},\psi^{N})\in\mathcal{C}([0,T];\mathcal{P}_{c}^{N}(\mathbb{R}^{2d}))\times\operatorname{Lip}([0,T];\mathbb{R}^{m})\times\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
are the unique solutions of (4.6)-(4.8) with initial conditions $\mu_{0}$ and
$\mu_{0}^{N}$ respectively. Then

$\max\bigg{\\{}\|\theta^{N}-\theta^{*}\|_{p}\;,\,\sup_{t\in[0,T]}W_{1}(\mu^{N}_{t},\mu^{*}_{t})\,,\,\|\psi^{N}-\psi^{*}\|_{\mathcal{C}([0,T]\times
B(R_{T}))}\bigg{\\}}\leq CW_{1}(\mu_{0}^{N},\mu_{0}),$ (4.35)

for a constant $C>0$ which only depends on the parameters of the model and
$p\in[1,\infty]$, and where $R_{T}>0$ is defined as in Proposition 4.2 above.
In particular, we obtain the following quantitative generalization error
estimate

$\bigg{|}\int_{\mathbb{R}^{2d}}\ell(x,y)\,\mathrm{d}\mu_{T}^{*}(x,y)-\frac{1}{N}\sum_{i=1}^{N}\ell\big{(}X^{i}_{T},Y^{i}_{T}\big{)}\bigg{|}\leq
CW_{1}(\mu_{0}^{N},\mu_{0}).$ (4.36)

###### Proof.

By using similar arguments as in the proof of Theorem 4.1, see in particular
(4.13) and (4.2)-(4.28), we can prove the stability estimate

$\displaystyle\sup_{t\in[0,T]}W_{1}(\mu_{t}^{N},\mu_{t}^{*})$
$\displaystyle\leq\sup_{t\in[0,T]}W_{1}(\mu_{t}^{N},\mu_{t}^{\theta^{N}})+\sup_{t\in[0,T]}W_{1}(\mu_{t}^{\theta^{N}},\mu_{t}^{*})$
$\displaystyle\leq
C\left(W_{1}(\mu_{0}^{N},\mu_{0})+\int_{0}^{T}|\theta^{N}_{t}-\theta^{*}_{t}|\,{\rm
d}t\right)$ (4.37) $\displaystyle\leq
C\Big{(}W_{1}(\mu_{0}^{N},\mu_{0})+\|\theta^{N}-\theta^{*}\|_{p}\Big{)},$
(4.38)

where $\mu_{t}^{\theta^{N}}$ is the unique solution of (4.5) driven by
$\theta^{N}$ with initial datum $\mu_{0}$, and $C>0$ is an overloaded constant
depending on the data of the problem. Similarly, from (4.2), (4.31) and
(4.32), we have that

$\|\psi^{N}-\psi^{*}\|_{\mathcal{C}([0,T]\times B(R_{T}))}\leq
C\int_{0}^{T}|\theta^{N}_{t}-\theta^{*}_{t}|\,{\rm d}t\leq
C\|\theta^{N}-\theta^{*}\|_{p},$ (4.39)

for any $p\in[1,+\infty]$. Finally, by using the fixed point equations

$\theta^{N}=\Lambda(\theta^{N})\qquad\text{and}\qquad\theta^{*}=\Lambda(\theta^{*}),$

and following the estimates in the proof of Theorem 4.1, see in particular
(4.24), (4.2), (4.2) and (4.2), we obtain

$\displaystyle\|\theta^{N}-\theta^{*}\|_{p}$
$\displaystyle=\|\Lambda(\theta^{N})-\Lambda(\theta^{*})\|_{p}$
$\displaystyle\leq\frac{C}{\lambda}\bigg{(}\|\theta^{N}-\theta^{*}\|_{p}+\sup_{t\in[0,T]}W_{1}(\mu_{t}^{N},\mu_{t}^{*})+\|\psi^{N}-\psi^{*}\|_{\mathcal{C}([0,T]\times
B(R_{T}))}\bigg{)}$
$\displaystyle\leq\frac{C}{\lambda}\left(W_{1}(\mu_{0}^{N},\mu_{0})+\|\theta^{N}-\theta^{*}\|_{p}\right),$

where we applied (4.2) and (4.39) in the last inequality. Hence for
$\lambda>0$ large enough, it holds

$\|\theta^{N}-\theta^{*}\|_{p}\leq CW_{1}(\mu_{0}^{N},\mu_{0}).$ (4.40)

Combining now (4.2), (4.39) and (4.40) finally yields (4.35). The
generalization error displayed in (4.36) follows from (4.2) and (4.40), since

$\displaystyle\left|\int_{\mathbb{R}^{2d}}\ell(x,y)\,\mathrm{d}(\mu_{T}^{*}(x,y)-\mu^{N}_{T}(x,y))\right|$
$\displaystyle\leq\operatorname{Lip}(\ell\,;B(R_{T}))\sup_{t\in[0,T]}W_{1}(\mu_{t}^{N},\mu_{t}^{*})$
$\displaystyle\leq
C\left(W_{1}(\mu_{0}^{N},\mu_{0})+\|\theta^{N}-\theta^{*}\|_{p}\right)$
$\displaystyle\leq CW_{1}(\mu_{0}^{N},\mu_{0}).$

This completes the proof of Corollary 4.4. ∎

######  Remark 4.4 (Data bounds, regularization parameters and error
estimates).

The estimate (4.36) is in the worst case affected by the curse of dimension,
although it will not be the case in practice e.g. for networks driven by
sigmoid activation functions. The constant $C$ in (4.36) is encoding the
complexity of the NeurODE and is derived as a consequence of (4.40) as

$C=C_{1}(1-C_{0}/\lambda)>0.$

Therein, the constant $C_{0}>0$ may depend exponentially on the constants
$C_{{\mathcal{F}}}$ and $L_{{\mathcal{F}}}$ appearing in Assumptions 1 – and
in particular on the dimension $d\geq 1$ of the state space –, and
polynomially on those of Assumptions 2, owing to the pessimistic nature of
deterministic Grönwall estimates. Thus, as long as the worst-case Grönwall
estimates do indeed reflect the actual stability of the PMP, the constant
$C_{0}>0$ may be extremely large. Nevertheless, in the case of sigmoidal-type
activation functions such as $\rho:=\tanh$, we detailed in Remark 3.1 how the
uniform boundedness of the velocity field $\mathcal{F}$ implied a polynomial
dependence of all the relevant constants of the problem with respect to the
state space dimension. Therefore, in that particular yet relevant case, the
quantity $C_{0}$ will in fact scale polynomially and not exponentially with
$d$.

For arbitrary initial measures $\mu_{0}$, it is known that empirical measures
$\mu_{N}$ supported on finite samples satisfy the estimate

$\mathbb{E}\big{[}W_{1}(\mu_{0}^{N},\mu_{0})\big{]}\leq CN^{-1/d},$

see for instance [10.1214/12-AIHP489, FG15], which scales quite badly with the
dimension $d\geq 1$ of the state space. However, if $\mu_{0}$ is concentrated
around manifolds of lower dimension, then the factor $C>0$ depends favorably
on that intrinsic lower dimension [10.3150/18-BEJ1065]. In practice, it is
expected that data distributions do concentrate around such lower-dimensional
structures.

###  4.3 Rigorous derivation of the mean-field maximum principle

The previous section, we proved the well-posedness of the mean-field PMP
(4.6)-(4.8) in the class of control that are Lipschitz continuous with respect
to time. Under this assumption, we rigorously derive in what follows the
optimality conditions by using a generalized Lagrange multiplier theorem over
convex sets. The method we present is to a certain extent a standard calculus
of variations approach, and allows to bypass the more technical ones based
either on the abstract differential calculus of Wasserstein as in
[SetValuedPMP, PMPWassConst, PMPWass], or on the fine structural results for
continuity equations leveraged in [burger].

Let it be stressed that the requirement of continuity of the control is purely
technical, and stems from our use of [piccoli2019wasserstein, Theorem 1]
concerning the well-posedness of transport equations with sources. Were such
results available in the case where the source terms are merely measurable in
time – which seems true but is not written anywhere yet –, we could then
remove the continuity assumption and prove the mean-field PMP in its full
generality using the Lagrangian approach.

####  4.3.1 A Lagrange Multiplier Theorem over convex sets

Let $X$ and $Y$ be Banach spaces, $E\subset X$ be a convex set,
$J:E\rightarrow\mathbb{R}$ be a continuous functional and $G:E\rightarrow Y$
be a linear mapping, both continuously $F$-differentiable on $E$ in the sense
of (2.17). For $x^{*}\in E$, we introduce the notation

$DG(x^{*}):=\Big{\\{}L\in\mathcal{L}(X_{E},Y)~{}\big{|}~{}L\mbox{ satisfies
}\eqref{A1}\Big{\\}}\,.$ (4.41)

It is known that every $L\in\mathcal{L}(X_{E},Y)$ can be uniquely extended to
a operator $\overline{L}\in\mathcal{L}(\overline{X}_{E},Y)$ over the Banach
space $\overline{X}_{E}$. In what follows, we will slightly abuse the notation
$DG(x^{*})$ to denote the set of operators obtained after extending the convex
subgradients to $\overline{X}_{E}$.

In the following theorem, we extend the Lagrange multiplier theorem for the
Banach space [zeidler1995applied, Section 4.14] to the setting of the calculus
for convex subsets introduced in Section 2. To ease the readability of the
paper, the proof of this result is reported in Appendix C.

######  Theorem 4.5.

Let $x^{*}\in E$ be a solution of the constrained optimization problem

$\left\\{\begin{aligned} \inf\limits_{x\in E}&J(x),\\\
\textnormal{s.t.}~{}&G(x)=0.\end{aligned}\right.$ (4.42)

Suppose moreover that the inclusion $x^{*}+X_{E}\subset E$ holds, and that
there exists some $G^{\prime}(x^{*})\in DG(x^{*})$ that is a surjective
operator from $\overline{X}_{E}$ into $Y$. Then for any $J^{\prime}(x^{*})\in
DJ(x^{*})$, there exists a non-zero covector $p^{*}\in Y^{\prime}$ which
satisfies

$\langle J^{\prime}(x^{\ast}),z\rangle+\langle
G^{\prime}(x^{\ast})z,p^{*}\rangle=0$ (4.43)

for all $z\in\overline{X}_{E}$.

####  4.3.2 Preparation and verification of assumptions

Recall that in Theorem 2.3, we have shown that for every $\theta\in
L^{2}([0,T];\mathbb{R}^{m})$, there exists a unique solution
$\mu\in\mathcal{C}([0,T];\mathcal{P}_{c}(\mathbb{R}^{2d}))$ to the continuity
equation 1.7. In the sequel, we assume that
$\theta\in\mathcal{C}([0,T];\mathbb{R}^{m})$ so that the map
$t\mapsto\mathcal{F}(t,x,\theta_{t})$ is continuous on $[0,T]$, and that
$\mathcal{F}$ satisfies Assumption 1.

Under these working assumption we can further prove that the solution $\mu$ is
such that
$\partial_{t}\mu\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$.
Indeed for any $\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$, one has

$\displaystyle\left\lVert\partial_{t}\mu_{t}\,\right\rVert_{(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}}$
$\displaystyle=\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\partial_{t}\mu_{t},\varphi\rangle|$ (4.44)
$\displaystyle=\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(t,\cdot,\theta_{t})\mu_{t},\nabla_{x}\varphi\rangle|$
$\displaystyle\leq\left\lVert\mathcal{F}\,\right\rVert_{L^{\infty}(\textnormal{supp}(\mu_{t}))}\leq
C_{{\mathcal{F}}}(1+|R_{T}|).$ (4.45)

Additionally, it holds for any $s,t\in[0,T]$ that

$\displaystyle\left\lVert\partial_{t}\mu_{t}-\partial_{s}\mu_{s}\,\right\rVert_{(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}}$
$\displaystyle=\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\partial_{t}\mu_{t}-\partial_{s}\mu_{s},\varphi\rangle|$ (4.46)
$\displaystyle=\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(t,\cdot,\theta_{t})\mu_{t}-\mathcal{F}(s,\cdot,\theta_{s})\mu_{s},\nabla_{x}\varphi\rangle|$
$\displaystyle\leq\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}\Big{|}\big{\langle}(\mathcal{F}(t,\cdot,\theta_{t})-\mathcal{F}(s,\cdot,\theta_{s}))\mu_{t},\nabla_{x}\varphi\big{\rangle}\Big{|}$
(4.47) $\displaystyle\hskip
11.38092pt+\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla_{x}\varphi\rangle|$
(4.48) $\displaystyle\leq
C|t-s|+\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla_{x}\varphi\rangle|,$
(4.49)

Observe that by standard density results, there exists for every
$\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$ a sequence
$(\varphi^{n})\subset\mathcal{C}_{b}^{2}(\mathbb{R}^{2d})$ such that
$\left\lVert\varphi^{n}-\varphi\,\right\rVert_{\mathcal{C}^{1}_{b}(\mathbb{R}^{2d})}\rightarrow
0$ as $n\to+\infty$. Thus, one has that

$\displaystyle\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla_{x}\varphi\rangle|$
$\displaystyle\leq\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}\big{|}\big{\langle}\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),(\nabla_{x}\varphi-\nabla_{x}\varphi^{n})\big{\rangle}\big{|}+\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla_{x}\varphi^{n}\rangle|$
$\displaystyle\leq
C\left\lVert\varphi^{n}-\varphi\,\right\rVert_{\mathcal{C}^{1}_{b}(\mathbb{R}^{2d})}+\mbox{Lip}\big{(}{\mathcal{F}}(t,\cdot,\theta_{t})\cdot\nabla_{x}\varphi^{n}\big{)}W_{1}(\mu_{t},\mu_{s})$
(4.50) $\displaystyle\leq
C\left\lVert\varphi^{n}-\varphi\,\right\rVert_{\mathcal{C}^{1}_{b}(\mathbb{R}^{2d})}+C_{n}|t-s|,$
(4.51)

where we have used the Kantorovitch duality (2.6) and (A.7), which further
yields that

$\lim_{s\to
t}\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla_{x}\varphi\rangle|\leq\left\lVert\varphi^{n}-\varphi\,\right\rVert_{\mathcal{C}^{1}_{b}(\mathbb{R}^{2d})},$
(4.52)

for every $n\in\mathbb{N}$. Therefore letting $n\to+\infty$ in (4.52), we can
conclude

$\sup\limits_{\left\lVert\varphi\,\right\rVert_{\mathcal{C}_{b}^{1}}\leq
1}|\langle\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla_{x}\varphi\rangle|~{}\underset{s\to
t}{\longrightarrow}~{}0.$ (4.53)

This combined with (4.46) and the fact that
$t\mapsto{\mathcal{F}}(t,x,\theta_{t})\in\mathbb{R}^{d}$ is continuous implies
that
$\partial_{t}\mu\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$.
In the sequel, we will therefore consider trajectory-control pairs
$(\mu^{*},\theta^{*})\in\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})\times\mathcal{C}([0,T];\mathbb{R}^{m})$
solution of the optimal control problem (1.8), where we have used the notation
$\mu\in\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$
to represent that
$\mu\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$ and
$\partial_{t}\mu\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$.

#####  $\circ$ The setup of spaces and sets.

Let us start by defining the spaces

$V:=\widetilde{\mathcal{C}}([0,T];\mathcal{M}_{1,c}(\mathbb{R}^{2d}))\,\cap\,\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})\qquad\text{and}\qquad
Q:=\mathcal{C}([0,T];\mathbb{R}^{m}),$ (4.54)

where

$\displaystyle\widetilde{\mathcal{C}}([0,T];\mathcal{M}_{1,c}(\mathbb{R}^{2d})):=\bigg{\\{}\mu\in\mathcal{C}([0,T];\mathcal{M}_{1,c}(\mathbb{R}^{2d}))~{}\big{|}{}$
$\displaystyle\textnormal{supp}(\mu_{t})\subset S_{\mu}~{}\text{for all
$t\in[0,T]$}$ (4.55) $\displaystyle\text{where $S_{\mu}\subset\mathbb{R}^{d}$
is a compact set}\bigg{\\}},$

and fix

$E:=V\times
Q~{}=~{}\widetilde{\mathcal{C}}([0,T];\mathcal{M}_{1,c}(\mathbb{R}^{2d}))\,\cap\,\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})\times\mathcal{C}([0,T];\mathbb{R}^{m}).$
(4.56)

Clearly, $(\mu^{*},\theta^{*})\in E$ since
$\mathcal{P}_{c}(\mathbb{R}^{2d})\subset\mathcal{M}_{1,c}(\mathbb{R}^{2d})$.
We also observe that $E$ is a convex subset of the Banach space

$X:=U\times
Q=\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})\times\mathcal{C}([0,T];\mathbb{R}^{m}).$
(4.57)

Due to this embedding, we shall from now on endow
$\mathcal{M}_{1,c}(\mathbb{R}^{2d})$ with the weak$-^{*}$ topology of
$(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}$. In what follows, we use the
notation $U_{V}:=\mathbb{R}(V-V)$ as well as the identity

$U_{V}:=\widetilde{\mathcal{C}}([0,T];\mathcal{M}_{0,c}(\mathbb{R}^{2d}))\cap\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}).$
(4.58)

For $\nu\in V$, we shall define $U_{\nu}$ as the convex cone of directions

$U_{\nu}:=\mathbb{R}_{+}(V-\nu)\subset U_{V}\,,$ (4.59)

in keeping with the concepts introduced in Section 2. In fact, one can easily
check that $U_{\nu}=U_{V}$, since for any $\mu\in U_{V}$, one has
$\mu=\mu+\nu-\nu$ with $\mu+\nu\in V$. Next we introduce

$X_{E}:=U_{V}\times
Q~{}=~{}\widetilde{\mathcal{C}}([0,T];\mathcal{M}_{0,c}(\mathbb{R}^{2d}))\cap\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})\times\mathcal{C}([0,T];\mathbb{R}^{m}).$
(4.60)

that is seen as a convex subset of $X$. It follows from the definitions of $E$
and $X_{E}$ that $(\mu^{*},\theta^{*})+X_{E}\subset E$, which is compatible
with the assumptions of Theorem 4.5.

#####  $\circ$ The setup of maps.

For any $(\mu,\theta)\in E$, we denote the full cost functional of (1.8) by

$J(\mu,\theta):=\int_{\mathbb{R}^{2d}}\ell(x,y)\mathrm{d}\mu_{T}(x,y)+\lambda\int_{0}^{T}|\theta_{t}|^{2}\,{\rm
d}t,$ (4.61)

and observe that it is a map from $E$ into $\mathbb{R}_{+}$. We also introduce
the notation

$G(\mu,\theta):=-\partial_{t}\mu-\nabla_{x}\cdot(\mathcal{F}(t,x,\theta)\mu)\,.$
(4.62)

Seeing $G(\mu,\theta)$ as time-dependent quantity, it is easy to check that
$G(\mu,\theta)\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$
for $(\mu,\theta)\in E$, and that $\langle G(\mu,\theta)_{t},1\rangle=0$ for
all $t\in[0,T]$. Indeed for any
$\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$, it holds

$\displaystyle\left\lVert
G(\mu,\theta)_{t}-G(\mu,\theta)_{s}\,\right\rVert_{(\mathcal{C}_{b}^{1})^{\prime}}$
$\displaystyle=\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\left|\langle G(\mu,\theta)_{t}-G(\mu,\theta)_{s},\varphi\rangle\right|$
$\displaystyle=\left\lVert\partial_{t}\mu_{t}-\partial_{s}\mu_{s}\,\right\rVert_{(\mathcal{C}_{b}^{1})^{\prime}}+\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\big{|}\big{\langle}(\mathcal{F}(t,\cdot,\theta_{t})-\mathcal{F}(s,\cdot,\theta_{s}))\mu_{t},\nabla\varphi\big{\rangle}\big{|}$
$\displaystyle\hskip
12.80365pt+\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\big{|}\big{\langle}\mathcal{F}(s,\cdot,\theta_{s})(\mu_{t}-\mu_{s}),\nabla\varphi\big{\rangle}\big{|}.\,.$

By performing density arguments similar to those of (4.3.2)-(4.53), one has
that

$\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\left|\langle\mathcal{F}(s,x,\theta_{s})(\mu_{t}-\mu_{s}),\nabla\varphi\rangle\right|\leq
C\|\mu_{t}-\mu_{s}\|_{(\mathcal{C}^{1})^{\prime}}.$ (4.63)

This with together with the fact that
$\mu\in\mathcal{C}^{1}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$
and that $t\in[0,T]\mapsto\mathcal{F}(t,\cdot,\theta_{t})$ is continuous in
time yields
$G(\mu,\theta)\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$.
Observe now that for any
$\mu\in\widetilde{\mathcal{C}}([0,T];\mathcal{M}_{1,c}(\mathbb{R}^{2d}))$,
there exists some compact set $S_{\mu}\subset\mathbb{R}^{d}$ such that

$\textnormal{supp}(\mu_{t})\subset S_{\mu}\quad\mbox{ for all }t\in[0,T]\,.$
(4.64)

This implies that $G(\mu,\theta)$ is uniformly compactly supported in the
sense of distribution, namely $G:E\to Y_{0}$ with

$\displaystyle Y_{0}:$
$\displaystyle=\widetilde{\mathcal{C}}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}_{0,c})$
$\displaystyle=\bigg{\\{}g\in\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})~{}\big{|}~{}\langle
g_{t},1\rangle=0\mbox{ and }\textnormal{supp}(g_{t})\subset
S_{g}\Subset\mathbb{R}^{2d},~{}\forall t\in[0,T]\bigg{\\}}.$

This allows us to define the Banach space

$Y:=\overline{Y}_{0}=\overline{\widetilde{\mathcal{C}}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}_{0,c})},$
(4.65)

which is a closed subspace of the Banach space
$\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$.

Now let us verify that $G\in\mathcal{C}^{1}(E;Y)$ and
$J\in\mathcal{C}^{1}(E;\mathbb{R})$. For any $t\in[0,T]$, it holds that

$\displaystyle\left\lVert
G(\mu^{1},\theta^{1})_{t}-G(\mu^{2},\theta^{2})_{t}\,\right\rVert_{(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}}$
$\displaystyle=\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\left|\langle
G(\mu^{1},\theta^{1})_{t}-G(\mu^{2},\theta^{2})_{t},\varphi\rangle\right|$
$\displaystyle=\left\lVert\partial_{t}\mu_{t}^{1}-\partial_{t}\mu_{t}^{2}\,\right\rVert_{(\mathcal{C}_{b}^{1})^{\prime}}+\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\left|\langle\mathcal{F}(t,x,\theta_{t}^{1})(\mu_{t}^{1}-\mu_{t}^{2}),\nabla\varphi\rangle\right|$
$\displaystyle\hskip
12.80365pt+\sup\limits_{\|\varphi\|_{\mathcal{C}_{b}^{1}}\leq
1}\left|\langle(\mathcal{F}(t,x,\theta_{t}^{1})-\mathcal{F}(t,x,\theta_{t}^{2}))\mu_{t}^{2},\nabla\varphi\rangle\right|$
$\displaystyle\leq\left\lVert\partial_{t}\mu_{t}^{1}-\partial_{t}\mu_{t}^{2}\,\right\rVert_{(\mathcal{C}_{b}^{1})^{\prime}}+C\left\lVert\mu_{t}^{1}-\mu_{t}^{2}\,\right\rVert_{(\mathcal{C}_{b}^{1})^{\prime}}+C(R_{T},T)|\theta_{t}^{1}-\theta_{t}^{2}|\
$

where we have again used density arguments similar to that of (4.3.2)-(4.53).
Thus, we have proven that

$\displaystyle\left\lVert
G(\mu^{1},\theta^{1})-G(\mu^{2},\theta^{2})\,\right\rVert_{\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})}$
$\displaystyle\leq
C\left\lVert\mu^{1}-\mu^{2}\,\right\rVert_{\mathcal{C}^{1}([0,T];\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))}$
$\displaystyle\hskip
12.80365pt+C(R_{T},T)\left\lVert\theta_{1}-\theta_{2}\,\right\rVert_{\mathcal{C}([0,T])},$
(4.66)

which implies that $G\in\mathcal{C}(E;Y)$. Similarly we have

$\displaystyle|J(\mu^{1},\theta^{1})-J(\mu^{2},\theta^{2})|$
$\displaystyle\leq\left|\int_{\mathbb{R}^{2d}}\ell(x,y)\mathrm{d}(\mu_{T}^{1}-\mu_{T}^{2})(x,y)+\int_{0}^{T}(|\theta_{t}^{1}|^{2}-|\theta_{t}^{2}|^{2})\,{\rm
d}t\right|$ $\displaystyle\leq
C\left\lVert\mu_{T}^{1}-\mu_{T}^{2}\,\right\rVert_{(\mathcal{C}_{b}^{1})^{\prime}}+C\big{(}T,\left\lVert\theta_{1}\,\right\rVert_{\mathcal{C}([0,T]},\left\lVert\theta_{2}\,\right\rVert_{\mathcal{C}([0,T])}\big{)}\left\lVert\theta_{1}-\theta_{2}\,\right\rVert_{\mathcal{C}([0,T]},$

where we used the fact that $\mu_{T}^{1}$ and $\mu_{T}^{2}$ are compactly
supported. This in turn implies that $J\in\mathcal{C}(E;\mathbb{R})$.

Next, we use Lemma 2.1 to prove that both mappings are in fact
$\mathcal{C}^{1}$-smooth. It follows from the definition (2.18) of
G-derivative that for all $\mu\in V$, $\nu\in U_{\mu}=U_{V}$ and
$\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$, one has

$\displaystyle\langle\mathrm{d}_{\mu}G(\mu,\theta)(\nu),\varphi\rangle$
$\displaystyle=\bigg{\langle}\lim\limits_{\varepsilon\rightarrow
0^{+}}\frac{G(\mu+\varepsilon\nu,\theta)-G(\mu,\theta)}{\varepsilon},\varphi\bigg{\rangle}$
(4.67) $\displaystyle=\lim\limits_{\varepsilon\rightarrow 0^{+}}\frac{\langle
G(\mu+\varepsilon\nu,\theta),\varphi\rangle-\langle
G(\mu,\theta),\varphi\rangle}{\varepsilon}$
$\displaystyle=\langle-\partial_{t}\nu-\nabla_{x}\cdot(\mathcal{F}(t,x,\theta)\nu),\varphi\rangle<+\infty.$
(4.68)

Thus we have found a continuous operator $\mu\in V\mapsto
L_{\theta}(\mu)\in\mathcal{L}(U_{V},Y)$ such that
$L_{\theta}(\mu)(\nu):=-\partial_{t}\nu-\nabla_{x}\cdot(\mathcal{F}(t,x,\theta)\nu)=\mathrm{d}_{\mu}G(\mu,\theta)(\nu)$
for all $\mu\in V$ and $\nu\in U_{\mu}$. Applying Lemma 2.1 allows us to
conclude that $L_{\theta}(\mu)\in D_{\mu}G(\mu,\theta)$ and
$G(\cdot,\theta)\in\mathcal{C}^{1}(V;Y)$. Additionally, remark that the
standard Fréchet differential $G^{\prime}_{\theta}(\mu,\theta):Q\to Y$ with
respect to the control curve satisfies

$\langle
G^{\prime}_{\theta}(\mu,\theta)(\alpha),\varphi\rangle=\lim\limits_{\varepsilon\rightarrow
0^{+}}\frac{\langle G(\mu,\theta+\varepsilon\alpha),\varphi\rangle-\langle
G(\mu,\theta),\varphi\rangle}{\varepsilon}=\langle-\nabla_{x}\cdot(\nabla_{\theta}\mathcal{F}(t,x,\theta)\alpha\mu),\varphi\rangle<+\infty\,.$
(4.69)

for all $\alpha\in Q$. The continuity of
$\theta\in\mathbb{R}^{m}\mapsto\nabla_{\theta}{\mathcal{F}}(t,x,\theta)\in\mathbb{R}^{d}$
implies that $G(\mu,\cdot)\in\mathcal{C}^{1}(Q;Y)$ for every $\mu\in V$, and
thus $G\in\mathcal{C}^{1}(E;Y)$. Similarly, we have

$J^{\prime}_{\mu}(\mu,\theta)(\nu)=\int_{\mathbb{R}^{2d}}\ell(x,y)\mathrm{d}\nu_{T}\qquad\mbox{
and }\qquad
J^{\prime}_{\theta}(\mu,\theta)(\alpha)=\int_{0}^{T}2\lambda\theta_{t}\cdot\alpha_{t}\,{\rm
d}t\,,$ (4.70)

for all $\nu\in U_{\mu}=U_{V}$ and $\alpha\in Q$. It is then easy to check
that $J\in\mathcal{C}^{1}(E;\mathbb{R})$.

####  4.3.3 The mean-field PMP for continuous controls: a Lagrangian approach

We are now ready to present the derivation of the first order optimality
condition (4.6)-(4.8) in the class of continuous controls, by means of a
Lagrange multiplier rule tailored to the calculus for convex functions
introduced in Section 2.3.

######  Theorem 4.6 (Abstract Lagrange multiplier theorem).

Let $(\mu^{*},\theta^{*})\in E\subset X=U\times Q$ be a solution to the
optimal control problem (1.8). Then there exists $p^{\ast}\in Y^{\prime}$ such
that

$\displaystyle\langle
G^{\prime}_{\mu}(\mu^{\ast},\theta^{\ast})(\nu),p^{*}\rangle+J^{\prime}_{\mu}(\mu^{\ast},\theta^{\ast})(\nu)=0,\quad\mbox{
for all }\nu\in\overline{U}_{V},$ (4.71) $\displaystyle\langle
G^{\prime}_{\theta}(\mu^{\ast},\theta^{\ast})(\alpha),p^{*}\rangle+J^{\prime}_{\theta}(\mu^{\ast},\theta^{\ast})(\alpha)=0,\quad\mbox{
for all }\alpha\in Q\,.$ (4.72)

######  Remark 4.5.

The solution
$\psi^{*}=p^{*}\in\mathcal{C}^{1}([0,T];\mathcal{C}_{c}^{2}(\mathbb{R}^{2d}))$
constructed in Proposition 4.3 is in $Y^{\prime}$. This comes from the fact
that, for any $\eta\in
Y\subset\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$,
one has $\langle p^{*},\eta\rangle<+\infty$.

###### Proof.

In order to prove our set of optimality conditions, we will use Theorem 4.5
which application has already been prepared above. Indeed we have shown that
both the cost and constraint functionals are continuously $F$-differentiable,
and it follows directly from the definitions (4.56) and (4.60) that
$(\mu^{*},\theta^{*})+X_{E}\subset E$. Thus, there remains to prove that the
linear operator
$G^{\prime}(\mu^{\ast},\theta^{\ast}):\overline{X}_{E}=\overline{U_{V}}\times
Q\rightarrow Y$ is surjective. We split the proof of the surjectivity into two
steps below.

#####  $\bullet$ Surjectivity of the partial derivative
$G^{\prime}_{\mu}(\mu^{\ast},\theta^{\ast}):\overline{U_{V}}\rightarrow Y$.

We first want to show that for any given element

$\eta\in
Y:=\overline{\widetilde{\mathcal{C}}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}_{0,c})},$

there exists a $\nu\in\overline{U}_{V}$ such that

$G^{\prime}_{\mu}(\mu^{\ast},\theta^{\ast})(\nu)=\eta\,,$ (4.73)

which is understood in the sense of

$\langle
G^{\prime}_{\mu}(\mu_{t}^{\ast},\theta_{t}^{\ast})(\nu_{t}),\varphi\rangle=\langle\eta_{t},\varphi\rangle\qquad\mbox{
for all }\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})\,.$ (4.74)

To this end, it suffices to show that for a given
$(\mu^{\ast},\theta^{\ast},\eta)\in V\times Q\times Y$, there exists some
$\nu\in\overline{U}_{V}$ solution of the following transport equation

$\partial_{t}\nu_{t}+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta^{\ast}_{t})\nu_{t})=-\eta_{t}\,,$
(4.75)

with source term $(-\eta)$ and initial condition $\nu_{0}\in U_{\mu_{0}}$.
Notice that $(\mathcal{C}_{b}(\mathbb{R}^{2d}))^{\prime}_{0,c}$ is dense in
$(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}_{0,c}$, namely for any
$\eta\in
Y=\overline{\widetilde{\mathcal{C}}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime}_{0,c})}$,
there exists a sequence
$(\eta^{n})_{n\in\mathbb{N}}\subset\widetilde{\mathcal{C}}([0,T];(\mathcal{C}_{b}(\mathbb{R}^{2d}))^{\prime}_{0,c})$
such that for all $\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$, it holds

$\sup\limits_{t\in[0,T]}|\langle\eta_{t}^{n}-\eta_{t},\varphi\rangle|~{}\underset{n\to+\infty}{\longrightarrow}~{}0.$
(4.76)

In particular, observe that
$\sup\limits_{t\in[0,T],n\in\mathbb{N}}\|\eta_{t}^{n}\|_{(\mathcal{C}_{b}^{1})^{\prime}}<+\infty$
is uniformly bounded.

Since
$\eta_{t}^{n}\in(\mathcal{C}_{b}(\mathbb{R}^{2d}))^{\prime}_{0,c}\subset(\mathcal{C}_{0}(\mathbb{R}^{2d}))_{0,c}^{\prime}=\mathcal{M}_{0,c}(\mathbb{R}^{2d})$,
it then follows from [piccoli2019wasserstein, Theorem 1] that there exists a
unique measure solution $\mu^{1,n}\in V$ to the following transport equation

$\partial_{t}\mu_{t}^{1,n}+\nabla_{x}\cdot(\mathcal{F}(t,x,\theta^{\ast}_{t})\mu_{t}^{1,n})=-\eta_{t}^{n},\qquad\mu_{t}^{1,n}|_{t=0}=\mu_{0}^{1}\in\mathcal{P}_{c}(\mathbb{R}^{2d})\,,$
(4.77)

understood analogously to (2.11) in the sense of distribution, namely

$\displaystyle\int_{\mathbb{R}^{2d}}\varphi(x,y)\mathrm{d}\mu_{t_{2}}^{1,n}(x,y)-\int_{\mathbb{R}^{2d}}\varphi(x,y)\mathrm{d}\mu_{t_{1}}^{1,n}(x,y)$
$\displaystyle=\int_{t_{1}}^{t_{2}}\int_{\mathbb{R}^{2d}}\nabla_{x}\varphi(x,y)\cdot\mathcal{F}(s,x,\theta_{s}^{*})\,\mathrm{d}\mu_{s}^{1,n}(x,y)\,{\rm
d}s-\int_{t_{1}}^{t_{2}}\int_{\mathbb{R}^{2d}}\varphi(x,y)\,\mathrm{d}\eta_{s}^{n}(x,y)\,{\rm
d}s$

for all $\varphi\in\mathcal{C}_{b}^{1}(\mathbb{R}^{2d})$ and every
$t_{1},t_{2}\in[0,T]$. Indeed, we can build a solution to above as a limit of
a sequence of approximated solutions satisfying the following Euler-explicit-
type splitting scheme. Fix $k\in\mathbb{N}$, and define $\Delta
t=\frac{T}{2^{k}}$ and set $\mu_{0}^{1,n,(k)}=\mu_{0}$. Given
$\mu_{i\Delta_{t}}^{1,n,(k)}$ for $i\in\\{0,1,\cdots,2^{k}-1\\}$, we denote by
${\mathcal{F}}_{i\Delta t}=\mathcal{F}(i\Delta t,x,\theta_{i\Delta t}^{*})$
and set

$\mu_{t}^{1,n,(k)}=\Gamma_{t-i\Delta t}^{{\mathcal{F}}_{i\Delta
t}}\sharp\mu_{i\Delta t}^{1,n,(k)}-(t-i\Delta t)\eta_{i\Delta t}^{n},\quad
t\in[i\Delta t,(i+1)\Delta t]\,,$ (4.78)

where $\Gamma_{t-i\Delta t}^{{\mathcal{F}}_{i\Delta t}}\sharp\mu_{i\Delta
t}^{1,n,(k)}$ is the unique solution of the linear transport equation

$\displaystyle\begin{cases}&\partial_{t}f_{t}+\nabla\cdot({\mathcal{F}}_{i\Delta
t}f_{t})=0,\qquad t\in(i\Delta t,(i+1)\Delta t],\\\ &f_{i\Delta
t}=\mu_{i\Delta t}^{1,n,(k)}\,,\end{cases}$ (4.79)

which is is explicitly written as a pushforward through a characteristic flow.
From (4.78), we know the sequence $(\mu_{t}^{1,n,(k)})_{k\in\mathbb{N}}$ has
uniformly bounded support, since

$\textnormal{supp}(\mu_{t}^{1,n,(k)})\subset B(R_{T})\cup S_{\eta^{n}}$ (4.80)

where $\textnormal{supp}(\eta_{t}^{n})\subset
S_{\eta^{n}}\Subset\mathbb{R}^{2d}$ for all $t\in[0,T]$ and we denoted by
$B(R_{T})$ the support of solutions to the linear transport equation obtained
in (2.12). Intuitively, the support of $\mu_{t}^{1,n,(k)}$ is the union of the
support of the solution to the linear transport equation (4.79) and the
support of the source term. Similarly, it holds for $t\in[i\Delta
t,(i+1)\Delta t]$

$\displaystyle\|\mu_{t}^{1,n,(k)}\|_{(\mathcal{C}_{b}^{1})^{\prime}}\leq\|\Gamma_{t-i\Delta
t}^{{\mathcal{F}}_{i\Delta t}}\sharp\mu_{i\Delta
t}^{1,n,(k)}\|_{(\mathcal{C}_{b}^{1})^{\prime}}+\Delta t\|\eta_{i\Delta
t}^{n}\|_{(\mathcal{C}_{b}^{1})^{\prime}}\leq\|\mu_{i\Delta
t}^{1,n,(k)}\|_{(\mathcal{C}_{b}^{1})^{\prime}}+\Delta t\|\eta_{i\Delta
t}^{n}\|_{(\mathcal{C}_{b}^{1})^{\prime}}\,.$ (4.81)

This provides us with the following upper-bound

$\sup_{t\in[0,T]}\|\mu_{t}^{1,n,(k)}\|_{(\mathcal{C}_{b}^{1})^{\prime}}\leq\|\mu_{0}^{1}\|_{(\mathcal{C}_{b}^{1})^{\prime}}+T\sup_{t\in[0,T]}\|\eta_{t}^{n}\|_{(\mathcal{C}_{b}^{1})^{\prime}}<+\infty\,,$
(4.82)

which is uniform with respect to $n,k\in\mathbb{N}$. By letting $k\to+\infty$,
we recover the existence of a solution $\mu^{1,n}$ to (4.77) such that

$\sup_{t\in[0,T]}\mathbb{W}_{1}^{1,1}(\mu^{1,n},\mu_{t}^{1,n,(k)})~{}\underset{k\to+\infty}{\longrightarrow}~{}0.$
(4.83)

Recall that the generalized Wasserstein metric introduced in
[piccoli2019wasserstein] is equivalent to the bounded-Lipschitz norm
$\|\cdot\|_{BL}$, so that the limit curves $(\mu^{1,n})_{n\in\mathbb{N}}$
satisfy

$\textnormal{supp}(\mu_{t}^{1,n})\subset B(R_{T})\cup
S_{\eta^{n}}\quad\text{and}\quad\|\mu_{t}^{1,n}\|_{(\mathcal{C}_{b}^{1})^{\prime}}<+\infty$
(4.84)

for all $t\in[0,T]$. This in turn implies that the sequence
$(\mu_{t}^{1,n})_{n\in\mathbb{N}}$ is uniformly equi-bounded in
$\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$.
According to [piccoli2019wasserstein, Theorem 1], it follows that each curve
$t\in[0,T]\mapsto\mu^{1,n}$ is Lipschitz continuous with respect to the
$\|\cdot\|_{BL}$-norm, and thus it is uniformly equi-continuous with respect
to the $(\mathcal{C}_{b}^{1})^{\prime}$-norm. By a direct application of the
Arzelà-Ascoli theorem, there exists a subsequence of
$(\mu^{1,n})_{n\in\mathbb{N}}$ that converges uniformly in
$\mathcal{C}([0,T];(\mathcal{C}_{b}^{1}(\mathbb{R}^{2d}))^{\prime})$ to some