# Consistency of sample-based stationary points for infinite-dimensional
stochastic optimization

Johannes Milz H. Milton Stewart School of Industrial and Systems Engineering,
Georgia Institute of Technology, Atlanta, GA 30332, USA
<EMAIL_ADDRESS>
(June 19, 2023)

###### Abstract

We consider stochastic optimization problems with possibly nonsmooth
integrands posed in Banach spaces and approximate these stochastic programs
via a sample-based approaches. We establish the consistency of approximate
Clarke stationary points of the sample-based approximations. Our framework is
applied to risk-averse semilinear PDE-constrained optimization using the
average value-at-risk and to risk-neutral bilinear PDE-constrained
optimization.

###### keywords:

stochastic programming, sample average approximation, optimization under
uncertainty, PDE-constrained optimization, uncertainty quantification,
bilinear optimal control

AMS subject classifications. 65C05, 90C15, 35R60, 90C48, 90C30, 60H25, 49M41,
35Q93

##  1 Introduction

Infinite-dimensional optimization problems arise in a plethora of research
fields such as dynamic programming [56], statistical estimation [26], feedback
stabilization of dynamical systems [54], and optimization problems governed by
partial differential equations (PDEs) [49]. PDE-constrained optimization is an
active research field with a focus on modeling, analyzing and solving complex
optimization problems with PDE constraints. For example, numerous applications
in the field of renewable and sustainable energy yield challenging PDE-
constrained optimization problems, such as wind plant layout optimization
[44], wind turbine blade planform design [1], and tidal stream array
optimization [28]. Parameters in PDEs may be uncertain, such as diffusion
coefficients and boundary conditions, and as such can have a significant
influence on the simulation output. Using parameter-dependent PDEs for
decision making naturally leads to decision making under uncertainty. A common
approach in optimization under uncertainty models uncertain parameters as
random variables with known probability distribution. Using this stochastic
programming approach, we can formulate an optimization problem having
solutions performing best on average; its objective function is the expected
value of a parameterized cost function, the integrand. Such optimization
problems are nowadays often referred to as risk-neutral problems to
distinguish them from risk-averse formulations. Since high-dimensional
integrals cannot be accurately evaluated and solving PDEs requires numerical
computations, we approximate the expectations using a Monte Carlo sample-based
approximation, the sample average approximation (SAA) approach [45, 72, 74],
and other schemes approximating the true probability distribution via weakly
convergent probability distributions. In this manuscript, we demonstrate the
consistency of Clarke-(C-)stationary points of the approximated problems
towards the risk-neutral problem’s set of C-stationary points.

Motivated by infinite-dimensional optimization problems governed by complex
physical systems under uncertainty, we consider the risk-neutral composite
optimization problem

$\displaystyle\min_{x\in
X}\,\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}(\xi)+\psi(x),$ (1.1)

where $X$ is a reflexive Banach space, $\psi\colon X\to(-\infty,\infty]$ is a
proper, convex, lower semicontinuous function having the so-called Kadec
property, $\mathbb{P}$ is a probability measure on the complete, separable
metric space $\Xi$, and $F_{\xi}$ is a locally Lipschitz continuous integrand
with integrable Lipschitz constants.

In this manuscript, we consider two types of approximations of the
expectations in (1.1). While our main focus is on analyzing the SAA approach
[45, 72], we also consider approximations of $\mathbb{P}$ via weakly
converging probability measures [56, 22]. To introduce, the SAA approach, let
$\xi^{1}$, $\xi^{2},\ldots$ be independent identically distributed
$\Xi$-valued random elements such that each $\xi^{i}$ has distribution
$\mathbb{P}$. The SAA problem with sample size $N\in\mathbb{N}$ corresponding
to (1.1) is given by

$\displaystyle\min_{x\in X}\,\frac{1}{N}\sum_{i=1}^{N}F_{\xi^{i}}(x)+\psi(x).$
(1.2)

The principle contributions of this manuscript are twofold.

  1. (i)

We demonstrate the consistency of approximate C-stationary points of the SAA
problem (1.2) for a class of risk-neutral nonlinear optimization problems
(1.1) by adapting and extending a framework developed in [59], where the
consistency of SAA optimal values and SAA solutions has been demonstrated.
More specifically, our analysis relies on the identity

$\displaystyle\partial_{C}F_{\xi}(x)=BM_{\xi}(x)\quad\text{for
all}\quad(x,\xi)\in X\times\Xi,$ (1.3)

where $\partial_{C}F_{\xi}(x)$ is Clarke’s subdifferential of $F_{\xi}$ at
$x$, $V$ is a Banach space, $B\colon V\to X^{*}$ is linear and compact, and
$M_{\xi}\colon X\rightrightarrows V$ is a multifunction for each $\xi\in\Xi$.
Moreover, using proof techniques developed in [62], we establish consistency
statements for the significantly larger class of functions $\psi$ than those
considered in [59], so-called functions having the Kadec property [9].

  2. (ii)

We apply our framework to risk-averse semilinear PDE-constrained optimization
problems using the average value-at-risk and to steady-state elliptic bilinear
optimal control problems under uncertainty. While the literature on risk-
neutral and risk-averse semilinear PDE-constrained optimization is extensive
[24, 48, 50, 64], bilinear PDE-constrained problems under uncertainty have not
been analyzed, but serve as yet another class of nonconvex problems and arise
in many applications.

The consistency of SAA solutions and SAA stationary points of nonlinear
possibly infinite-dimensional stochastic programs are typically based on some
form of the feasible set’s compactness. In particular, the consistency results
for SAA stationary points of infinite-dimensional stochastic programs
established in [5, 78] require the feasible set’s compactness. However,
compact sets in infinite dimensions may regarded as rare phenomena, given the
fact that closed unit balls in Banach spaces are compact if and only if the
spaces are finite-dimensional. This fact complicates the consistency analysis
for nonconvex stochastic programs. For strongly convex risk-neutral PDE-
constrained optimization, the SAA approach has been analyzed in [38, 58, 71,
60, 61] without requiring the compactness of the feasible set.

Contributions to analyzing the SAA approach for optimization problems governed
by nonlinear operator equations with random inputs are relatively recent. The
SAA objective function’s almost sure epiconvergence and the SAA stationary
points’ weak consistency is analyzed in [68] for optimization problems
governed by ordinary differential equations with random inputs. Utilizing
epiconvergence, performance guarantees of optimal values of PDE-constrained
optimization problems under uncertainty are provided in [16] with respect to
sample average approximations and control and state space discretizations, for
example. The asymptotic consistency of SAA optimal values and SAA solutions to
risk-averse PDE-constrained optimization problems has recently been
demonstrated in [62] using (Mosco-)epiconvergence, and sample size estimates
for risk-neutral semilinear PDE-constrained problems are derived in [64].

We use the compact operator $B$ in (1.3) to mathematically model problem
structure typically found in PDE-constrained optimization problems. Related
problem characteristics are used for establishing error estimates for finite-
dimensional approximations to PDE-constrained optimization problems [13, 20,
52] and for algorithmic design and analysis in function space [33, 36, 35,
80], for example.

The analysis of bilinear PDE-constrained optimization problems is generally
more challenging than those of their linear counterparts, as bilinear PDEs may
lack solutions for each point in the control space and bilinear control
problems are generally nonconvex. Bilinear control problems can be governed by
steady-state elliptic PDEs [52, 81], convection-diffusion equations [11], and
advection-reaction-diffusion equations [27], for instance. A substantial body
of literature is available on the analysis, algorithmic design, and
applications of deterministic bilinear control problems. We refer the reader
to [14, 27, 31] for the analysis of deterministic bilinear control problems.
Finite element error estimates are established in [14, 23, 52, 82, 84] and
algorithms are designed in [11, 27, 41, 43, 53, 81]. Bilinear control problems
arise in medical image analysis [57], damping design [75], groundwater
remediation system design [30], optical flow [40], PDE-constrained regression
problems [65], and chemorepulsion production [31], for example.

## Outline

We introduce notation and terminology in section 2. Section 3 establishes
consistency of C-stationary points of optimization problems obtained through
approximations of infinite-dimensional nonconvex optimization posed in
reflexive Banach spaces. This basic result is used to establish consistency of
SAA C-stationary points in section 4 and of C-stationary points of approximate
stochastic programs defined by weakly convergent probability measures in
section 5. In section 6, we apply the framework developed in section 4 to
risk-averse semilinear PDE-constrained optimization using the average value-
at-risk. Section 7 discusses the application of our theory to a risk-neutral
bilinear PDE-constrained optimization problem. We summarize our contributions
and discuss open research questions in section 8.

##  2 Notation and preliminaries

Metric spaces are defined over the real numbers and equipped with their Borel
sigma-algebra if not specified otherwise. For a Hilbert space $H$, we denote
by $(\cdot,\cdot)_{H}$ its inner product. Let $X_{1}$ and $X_{2}$ be Banach
spaces. The space of linear, bounded operators from $X_{1}$ to $X_{2}$ is
denoted by $\mathscr{L}(X_{1},X_{2})$. An operator
$A\in\mathscr{L}(X_{1},X_{2})$ is compact if the image $A(W)$ is precompact in
$X_{2}$ for each bounded set $W\subset X_{1}$. The operator
$A^{*}\in\mathscr{L}(X_{2}^{*},X_{1}^{*})$ is the adjoint operator of
$A\in\mathscr{L}(X_{1},X_{2})$. We denote by $\mathrm{D}f$ the Fréchet
derivative of $f$ and by $\mathrm{D}_{y}f$ or $f_{y}$ the Fréchet derivative
with respect to $y$. The dual to a Banach space $\Lambda$ is $\Lambda^{*}$ and
we use $\langle\cdot,\cdot\rangle_{{\Lambda}^{*}\\!,\Lambda}$ to denote the
dual pairing between $\Lambda^{*}$ and $\Lambda$. Let $X_{0}\subset X$ be an
open, nonempty subset of a Banach space $X$. We denote Clarke’s generalized
directional derivative of a locally Lipschitz continuous mapping $f\colon
X_{0}\to$ at $x\in X_{0}$ in the direction $h\in X$ by $f^{\circ}(x;h)$ and
Clarke’s subdifferential at $x$ by $\partial_{C}f(x)\subset X^{*}$ [18, pp. 25
and 27]. We refer the reader to [18, p. 39] for the definition of Carke
regularity of a function $f$. We use $\partial f$ to denote the usual convex
subdifferential of a function $f$. We define the distance
$\mathrm{dist}({v,\Upsilon})$ from $v\in\Lambda\subset X$ to $\Upsilon\subset
X$ by

$\displaystyle\mathrm{dist}({v,\Upsilon})=\inf_{w\in\Upsilon}\,\|v-w\|_{X}\quad\text{if}\quad\Upsilon\neq\emptyset,\quad\text{and}\quad\mathrm{dist}({v,\Upsilon})=\infty\quad\text{otherwise}.$

and the deviation $\mathbb{D}({\Lambda,\Upsilon})$ between the sets $\Lambda$
and $\Upsilon$ by
$\mathbb{D}({\Lambda,\Upsilon})=\sup_{v\in\Lambda}\,\mathrm{dist}({v,\Upsilon})$.
Here $\|\cdot\|_{X}$ is a norm of $X$. If $\phi\colon X\to(-\infty,\infty]$,
then $\mathrm{dom}(\phi)=\\{x\in X\colon\phi(x)<\infty\\}$ denotes its domain.
Following [9, p. 147] and [10, p. 306], we say that a proper function
$\phi\colon X\to(-\infty,\infty]$ has the Kadec property if for each
$\bar{x}\in\mathrm{dom}(\phi)$, we have $x_{k}\to\bar{x}$ whenever
$\phi(x_{k})\to\phi(\bar{x})$, $(x_{k})\subset\mathrm{dom}(\psi)$, and
$x_{k}\rightharpoonup\bar{x}$. Here $\to$ denotes strong convergence and
$\rightharpoonup$ denotes weak convergence. For $x\in X$ and $\varepsilon\geq
0$, we denote by $\mathbb{B}_{X}(x;\varepsilon)$ the open ball about $x$ in
$X$ with radius $\varepsilon$ and by
$\overline{\mathbb{B}}_{X}(x;\varepsilon)$ its closure in $X$.

The following notation is used in sections 7 and 6. Let $D\subset^{d}$ be a
bounded domain. The space $C^{0,1}(\bar{D})$ is the set of Lipschitz
continuous functions on $\bar{D}$, $L^{p}(D)$ ($1\leq p\leq\infty$) and
$H^{1}(D)$, $H^{2}(D)$, and $H_{0}^{1}(D)$ are the “usual” Lebesgue and
Sobolev spaces, respectively. The space $H_{0}^{1}(D)$ is equipped with the
norm $\|v\|_{H_{0}^{1}(D)}=\|\nabla v\|_{L^{2}(D)^{d}}$ and the dual to
$H_{0}^{1}(D)$ is denoted by $H^{-1}(D)$. Moreover, let $C_{D}\in(0,\infty)$
be Friedrichs’ constant of $D$. If $X$ is a reflexive Banach space, then we
identify $(X^{*})^{*}$ with $X$ and write $(X^{*})^{*}=X$. Moreover, we
identify $L^{2}(D)^{*}$ with $L^{2}(D)$ and write $L^{2}(D)^{*}=L^{2}(D)$. We
denote by $\iota_{0}\colon H_{0}^{1}(D)\to L^{2}(D)$ the embedding operator of
the compact embedding $H_{0}^{1}(D)\xhookrightarrow{}L^{2}(D)$ and by
$\iota_{1}\colon H^{1}(D)\to L^{2}(D)$ that of
$H^{1}(D)\xhookrightarrow{}L^{2}(D)$. We introduce further application-
specific notation in section 7.

##  3 Consistency of C-stationary points

We consider the optimization problem

$\displaystyle\min_{x\in\mathrm{dom}(\psi)}\,f(x)+\psi(x)$ (3.1)

and its approximations

$\displaystyle\min_{x\in\mathrm{dom}(\psi)}\,f_{k}(x)+\psi(x).$ (3.2)

We explicitly add $\mathrm{dom}(\psi)$ as a constraint sets in (3.1) and
(3.2), as $f$ and $f_{k}$ may only be finite-valued on an open neighborhood of
$\mathrm{dom}(\psi)$. We provide conditions sufficient for the asymptotic
consistency of C-stationary points of the approximated problems (3.2) to those
of “true” problem (3.1). Our analysis is inspired by those in [72, 74, 17, 62]
and based on technical assumptions used in [59]. Using these consistency
results, we establish asymptotic consistency of C-stationary points of
infinite-dimensional stochastic programs in sections 4 and 5.

######  Assumption 3.1.

  1. 1.

The spaces $X$ and $V$ are Banach spaces, $X$ is reflexive, and $X_{0}\subset
X$ is nonempty and open.

  2. 2.

The function $\psi\colon X\to(-\infty,\infty]$ is proper, convex, and lower
semicontinuous. Moreover, $\psi$ has the Kadec property and
$\mathrm{dom}(\psi)\subset X_{0}$.

  3. 3.

The functions $f\colon X_{0}\to$ and $f_{k}\colon X_{0}\to$, $k\in\mathbb{N}$,
are locally Lipschitz continuous.

  4. 4.

The operator $B\in\mathscr{L}(V,X^{*})$ is compact,
$M_{k}\colon\mathrm{dom}(\psi)\rightrightarrows V$ is a set-valued mapping,
and $\partial_{C}f_{k}(x)=BM_{k}(x)$ for all $x\in\mathrm{dom}(\psi)$ and
$k\in\mathbb{N}$.

  5. 5.

For each $\bar{x}\in\mathrm{dom}(\psi)$, each sequence
$(x_{k})\subset\mathrm{dom}(\psi)$ with $x_{k}\rightharpoonup\bar{x}$, and
each sequence $(v_{k})$ with $v_{k}\in M_{k}(x_{k})$ for all $k\in\mathbb{N}$,
$\limsup_{k\to\infty}\,\|v_{k}\|_{V}<\infty$.

  6. 6.

For each $h\in X$, each $\bar{x}\in\mathrm{dom}(\psi)$, and each sequence
$(x_{k})\subset\mathrm{dom}(\psi)$ with $x_{k}\to\bar{x}$, it holds that
$\limsup_{k\to\infty}f_{k}^{\circ}(x_{k};h)\leq f^{\circ}(\bar{x};h)$.

Under 3.1.1, 3.1.3, and 3.1.2, Clarke’s subdifferential $\partial_{C}f_{k}(x)$
for each $x\in X_{0}$ is nonempty, convex, bounded, and weakly∗-compact [18,
Prop. 2.1.2]. Hence the set-valued mapping $M_{k}$ in 3.1.4 is nonempty-
valued. If $M_{k}$ has bounded images, then 3.1.4 ensures that
$\partial_{C}f_{k}(x)$ is precompact-valued. For Monte Carlo sample-based
approximations of stochastic programs, we verify 3.1.6 using an epigraphical
law of large numbers [3], basic properties of Clarke’s generalized directional
derivative, and by imposing Clarke regularity of integrands studied in section
4.

3.1.4 is satisfied for large problem classes as demonstrated next.

######  Example 3.2.

Let 3.1.1, 3.1.2, and 3.1.3 hold. If $W$ is a Banach space,
$\iota\in\mathscr{L}(X,W)$ is compact, $F_{k}\colon W\to$ is locally Lipschitz
continuous and Clarke regular, and $f_{k}\colon X\to$ is defined by
$f_{k}(x)=F_{k}(\iota x)$, then
$\partial_{C}f_{k}(x)=\iota^{*}\partial_{C}F_{k}(\iota x)$ [18, Thm. 2.3.10],
and we may choose $B=\iota^{*}$ and $M_{k}(x)=\partial_{C}F_{k}(\iota x)$ to
satisfy 3.1.4. Composite objective functions with compact linear operators
arise in nonsmooth optimal control [32, 33, 35, 76], for example.

Next we discuss 3.1.5 and 3.1.4 when $X=^{n}$.

######  Remark 3.3.

Let 3.1.1, 3.1.2, and 3.1.3 hold. Let $X=^{n}$ (equipped with the Euclidean
norm $\|\cdot\|_{2}$) and let $X^{*}=V$. We identify the dual to n with n. Let
$B\colon^{n}\to^{n}$ be the identity mapping. Then 3.1.4 holds true with
$M_{k}=\partial_{C}f_{k}$. Next we discuss 3.1.5. Let
$\bar{x}\in\mathrm{dom}(\psi)$ and let $L_{k}\geq 0$ be Lipschitz constant of
$f_{k}$ on $\mathbb{B}_{X}(\bar{x};\varepsilon)$, where
$\varepsilon=\varepsilon(\bar{x})>0$. If $\limsup_{k\to\infty}\,L_{k}<\infty$,
then 3.1.5 holds true. Let us verify this assertion. Let
$(x_{k})\subset\mathrm{dom}(\psi)$ with $x_{k}\rightharpoonup\bar{x}$, and let
$(v_{k})$ with $v_{k}\in M_{k}(x_{k})$ for all $k\in\mathbb{N}$. Then
$x_{k}\to\bar{x}$ and there exists $K=K(\bar{x})$ such that
$x_{k}\in\mathbb{B}_{X}(\bar{x};\varepsilon)$ for all $k\geq K$. Since
$\partial_{C}f_{k}(x_{k})=M_{k}(x_{k})$, [18, Prop. 2.1.2] ensures
$\|v_{k}\|_{2}\leq L_{k}$ for all $k\geq K$. Hence
$\limsup_{k\to\infty}\,\|v_{k}\|_{2}<\infty$.

Theorem 3.4 establishes consistency of approximate C-stationary points. Let
$\mathcal{C}$ be the set of C-stationary points of (3.1), that is,
$\mathcal{C}$ is the set of all points $x\in\mathrm{dom}(\psi)$ with
$0\in\partial_{C}f(x)+\partial\psi(x)$. Let $\varepsilon\geq 0$ and let
$\mathcal{C}_{k}^{\varepsilon}$ be the set of all points
$x\in\mathrm{dom}(\psi)$ with
$0\in\bigcup_{y\in\overline{\mathbb{B}}_{X}(x;\varepsilon)\cap\mathrm{dom}(\psi)}\partial_{C}f_{k}(y)+\partial\psi(x)$
(cf. [74, eqns. (3.2) and (4.2)]). Each point in
$\mathcal{C}_{k}^{\varepsilon}$ is referred to as an approximate C-stationary
point of (3.2).

######  Theorem 3.4.

Let Assumption 3.1 hold and let $(\varepsilon_{k})\subset[0,\infty)$ satisfy
$\varepsilon_{k}\to 0$ as $k\to\infty$. Moreover, let
$\mathcal{C}_{k}^{\varepsilon_{k}}$ be nonempty for each $k\in\mathbb{N}$.

  1. (i)

If $(x_{k})$ is a bounded sequence with
$x_{k}\in\mathcal{C}_{k}^{\varepsilon_{k}}$ for each $k\in\mathbb{N}$, then
$\mathcal{C}$ is nonempty and $\mathrm{dist}({x_{k},\mathcal{C}})\to 0$ as
$k\to\infty$.

  2. (ii)

If there exists a bounded set $\mathscr{C}\subset X$ with
$\mathcal{C}_{k}^{\varepsilon_{k}}\subset\mathscr{C}$ for all sufficiently
large $k\in\mathbb{N}$, then
$\mathbb{D}({\mathcal{C}_{k}^{\varepsilon_{k}},\mathcal{C}})\to 0$ as
$k\to\infty$.

###### Proof.

3.1.3 ensures that the Clarke subdifferentials of $f$ and $f_{k}$ are well-
defined [18, Prop. 2.1.2].

(i) Let $(\mathrm{dist}({x_{k},\mathcal{C}}))_{K_{0}}$ be a subsequence of
$(\mathrm{dist}({x_{k},\mathcal{C}}))\subset[0,\infty]$. Since
$x_{k}\in\mathcal{\mathcal{C}}_{k}^{\varepsilon_{k}}$, there exists
$y_{k}\in\mathrm{dom}(\psi)$ with $\|x_{k}-y_{k}\|_{X}\leq\varepsilon_{k}$ and
$g_{k}\in\partial_{C}f_{k}(y_{k})$ with

$\displaystyle\langle
g_{k},x-x_{k}\rangle_{{X}^{*}\\!,X}+\psi(x)\geq\psi(x_{k})\quad\text{for
all}\quad x\in X.$ (3.3)

The boundedness of $(x_{k})$ implies that of $(x_{k})_{K_{0}}$. Combined with
the fact that $X$ is reflexive (see 3.1.1) and that $\mathrm{dom}(\psi)$ is
weakly sequentially closed (see 3.1.2 and [7, Thm. 2.23 (ii)]), we find that
$(x_{k})_{K_{0}}$ has a subsequence $(x_{k})_{K_{1}}$ with
$x_{k}\rightharpoonup\bar{x}\in\mathrm{dom}(\psi)$ as $K_{1}\ni k\to\infty$.
Since $\|x_{k}-y_{k}\|_{X}\leq\varepsilon_{k}$ and $\varepsilon_{k}\to 0$, we
find that $y_{k}\rightharpoonup\bar{x}$ as $K_{1}\ni k\to\infty$. Since
$g_{k}\in\partial_{C}f_{k}(x_{k})$, 3.1.4 ensures $g_{k}=Bv_{k}$ for some
$v_{k}\in M_{k}(y_{k})$. 3.1.5 implies that $(v_{K})_{K_{1}}$ is bounded.
Combined with the compactness of $B$ (see 3.1.4), we obtain that
$(g_{k})_{K_{1}}\subset X^{*}$ has a further subsequence $(g_{k})_{K_{2}}$
with $g_{k}\to\bar{g}$ as $K_{2}\ni k\to\infty$ [51, Thm. 8.1-5]. Hence
$\langle
g_{k},x-x_{k}\rangle_{{X}^{*}\\!,X}\to\langle\bar{g},x-\bar{x}\rangle_{{X}^{*}\\!,X}$
as $K_{2}\ni k\to\infty$ [7, Thm. 2.23 (iv)]. 3.1.2 implies that $\psi$ is
weakly lower semicontinuous. Hence $\liminf_{K_{2}\ni
k\to\infty}\,\psi(x_{k})\geq\psi(\bar{x})$. Putting together the pieces and
using (3.3), we have

$\displaystyle\langle\bar{g},x-\bar{x}\rangle_{{X}^{*}\\!,X}+\psi(x)\geq\psi(\bar{x})\quad\text{for
all}\quad x\in X.$ (3.4)

Using (3.3) once more, we obtain $\psi(\bar{x})=\lim_{K_{2}\ni
k\to\infty}\langle
g_{k},\bar{x}-x_{k}\rangle_{{X}^{*}\\!,X}+\psi(\bar{x})\geq\limsup_{K_{2}\ni
k\to\infty}\psi(x_{k})$. Hence $\psi(x_{k})\to\psi(\bar{x})$ as $K_{2}\ni
k\to\infty$. Since $\psi$ is Kadec according to 3.1.2, $x_{k}\to\bar{x}$ as
$K_{2}\ni k\to\infty$. Since $\|x_{k}-y_{k}\|_{X}\leq\varepsilon_{k}$, we also
have $y_{k}\to\bar{x}$ as $K_{2}\ni k\to\infty$.

We define $\widetilde{x}_{k}=y_{k}$ if $k\in K_{2}$ and
$\widetilde{x}_{k}=\bar{x}$ otherwise. We have $\widetilde{x}_{k}\to\bar{x}$
as $k\to\infty$. Fix $h\in X$. Since $g_{k}\in\partial_{C}f_{k}(y_{k})$,
$f_{k}^{\circ}(y_{k};h)\geq\langle g_{k},h\rangle_{{X}^{*}\\!,X}$ [18, Prop.
2.1.5]. Combined with 3.1.6, we have

$\displaystyle\langle\bar{g},h\rangle_{{X}^{*}\\!,X}=\lim_{K_{2}\ni
k\to\infty}\,\langle g_{k},h\rangle_{{X}^{*}\\!,X}\leq\limsup_{K_{2}\ni
k\to\infty}\,f_{k}^{\circ}(y_{k};h)\leq\limsup_{k\to\infty}\,f_{k}^{\circ}(\widetilde{x}_{k};h)\leq
f^{\circ}(\bar{x};h).$

Hence $\bar{g}\in\partial_{C}f(\bar{x})$ [18, Prop. 2.1.5]. Now (3.4) ensures
$0\in\partial_{C}f(\bar{x})+\partial\psi(\bar{x})$, that is,
$\bar{x}\in\mathcal{C}$. Hence
$\mathrm{dist}({x_{k},\mathcal{C}})\leq\|x_{k}-\bar{x}\|_{X}\to 0$ as
$K_{2}\ni k\to 0$, and $(\mathrm{dist}({x_{k},\mathcal{C}}))$ is bounded. Our
derivations also show that each subsequence of
$(\mathrm{dist}({x_{k},\mathcal{C}}))$ has a further subsequence converging to
zero. Hence $\mathrm{dist}({x_{k},\mathcal{C}})\to 0$ as $k\to\infty$.

(ii) Since $\mathcal{C}_{k}^{\varepsilon_{k}}\subset\mathscr{C}$, and
$\mathcal{C}_{k}^{\varepsilon_{k}}$ and $\mathcal{C}$ are nonempty, and
$\mathscr{C}$ is bounded, we have
$\mathbb{D}({\mathcal{C}_{k}^{\varepsilon_{k}},\mathcal{C}})\leq\mathbb{D}({\mathscr{C},\mathcal{C}})<\infty$
for all sufficiently large $k\in\mathbb{N}$. Hence for all sufficiently large
$k\in\mathbb{N}$, there exists $x_{k}\in\mathcal{C}_{k}^{\varepsilon_{k}}$
with
$\mathbb{D}({\mathcal{C}_{k}^{\varepsilon_{k}},\mathcal{C}})\leq\mathrm{dist}({x_{k},\mathcal{C}})+1/k$.
Using part (i), we have $\mathrm{dist}({x_{k},\mathcal{C}})\to 0$ as
$k\to\infty$. Hence
$\mathbb{D}({\mathcal{C}_{k}^{\varepsilon_{k}},\mathcal{C}})\to 0$ as
$k\to\infty$. ∎

##  4 Empirical approximations via Monte Carlo sampling

Let $\Xi$ be a complete, separable metric space, and let
$(\Xi,\mathcal{A},\mathbb{P})$ be a complete probability space. Let $\xi^{1}$,
$\xi^{2},\ldots$ be independent identically distributed $\Xi$-valued random
elements defined on a complete probability space $(\Omega,\mathcal{F},P)$ such
that each $\xi^{i}$ has distribution $\mathbb{P}$.

We consider the risk-neutral optimization problem

$\displaystyle\min_{x\in\mathrm{dom}(\psi)}\,\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}(\xi)+\psi(x)$
(4.1)

and its SAA problem

$\displaystyle\min_{x\in\mathrm{dom}(\psi)}\,\frac{1}{N}\sum_{i=1}^{N}F_{\xi^{i}}(x)+\psi(x).$
(4.2)

We study the almost sure convergence of SAA C-stationary points of (4.2) to
the C-stationary point’s set of (4.1). We analyze the consistency under
conditions related to those in Assumption 3.1.

######  Assumption 4.1.

  1. 1.

The spaces $X$ and $V$ are separable Banach spaces, $X$ is reflexive, and
$X_{0}\subset X$ is nonempty and open.

  2. 2.

The function $\psi\colon X\to(-\infty,\infty]$ is proper, convex, and lower
semicontinuous. Moreover, $\psi$ has the Kadec property and
$\mathrm{dom}(\psi)\subset X_{0}$.

  3. 3.

For each $\bar{x}\in\mathrm{dom}(\psi)$, there exists an open neighborhood
$\mathcal{V}_{\bar{x}}\subset X_{0}$ of $\bar{x}$ and a
$\mathbb{P}$-integrable random variable $L_{\bar{x}}\colon\Xi\to[0,\infty)$
such that for each $\xi\in\Xi$, $F_{\xi}\colon X_{0}\to$ is Lipschitz
continuous on $\mathcal{V}_{\bar{x}}$ with Lipschitz constant
$L_{\bar{x}}(\xi)$.

  4. 4.

For each $\xi\in\Xi$, $F_{\xi}\colon X_{0}\to$ is Clarke regular.

  5. 5.

For each $x\in X_{0}$, $\Xi\ni\xi\mapsto F_{\xi}(x)\in$ is measurable and
$\mathbb{P}$-integrable.

  6. 6.

The operator $B\in\mathscr{L}(V,X^{*})$ is compact,
$M_{\xi}\colon\mathrm{dom}(\psi)\rightrightarrows V$ is a set-valued map with
nonempty images for each $\xi\in\Xi$, and $\partial_{C}F_{\xi}(x)=BM_{\xi}(x)$
for each $(x,\xi)\in\mathrm{dom}(\psi)\times\Xi$.

  7. 7.

There exists $\Omega_{0}\subset\Omega$ with $\Omega_{0}\in\mathcal{F}$ and
$P(\Omega_{0})=1$ such that for each $\bar{x}\in\mathrm{dom}(\psi)$, each
sequence $(x_{N})\subset\mathrm{dom}(\psi)$ with
$x_{N}\rightharpoonup\bar{x}$, and each sequence $(v_{N})\subset V$, we have
$\limsup_{N\to\infty}\,\|v_{N}\|_{V}<\infty$ whenever $\omega\in\Omega_{0}$
and $v_{N}\in(1/N)\sum_{i=1}^{N}M_{\xi^{i}(\omega)}(x_{N})$ for each
$N\in\mathbb{N}$.

4.1.1 and 4.1.2 correspond to 3.1.1 and 3.1.2, but we impose in addition
separability of $X$ and $V$. 4.1.1, 4.1.4, and 4.1.3 allow us to use Clarke’s
calculus for generalized gradients of integral functions [18, sect. 2.7]. To
apply the epigraphical law of large numbers derived in [3], we use 4.1.4 and
4.1.3. 4.1.6 and 4.1.7 generalize technical conditions used in [59].

######  Lemma 4.2.

Let $\zeta\colon\Xi\to[0,\infty)$ be measurable and $\mathbb{P}$-integrable,
and let 4.1.1, 4.1.6, and 4.1.2 hold. If $\|m_{\xi}(x)\|_{V}\leq\zeta(\xi)$
for all $m_{\xi}(x)\in M_{\xi}(x)$, each $x\in\mathrm{dom}(\psi)$, and every
$\xi\in\Xi$, then 4.1.7 holds true.

###### Proof.

Since $\zeta(\xi^{1}),\zeta(\xi^{2}),\ldots$ are independent, the law of large
numbers ensures that $(1/N)\sum_{i=1}^{N}\zeta(\xi^{i})$ converges almost
surely to $\int_{\Xi}\zeta(\xi)\,\mathrm{d}\mathbb{P}(\xi)$ as $N\to\infty$.
Hence there exists a set $\Omega_{0}\subset\Omega$ with
$\Omega_{0}\in\mathcal{F}$, $P(\Omega_{0})=1$, and for each
$\omega\in\Omega_{0}$, we have
$(1/N)\sum_{i=1}^{N}\zeta(\xi^{i}(\omega))\to\int_{\Xi}\zeta(\xi)\,\mathrm{d}\mathbb{P}(\xi)$
as $N\to\infty$.

Fix $\omega\in\Omega_{0}$. Let $\bar{x}\in\mathrm{dom}(\psi)$, let
$(x_{N})\subset\mathrm{dom}(\psi)$ fulfill $x_{N}\rightharpoonup\bar{x}$, and
let $(v_{N})$ satisfy $v_{N}\in(1/N)\sum_{i=1}^{N}M_{\xi^{i}(\omega)}(x_{N})$
for all $N\in\mathbb{N}$. The latter ensures the existence of $v_{N}^{i}\in
M_{\xi^{i}(\omega)}(x_{N})$ with $v_{N}=(1/N)\sum_{i=1}^{N}v_{N}^{i}$. Now the
lemma’s hypotheses ensure $\|v_{N}^{i}\|_{V}\leq\zeta(\xi^{i})$. Hence
$\|v_{N}\|_{V}\leq(1/N)\sum_{i=1}^{N}\|v_{N}^{i}\|_{V}\leq(1/N)\sum_{i=1}^{N}\zeta(\xi^{i})$.
We obtain $\limsup_{N\to\infty}\,\|v_{N}\|_{V}<\infty$. ∎

We impose a technical measurability condition on Clarke’s generalized
directional derivative of $F_{\xi}$ in Theorem 4.3. The measurability of
Clarke’s subdifferential as a function of the decision variables and
parameters has been analyzed, for example, in [66, Lem. 4] and [83, Prop.
2.1].

Let $\mathcal{C}$ be the set of all C-stationary points of (4.1), that is, the
set of all $x\in\mathrm{dom}(\psi)$ with
$0\in\partial_{C}\big{[}\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}(\xi)\big{]}+\partial\psi(x)$.
Let $\varepsilon\geq 0$ and for each $\omega\in\Omega$, let
$\hat{\mathcal{C}}_{N}^{\varepsilon}(\omega)$ be the set of all points
$x\in\mathrm{dom}(\psi)$ with

$0\in\bigcup_{v\in\overline{\mathbb{B}}_{X}(x;\varepsilon)\cap\mathrm{dom}(\psi)}\partial_{C}\Big{[}\frac{1}{N}\sum_{i=1}^{N}F_{\xi^{i}(\omega)}(v)\Big{]}+\partial\psi(x);$

cf. [74, eqns. (3.2) and (4.2)]. Each element in
$\hat{\mathcal{C}}_{N}^{\varepsilon}(\omega)$ is referred to as an approximate
C-stationary point of (4.2).

######  Theorem 4.3.

Let Assumption 4.1 hold true and let $(\varepsilon_{N})\subset[0,\infty)$ be a
sequence with $\varepsilon_{N}\to 0$ as $N\to\infty$. Suppose that
$\mathrm{dom}(\psi)\times X\times\Xi\ni(x,h,\xi)\mapsto F_{\xi}^{\circ}(x;h)$
is measurable, and let $\mathcal{C}$ and
$\hat{\mathcal{C}}_{N}^{\varepsilon_{N}}(\omega)$ be nonempty for each
$N\in\mathbb{N}$ and $\omega\in\Omega$.

  1. (i)

If $\Omega_{1}\subset\Omega$ with $\Omega_{1}\in\mathcal{F}$ and
$P(\Omega_{1})=1$ and for each $\omega\in\Omega_{1}$, $(x_{N}(\omega))$ is a
bounded sequence with
$x_{N}(\omega)\in\hat{\mathcal{C}}_{N}^{\varepsilon_{N}}(\omega)$ for all
sufficiently large $N\in\mathbb{N}$, then with probability one (w.p. $1$),
$\mathrm{dist}({x_{N},\mathcal{C}})\to 0$ as $N\to\infty$.

  2. (ii)

If there exists a bounded set $\mathscr{C}\subset X$ such that w.p. $1$,
$\hat{\mathcal{C}}_{N}^{\varepsilon_{N}}\subset\mathscr{C}$ for all
sufficiently large $N\in\mathbb{N}$, then w.p. $1$,
$\mathbb{D}({\hat{\mathcal{C}}_{N}^{\varepsilon_{N}},\mathcal{C}})\to 0$ as
$N\to\infty$.

###### Proof.

We establish the assertions via applications of Theorem 3.4.

First, we observe that for each $\omega\in\Omega_{0}$, 4.1.7 implies 3.1.5
with $M_{k}(\cdot)$ replaced by
$\hat{M}_{N}(\cdot,\omega)=(1/N)\sum_{i=1}^{N}M_{\xi^{i}(\omega)}(\cdot)$.

Second, 4.1.4, 4.1.3, and 4.1.6, and the sum rule [18, Cor. 3 on p. 40] ensure
for all $\omega\in\Omega$, $x\in X_{0}$, and every $N\in\mathbb{N}$,

$\displaystyle\partial_{C}\Big{[}\frac{1}{N}\sum_{i=1}^{N}F_{\xi^{i}(\omega)}(x)\Big{]}=\frac{1}{N}\sum_{i=1}^{N}\partial_{C}F_{\xi^{i}(\omega)}(x)=\frac{1}{N}\sum_{i=1}^{N}BM_{\xi^{i}(\omega)}(x)=B\hat{M}_{N}(x,\omega).$

Hence for all $\omega\in\Omega$, 3.1.4 holds true with $M_{k}(\cdot)$ replaced
by $\hat{M}_{N}(\cdot,\omega)$.

Third, we show that for almost every $\omega\in\Omega$, 3.1.6 is satisfied
with $f(\cdot)$ replaced by the expectation function
$\int_{\Xi}F_{\xi}(\cdot)\,\mathrm{d}\mathbb{P}(\xi)$ and with $f_{k}(\cdot)$
replaced by the SAA objective function
$(1/N)\sum_{i=1}^{N}F_{\xi^{i}(\omega)}(\cdot)$. For each $\xi\in\Xi$,
$X_{0}\times X\ni(x,h)\mapsto F_{\xi}^{\circ}(x;h)$ is upper semicontinuous
[18, Prop. 2.1.1]. Fix $\bar{x}\in\mathrm{dom}(\psi)$. Let
$\mathcal{V}_{\bar{x}}$ be an open neighborhood and let $L_{\bar{x}}$ be a
Lipschitz constant given by 4.1.3. Using [18, Prop. 2.1.1], we have
$|F_{\xi}^{\circ}(x;h)|\leq L_{\bar{x}}(\xi)$ for each
$(x,h,\xi)\in\mathcal{V}_{\bar{x}}\times X\times\Xi$. Moreover
$(x,h,\xi)\mapsto F_{\xi}^{\circ}(x;h)$ is measurable by assumption. Combined
with [3, Thm. 2.3], we find that w.p. $1$, $\mathrm{dom}(\psi)\times
X\ni(x,h)\mapsto(1/N)\sum_{i=1}^{N}F_{\xi^{i}}^{\circ}(x;h)$ hypoconverges to
$(x,h)\mapsto\int_{\Xi}F_{\xi}^{\circ}(x;h)\,\mathrm{d}\mathbb{P}(\xi)$.111We
refer the reader to [3, p. 3] for a definition of epiconvergence of a sequence
of functions $(g_{k})$ to some function $g$. A sequence of functions $(g_{k})$
hypoconverges to $g$ if (and only if) $(-g_{k})$ epiconverges to $-g$ [70, pp.
242–243]. Using 4.1.1, [18, Cor. 3 on p. 40 and Thm. 2.7.2] and Clarke
regularity of $F_{\xi}$ for each $\xi\in\Xi$ (see 4.1.4), we have the
identities
$\int_{\Xi}F_{\xi}^{\circ}(x;h)\,\mathrm{d}\mathbb{P}(\xi)=(\int_{\Xi}F_{\xi}(\cdot)\,\mathrm{d}\mathbb{P}(\xi))^{\circ}(x;h)$
and
$(1/N)\sum_{i=1}^{N}F_{\xi^{i}}^{\circ}(x;h)=\big{(}(1/N)\sum_{i=1}^{N}F_{\xi^{i}}(\cdot)\big{)}^{\circ}(x;h)$
for every $(x,h)\in X_{0}\times X$. Hence there exists
$\Omega_{2}\subset\Omega$ with $\Omega_{2}\in\mathcal{F}$ and
$P(\Omega_{2})=1$ such that for each $\omega\in\Omega_{2}$, all $h\in X$ and
each $(x_{N})\subset\mathrm{dom}(\psi)$ with $x_{N}\to x$, we have

$\limsup_{N\to\infty}\,\Big{(}\frac{1}{N}\sum_{i=1}^{N}F_{\xi^{i}(\omega)}(\cdot)\Big{)}^{\circ}(x_{N};h)\leq\Big{(}\int_{\Xi}F_{\xi}(\cdot)\,\mathrm{d}\mathbb{P}(\xi)\Big{)}^{\circ}(x;h).$

Now we establish parts (i) and (ii).

(i) By assumption and construction, we have
$\Omega_{0}\cap\Omega_{1}\cap\Omega_{2}\in\mathcal{F}$. Moreover this event
happens w.p. $1$. Combining our derivations, Theorem 3.4 ensures for each
$\omega\in\Omega_{0}\cap\Omega_{1}\cap\Omega_{2}$, we have
$\mathrm{dist}({x_{N}(\omega),\mathcal{C}})\to 0$ as $N\to\infty$.

(ii) Since w.p. $1$,
$\hat{\mathcal{C}}_{N}^{\varepsilon_{N}}\subset\mathscr{C}$ for all
sufficiently large $N\in\mathbb{N}$, there exists a set
$\Omega_{3}\in\mathcal{F}$ with $P(\Omega_{3})=1$ such that for each
$\omega\in\Omega_{3}$, there exists $n(\omega)\in\mathbb{N}$ with
$\hat{\mathcal{C}}_{N}^{\varepsilon_{N}}(\omega)\subset\mathscr{C}$ for all
$N\geq n(\omega)$. In particular
$\Omega_{0}\cap\Omega_{2}\cap\Omega_{3}\in\mathcal{F}$ and
$P(\Omega_{0}\cap\Omega_{2}\cap\Omega_{3})=1$. Now Theorem 3.4 ensures for
each $\omega\in\Omega_{0}\cap\Omega_{2}\cap\Omega_{3}$,
$\mathbb{D}({\hat{\mathcal{C}}_{N}^{\varepsilon_{N}}(\omega),\mathcal{C}})\to
0$ as $N\to\infty$. ∎

##  5 Approximations via weakly converging probability measures

Motivated by recent contributions on epiconvergent discretizations of
stochastic programs [19, 22, 34, 67], we establish consistency of
approximations to the stochastic program (4.1) which result from approximating
$\mathbb{P}$ by a sequence of weakly convergent probability measures
$(\mathbb{P}_{N})$. Our main interest in considering weakly convergent
probability measures stems from those considered in [67, sect. 2] and [46,
sect. 3]. Throughout the section, we assume $\Xi$ be a complete, separable
metric space and $\mathbb{P}$ be a probability measure on $\Xi$. We refer the
reader to [42, p. 65] for the definition of weak convergence of probability
measures.

For a sequence $(\mathbb{P}_{N})$ of probability measures defined on $\Xi$, we
approximate (4.1) via the optimization problems

$\displaystyle\min_{x\in\mathrm{dom}(\psi)}\,\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}_{N}(\xi)+\psi(x).$
(5.1)

Our consistency analysis is built on the conditions in assumption 5.1 and
those needed to apply [56, Cor. 3.4].

######  Assumption 5.1.

In addition to 4.1.1, 4.1.2, 4.1.6, and 4.1.5, we consider the following
conditions.

  1. 1.

For each $N\in\mathbb{N}$, $\mathbb{P}_{N}$ is a probability measure on $\Xi$,
and $(\mathbb{P}_{N})$ weakly converges to $\mathbb{P}$ as $N\to\infty$.

  2. 2.

For each $\xi\in\Xi$, $F_{\xi}\colon X_{0}\to$ is continuously differentiable.
For every $x\in X_{0}$, $\Xi\ni\xi\mapsto F_{\xi}(x)\in$ is
$\mathbb{P}_{N}$-integrable. For all $h\in X$,
$X_{0}\times\Xi\ni(x,\xi)\mapsto\langle\mathrm{D}F_{\xi}(x),h\rangle_{{X}^{*}\\!,X}\in$
is upper semicontinuous.

  3. 3.

4.1.3 holds true with $L_{\bar{x}}$ being continuous,
$\mathbb{P}_{N}$-integrable for each $N\in\mathbb{N}$, and satisfying
$\limsup_{N\to\infty}\,\int_{\Xi}L_{\bar{x}}(\xi)\,\mathrm{d}\mathbb{P}_{N}(\xi)\leq\int_{\Xi}L_{\bar{x}}(\xi)\,\mathrm{d}\mathbb{P}(\xi)$.

  4. 4.

For each $\bar{x}\in\mathrm{dom}(\psi)$, each sequence
$(x_{N})\subset\mathrm{dom}(\psi)$ with $x_{N}\rightharpoonup\bar{x}$, and
each sequence $(v_{N})$ with
$v_{N}\in\int_{\Xi}M_{\xi}(x_{N})\,\mathrm{d}\mathbb{P}_{N}(\xi)$ for all
$N\in\mathbb{N}$, $\limsup_{N\to\infty}\,\|v_{N}\|_{V}<\infty$.

While 5.1.2 imposes continuity and differentiablity, these conditions are
satisfied for certain problem classes [63]. However the continuity conditions
may be relaxed in view of [2, Thms. 3.6 and 5.2]. Under 5.1.2, the
continuously differentiable function $F_{\xi}\colon X_{0}\to$ is Clarke
regular for each $\xi\in\Xi$ [18, Prop. 2.3.6] and
$\partial_{C}F_{\xi}(x)=\\{\,\mathrm{D}F_{\xi}(x)\,\\}$ [18, Cor. on p. 32 and
Prop. 2.2.4]. Using Assumption 5.1 and [24, Lem. C.3], we find that $X_{0}\ni
x\mapsto\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}(\xi)$ and $X_{0}\ni
x\mapsto\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}_{N}(\xi)$ are Fréchet
differentiable and we can interchange derivatives and integrals.

Let $\mathfrak{C}$ be the set of all $x\in\mathrm{dom}(\psi)$ with
$0\in\mathrm{D}\big{[}\int_{\Xi}\,F_{\xi}(x)\,\mathrm{d}\mathbb{P}(\xi)\big{]}+\partial\psi(x)$.
Let $\varepsilon\geq 0$ and for each $\omega\in\Omega$, let
$\mathfrak{C}_{N}^{\varepsilon}$ be the set of all points
$x\in\mathrm{dom}(\psi)$ with
$0\in\bigcup_{v\in\overline{\mathbb{B}}_{X}(x;\varepsilon)\cap\mathrm{dom}(\psi)}\mathrm{D}\big{[}\int_{\Xi}\,F_{\xi}(v)\,\mathrm{d}\mathbb{P}_{N}(\xi)\big{]}+\partial\psi(x)$.

######  Theorem 5.2.

Let Assumption 5.1 hold true and let $(\varepsilon_{N})\subset[0,\infty)$ be a
sequence with $\varepsilon_{N}\to 0$ as $N\to\infty$. Let $\mathfrak{C}$ and
$\mathfrak{C}_{N}^{\varepsilon_{N}}$ be nonempty for each $N\in\mathbb{N}$.

  1. (i)

If $(x_{N})$ is bounded with $x_{N}\in\mathfrak{C}_{N}^{\varepsilon_{N}}$ for
each $N\in\mathbb{N}$, then $\mathrm{dist}({x_{N},\mathcal{C}})\to 0$ as
$N\to\infty$.

  2. (ii)

If there exists a bounded set $\mathscr{C}\subset X$ such that
$\mathfrak{C}_{N}^{\varepsilon_{N}}\subset\mathscr{C}$ for all sufficiently
large $N\in\mathbb{N}$, then
$\mathbb{D}({\mathfrak{C}_{N}^{\varepsilon_{N}},\mathfrak{C}})\to 0$ as
$N\to\infty$.

###### Proof.

Using [56, Cor. 3.4], we show that 3.1.6 holds true with $f(\cdot)$ replaced
by $\int_{\Xi}\,F_{\xi}(\cdot)\,\mathrm{d}\mathbb{P}(\xi)$, and $f_{k}(\cdot)$
replaced by $\int_{\Xi}\,F_{\xi}(\cdot)\,\mathrm{d}\mathbb{P}_{N}(\xi)$. Fix
$h\in X$, let $\bar{x}\in\mathrm{dom}(\psi)$ and let
$(x_{N})\subset\mathrm{dom}(\psi)$ with $x_{N}\to\bar{x}$ be arbitrary.
According to 5.1.3, there exists $\varepsilon=\varepsilon(\bar{x})>0$ such
$F_{\xi}$ is Lipschitz continuous on $\mathbb{B}_{X}(\bar{x},\varepsilon)$
with a Lipschitz constant $L_{\bar{x}}(\xi)$ for each $\xi\in\Xi$,
$L_{\bar{x}}$ is continuous, $\mathbb{P}$\- and $\mathbb{P}_{N}$-integrable,
and
$\limsup_{N\to\infty}\,\int_{\Xi}L_{\bar{x}}(\xi)\,\mathrm{d}\mathbb{P}_{N}(\xi)\leq\int_{\Xi}L_{\bar{x}}(\xi)\,\mathrm{d}\mathbb{P}(\xi)$.
The mean value theorem yields
$|\langle\mathrm{D}F_{\xi}(x),h\rangle_{{X}^{*}\\!,X}|\leq
L_{\bar{x}}(\xi)\|h\|_{X}$ for all $x\in\mathbb{B}_{X}(\bar{x},\varepsilon)$,
$h\in X$, and $\xi\in\Xi$. Combined with 5.1.1 and 5.1.2, [56, Cor. 3.4]
yields

$\displaystyle\limsup_{N\to\infty}\,\int_{\Xi}\langle\mathrm{D}F_{\xi}(x_{N}),h\rangle_{{X}^{*}\\!,X}\,\mathrm{d}\mathbb{P}_{N}(\xi)\leq\int_{\Xi}\langle\mathrm{D}F_{\xi}(\bar{x}),h\rangle_{{X}^{*}\\!,X}\,\mathrm{d}\mathbb{P}(\xi).$

Using the chain rule [24, Lem. C.3], we can interchange derivatives and
integrals in the above equation. Hence 3.1.6 holds true.

Both assertions are now implied by Theorem 3.4. ∎

##  6 Application to risk-averse semilinear PDE-constrained optimization

We show that our main result, Theorem 4.3, applies to risk-averse PDE-
constrained optimization using the average value-at-risk. For $\beta\in(0,1)$,
and a $\mathbb{P}$-integrable random variable $Z\colon\Xi\to$, the average
value-at-risk $\mathrm{AVaR}_{\beta}$ evaluated at $Z$ is defined by

$\displaystyle\mathrm{AVaR}_{\beta}[Z]=\inf_{t\in}\,\big{\\{}\,t+\tfrac{1}{1-\beta}\mathbb{E}_{\mathbb{P}}[(Z-t)_{+}]\,\big{\\}},$
(6.1)

where $(s)_{+}=\max\\{0,s\\}$ for $s\in$ [73, eq. (6.23)]. We consider the
risk-averse semilinear PDE-constrained problem

$\displaystyle\min_{u\in
U_{\text{ad}}}\,(1/2)\mathrm{AVaR}_{\beta}[\|\iota_{0}S(u,\cdot)-y_{d}\|_{L^{2}(D)}^{2}]+\wp(\|u\|_{L^{2}(D)}),$
(6.2)

where $\wp\colon[0,\infty)\to[0,\infty)$, $y_{d}\in L^{2}(D)$,
$U_{\text{ad}}\subset L^{2}(D)$, and for each $(u,\xi)\in L^{2}(D)\times\Xi$,
$y=S(u,\xi)\in H_{0}^{1}(D)$ solves the semilinear PDE

$\displaystyle(\kappa(\xi)\nabla y,\nabla
v)_{L^{2}(D)^{d}}+(y^{3},v)_{L^{2}(D)}=(u,v)_{L^{2}(D)}+(b(\xi),v)_{L^{2}(D)}\>\>\text{for
all}\>\>v\in H_{0}^{1}(D).$

We impose mild conditions on the semilinear PDE, the feasible set
$U_{\text{ad}}$, and $\wp$.

######  Assumption 6.1 (semilinear control problem).

  1. 1.

$D\subset^{d}$ is a bounded, convex, polygonal/polyhedral domain with
$d\in\\{2,3\\}$.

  2. 2.

$\kappa:\Xi\to L^{\infty}(D)$ is strongly measurable and there exists
$0<\kappa_{\min}\leq\kappa_{\max}<\infty$ such that for all $\xi\in\Xi$,
$\kappa_{\min}\leq\kappa(\xi)(x)\leq\kappa_{\max}$ for a.e. $x\in D$.

  3. 3.

$b:\Xi\to L^{2}(D)$ is measurable and there exists $b_{\max}\geq 0$ with
$\|b(\xi)\|_{L^{2}(D)}\leq b_{\max}$ for all $\xi\in\Xi$.

  4. 4.

$U_{\text{ad}}\subset L^{2}(D)$ is closed, convex, nonempty, and there exists
$r_{\text{ad}}\in(0,\infty)$ such that $\|u\|_{L^{2}(D)}\leq r_{\text{ad}}$
for all $u\in U_{\text{ad}}$.

  5. 5.

$\wp\colon[0,\infty)\to[0,\infty)$ is convex and strictly increasing.

If not stated otherwise, let Assumption 6.1 hold throughout the section. Let
us define $G\colon\times\to$ and $\widehat{J}\colon
L^{2}(D)\times\Xi\to[0,\infty)$ by

$\displaystyle
G(s,t)=t+\tfrac{1}{1-\beta}(s-t)_{+}\quad\text{and}\quad\widehat{J}(u,\xi)=(1/2)\|\iota_{0}S(u,\xi)-y_{d}\|_{L^{2}(D)}^{2}.$

Moreover, we define $L^{2}(D)\times\times\Xi\ni(u,t,\xi)\mapsto
F_{\xi}(u,t)\in[0,\infty)$ by

$\displaystyle F_{\xi}(u,t)=G(\widehat{J}(u,\xi),t).$

To study consistency of SAA C-stationary points, we reformulate (6.2)
equivalently as

$\displaystyle\min_{(u,t)\in
U_{\text{ad}}\times}\,\int_{\Xi}G(\widehat{J}(u,\xi),t)\,\mathrm{d}\mathbb{P}(\xi)+\wp(\|u\|_{L^{2}(D)}).$
(6.3)

Let $\Xi$, let $\mathbb{P}$, and let $\xi^{1}$, $\xi^{2},\ldots$ be as in
section 4. The SAA problem of (6.3) is given by

$\displaystyle\min_{(u,t)\in
U_{\text{ad}}\times}\,\frac{1}{N}\sum_{i=1}^{N}G(\widehat{J}(u,\xi^{i}),t)+\wp(\|u\|_{L^{2}(D)}).$
(6.4)

For the remainder of the section, we verify Assumption 4.1. For the spaces
$L^{2}(D)\times$, $V=H_{0}^{1}(D)\times$, and the open, convex, and bounded
set $U_{0}=U_{\text{ad}}+\mathbb{B}_{L^{2}(D)}(0;1)$, 4.1.1 holds true. Let
$\psi$ be the sum of the indicator function of $U_{\text{ad}}$ and the
regularizer $\wp\circ\|\cdot\|_{L^{2}(D)}$. Since $\wp$ is convex and strictly
increasing, and $L^{2}(D)$ is a Hilbert space, $\psi$ has the Kadec property;
see [62, Lem. 1] and [8, Thm. 6.5]. Hence 4.1.2 holds true.

For each $(u,\xi)\in L^{2}(D)\times\Xi$, the PDE solution $S(u,\xi)$ is well-
defined [85, Thm. 26.A], and we have for all $(u,\xi)\in L^{2}(D)\times\Xi$,
the stability estimate

$\displaystyle\|S(u,\xi)\|_{H_{0}^{1}(D)}\leq(C_{D}/\kappa_{\min})\|u\|_{L^{2}(D)}+C_{D}(b_{\max}/\kappa_{\min}).$
(6.5)

Defining

$\displaystyle\mathfrak{c}_{S}=(C_{D}/\kappa_{\min})(r_{\text{ad}}+1)+C_{D}(b_{\max}/\kappa_{\min}),$

we obtain for all $(u,\xi)\in U_{0}\times\Xi$, the stability estimate

$\displaystyle\widehat{J}(u,\xi)\leq\|\iota_{0}S(u,\xi)\|_{L^{2}(D)}^{2}+\|y_{d}\|_{L^{2}(D)}^{2}\leq
C_{D}^{2}\mathfrak{c}_{S}^{2}+\|y_{d}\|_{L^{2}(D)}^{2}.$ (6.6)

The adjoint approach [37, sect. 1.6.2] ensures that for each $\xi\in\Xi$,
$\widehat{J}(\cdot,\xi)$ is continuously differentiable and

$\displaystyle\nabla_{u}\widehat{J}(\cdot,\xi)=-\iota_{0}z(u,\xi)\quad\text{for
all}\quad(u,\xi)\in L^{2}(D)\times\Xi,$

where for each $(u,\xi)\in L^{2}(D)\times\Xi$, $z=z(u,\xi)\in H_{0}^{1}(D)$
solves the adjoint equation

$\displaystyle(\kappa(\xi)\nabla z,\nabla
v)_{L^{2}(D)^{d}}+3(S(u,\xi)^{2}z,v)_{L^{2}(D)}=-(\iota_{0}S(u,\xi)-y_{d},v)_{L^{2}(D)}$

for all $v\in H_{0}^{1}(D)$. For all $(u,\xi)\in L^{2}(D)\times\Xi$, we have

$\displaystyle\|z(u,\xi)\|_{H_{0}^{1}(D)}\leq(C_{D}/\kappa_{\min})\|\iota_{0}S(u,\xi)\|_{L^{2}(D)}+(C_{D}/\kappa_{\min})\|y_{d}\|_{L^{2}(D)}.$

Combined with the stability estimate (6.5), we have for each $(u,\xi)\in
U_{0}\times\Xi$,

$\displaystyle\|z(u,\xi)\|_{H_{0}^{1}(D)}\leq(C_{D}^{2}/\kappa_{\min})\mathfrak{c}_{S}+(C_{D}/\kappa_{\min})\|y_{d}\|_{L^{2}(D)}.$
(6.7)

Let $\mathfrak{c}_{z}$ be the right-hand side in (6.7). Since $G$ is Lipschitz
continuous and the bound $\|\nabla_{u}\widehat{J}(\cdot,\xi)\|_{L^{2}(D)}\leq
C_{D}\mathfrak{c}_{z}$ is valid for all $(u,\xi)\in U_{0}\times\Xi$, 4.1.3 is
satisfied.

Fix $\xi\in\Xi$. Since the mapping $(u,t)\mapsto(\widehat{J}(u,\xi),t)\in^{2}$
is continuously differentiable and $g$ is finite-valued and convex, $F_{\xi}$
is Clarke regular on $L^{2}(D)\times$ and

$\displaystyle\partial_{C}F_{\xi}(u,t)=(\nabla_{u}\widehat{J}(u,\xi),-1)^{*}\partial
G(\widehat{J}(u,\xi),t)=\begin{bmatrix}\frac{1}{1-\beta}\partial(\widehat{J}(u,\xi)-t)_{+}\nabla_{u}\widehat{J}(u,\xi)\\\
1-\frac{1}{1-\beta}\partial(\widehat{J}(u,\xi)-t)_{+}\end{bmatrix};$

see [18, Prop. 2.3.6 and Thm. 2.3.10]. The Clarke regularity ensures 4.1.4.
Using [4, Thm. 8.2.9], we can show that $S(u,\cdot)$ is measurable for each
$u\in L^{2}(D)$. Combined with (6.6), we find that 4.1.5 holds true.

We define $B\colon H_{0}^{1}(D)\times\to L^{2}(D)\times$ and $M_{\xi}\colon
L^{2}(D)\times\rightrightarrows H_{0}^{1}(D)\times$ by

$\displaystyle B(v,s)=(-\iota_{0}v,s)\quad\text{and}\quad
M_{\xi}(u,t)=\begin{bmatrix}\frac{1}{1-\beta}\partial(\widehat{J}(u,\xi)-t)_{+}z(u,\xi)\\\
1-\frac{1}{1-\beta}\partial(\widehat{J}(u,\xi)-t)_{+}\end{bmatrix}.$ (6.8)

We obtain

$\displaystyle\partial_{C}F_{\xi}(u,t)=BM_{\xi}(u,t)\quad\text{for
all}\quad(u,t,\xi)\in L^{2}(D)\times\times\Xi.$

Combined with the fact that $\iota_{0}$ is linear and compact [37, Thm. 1.14],
we conclude that 4.1.6 is fulfilled.

Next we verify 4.1.7 using Lemma 4.2. For each $s\in$,
$\partial(s)_{+}\subset[0,1]$. Fix $(u,t,\xi)\in
U_{\text{ad}}\times\times\Xi$. For all $m=(m_{1},m_{2})\in M_{\xi}(u,t)$,
(6.8) yields

$\displaystyle\|m\|_{H_{0}^{1}(D)\times}=\|m_{1}\|_{H_{0}^{1}(D)}+|m_{2}|\leq\tfrac{1}{1-\beta}\mathfrak{c}_{z}+1+\tfrac{1}{1-\beta}.$

Hence Lemma 4.2 implies 4.1.7.

To summarize, we have shown that Assumption 4.1 holds true. In order to
demonstrate the convergence of SAA C-stationary points of (6.4) towards those
of (6.3) via Theorem 4.3, we need to show that the SAA C-stationary points are
contained in a bounded, deterministic set. Let $(u_{N},t_{N})\in
U_{\text{ad}}\times$ be a C-stationary point of (6.3). We have $u_{N}\in
U_{\text{ad}}$, which is by assumption a bounded set. Next we show that

$\displaystyle
t_{N}\in[0,C_{D}^{2}\mathfrak{c}_{S}^{2}+\|y_{d}\|_{L^{2}(D)}^{2}].$

Since $(u_{N},t_{N})$ is a C-stationary point of (6.3), the second formula in
(6.8) implies that
$1\in\tfrac{1}{N(1-\beta)}\sum_{i=1}^{N}\partial(\widehat{J}(u_{N},\xi^{i})-t_{N})_{+}$.
Combined with $\beta\in(0,1)$ and $\widehat{J}\geq 0$, we have $t_{N}\geq 0$.
Let $\widehat{\mathrm{AVaR}}_{\beta,N}$ be the empirical estimate of
$\mathrm{AVaR}_{\beta}$ with $\mathbb{P}$ replaced by the empirical
distribution of the sample $\xi^{1},\ldots,\xi^{N}$ in (6.1). The minimization
problem in (6.1) with $\mathbb{P}$ replaced by the empirical distribution is
convex. Hence $t_{N}$ solves (6.1) with $\mathbb{P}$ replaced by the empirical
distribution. As with $\mathrm{AVaR}_{\beta}$ [73, p. 231], the empirical
average value-at-risk $\widehat{\mathrm{AVaR}}_{\beta,N}$ is a coherent risk
measure. Since $(s)_{+}\geq 0$ for all $s\in$, $\beta\in(0,1)$, and coherent
risk measures are monotone, translation equivariant, and positively
homogeneous [73, Def. 6.4], the upper bound on $\widehat{J}$ in (6.6) yields

$\displaystyle
t_{N}\leq\widehat{\mathrm{AVaR}}_{\beta,N}[\widehat{J}(u_{N},\xi)]\leq
C_{D}^{2}\mathfrak{c}_{S}^{2}+\|y_{d}\|_{L^{2}(D)}^{2}.$

##  7 Application to risk-neutral bilinear PDE-constrained optimization

Motivated by the deterministic bilinear control problems studied in [14, 27,
41, 52], we consider risk-neutral optimization problems governed by a bilinear
elliptic PDEs with random inputs:

$\displaystyle\min_{u\in
U_{\text{ad}}}\,\int_{\Xi}J(S(u,\xi))\,\mathrm{d}\mathbb{P}(\xi)+(\alpha/2)\|u\|_{L^{2}(D)}^{2},$
(7.1)

where $\alpha>0$, and for each $(u,\xi)\in U_{\text{ad}}\times\Xi$,
$y=S(u,\xi)\in H_{0}^{1}(D)$ is the solution to

$\displaystyle(\kappa(\xi)\nabla y,\nabla
v)_{L^{2}(D)^{d}}+(g(\xi)uy,v)_{L^{2}(D)}=(b(\xi),v)_{L^{2}(D)}\quad\text{for
all}\quad v\in H_{0}^{1}(D).$ (7.2)

The following assumptions ensure existence and regularity of the solutions to
(7.2), and impose conditions on the function $J$ and the set $U_{\text{ad}}$.
These conditions allow us to Assumption 4.1 and apply Theorem 4.3 to study the
consistency of SAA C-stationary points corresponding to (7.1). Conditions
beyond those formulated in Assumption 7.1 may be required to allow for
applications of Theorem 5.2.

######  Assumption 7.1 (bilinear control problem).

  1. 1.

$D\subset^{d}$ is a bounded, convex, polygonal/polyhedral domain with
$d\in\\{2,3\\}$.

  2. 2.

$\kappa:\Xi\to C^{1}(\bar{D})$ is measurable and there exists
$\kappa_{\min}>0$ such that $\kappa_{\min}\leq\kappa(\xi)(x)$ for all
$(\xi,x)\in\Xi\times\bar{D}$. Moreover
$\mathbb{E}_{\mathbb{P}}[\|\kappa\|_{C^{1}(\bar{D})}^{p}]<\infty$ for all
$p\in[1,\infty)$.

  3. 3.

$b:\Xi\to L^{2}(D)$ and $g:\Xi\to C^{0,1}(\bar{D})$ are measurable and there
exist $b_{\max}$, $g_{\max}>0$ with $\|b(\xi)\|_{L^{2}(D)}\leq b_{\max}$ and
$\|g(\xi)\|_{C^{0,1}(\bar{D})}\leq g_{\max}$ for all $\xi\in\Xi$. Moreover,
$g(\xi)(x)\geq 0$ for all $(\xi,x)\in\Xi\times\bar{D}$.

  4. 4.

$J:H_{0}^{1}(D)\to[0,\infty)$ is convex, continuously differentiable and its
derivative is Lipschitz continuous with Lipschitz constant $\ell>0$. For some
nondecreasing function $\varrho:[0,\infty)\to[0,\infty)$, it holds that

$\displaystyle J(y)\leq\varrho(\|y\|_{H_{0}^{1}(D)})\quad\text{for all}\quad
y\in H_{0}^{1}(D).$

  5. 5.

$U_{\text{ad}}=\\{\,u\in L^{2}(D):\,0\leq
u(x)\leq\mathfrak{u}(x)\;\text{a.e.}\;x\in D\,\\}$ with $\mathfrak{u}\in
L^{\infty}(D)$ and $\mathfrak{u}(x)\geq 0$ a.e. $x\in D$.

If not stated otherwise, let Assumption 7.1 hold throughout the section. We
introduce notation used throughout the section. Let
$C_{H_{0}^{1};L^{4}}\in(0,\infty)$ be the embedding constant of the compact
embedding $H_{0}^{1}(D)\xhookrightarrow{}L^{4}(D)$ and let
$C_{H^{2};L^{\infty}}>0$ be that of the continuous embedding
$H^{2}(D)\xhookrightarrow{}L^{\infty}(D)$ [37, Thm. 1.14]. We define
$C_{\mathfrak{u}}=\|\mathfrak{u}\|_{L^{\infty}(D)}$ and
$|D|=\int_{D}1\,\mathrm{d}x$.

Let $\psi$ be the sum of the indicator function of $U_{\text{ad}}$ and
$(\alpha/2)\|u\|_{L^{2}(D)}^{2}$. The function $\psi$ has the Kadec property
[8, Thm. 6.5], and 4.1.2 holds true.

Next we provide an example function satisfying 7.1.4. Let $y_{d}\in L^{2}(D)$.
We define $J:H_{0}^{1}(D)\to[0,\infty)$ by
$J(y)=(1/2)\|\iota_{0}y-y_{d}\|_{L^{2}(D)}^{2}$. The function $J$ satisfies
7.1.4 with $\ell=C_{D}^{2}$ and
$\varrho(t)=C_{D}^{2}t^{2}+\|y_{d}\|_{L^{2}(D)}^{2}$.

We use two technical facts for our analysis. Lemma 7.2 establishes a bound on
the derivatives of the objective functions fulfilling 7.1.4.

######  Lemma 7.2.

If $D\subset^{d}$ is a bounded domain and 7.1.4 holds, then
$\|\mathrm{D}J(y)\|_{H^{-1}(D)}^{2}\leq 2\ell J(y)$ for all $y\in
H_{0}^{1}(D)$.

###### Proof.

Fix $\tilde{y}$, $y\in H_{0}^{1}(D)$. Let $\nabla J(y)\in H_{0}^{1}(D)$ be the
Riesz representation of $\mathrm{D}J(y)$. Using 7.1.4 and [86, Cor. 3.5.7 and
Rem. 3.5.2], we have $0\leq J(\tilde{y})\leq J(y)+(\nabla
J(y),\tilde{y}-y)_{H_{0}^{1}(D)}+(\ell/2)\|\tilde{y}-y\|_{H_{0}^{1}(D)}^{2}$.
Minimizing over $\tilde{y}\in H_{0}^{1}(D)$ yields $0\leq
J(y)-(1/(2\ell))\|\nabla J(y)\|_{H_{0}^{1}(D)}^{2}$. ∎

Using 7.1.1 and theorems on the multiplication of Sobolev functions (see [6,
Thm. 7.4] and [29, Thm. 1.4.1.1]), we can show that the trilinear (pointwise
multiplication) operator

$\displaystyle C^{0,1}(\bar{D})\times H^{1}(D)\times H^{2}(D)\ni(g,v,w)\mapsto
gvw\in H^{1}(D)$ (7.3)

is continuous. Hence it is bounded [55, p. 68]. Therefore, we obtain the
existence of a constant $C_{\mathrm{tri}}\in(0,\infty)$ such that for all
$(g,v,w)\in C^{0,1}(\bar{D})\times H^{1}(D)\times H^{2}(D)$,

$\displaystyle\|gvw\|_{H^{1}(D)}\leq
C_{\mathrm{tri}}\|g\|_{C^{0,1}(\bar{D})}\|v\|_{H^{1}(D)}\|w\|_{H^{2}(D)}.$
(7.4)

###  7.1 Existence and uniqueness of PDE solutions

We show that the PDE (7.2) has a unique solution $S(u,\xi)\in H_{0}^{1}(D)$
for each $\xi\in\Xi$ and $u$ in an open neighborhood of the set
$U_{\text{ad}}$ in $L^{2}(D)$. This allows us to verify 4.1.3, 4.1.4, 4.1.5,
and 4.1.1. The existence results established in [14, sect. 3.1] are not
applicable to our setting, as we consider $L^{2}(D)$ instead of
$L^{\infty}(D)$ the control space. An approach different from ours to
constructing such a neighborhood is developed in [82, p. 158].

We define

$\displaystyle U_{0}=\bigcup_{u_{0}\in
U_{\text{ad}}}\,\mathbb{B}_{L^{2}(D)}(u_{0};\delta),\quad\text{where}\quad\delta=\tfrac{\kappa_{\min}}{2g_{\max}C_{H_{0}^{1};L^{4}}^{2}}>0.$
(7.5)

We have the identity
$U_{0}=U_{\text{ad}}+\mathbb{B}_{L^{2}(D)}(u_{0};\delta)$.

Fix $(u,\xi)\in U_{0}\times\Xi$ and $y\in H_{0}^{1}(D)$. Since $u\in U_{0}$,
there exists $u_{0}\in U_{\text{ad}}$ with
$u\in\mathbb{B}_{L^{2}(D)}(u_{0};\delta)$. Combining Hölder’s inequality and
the continuity of $H_{0}^{1}(D)\xhookrightarrow{}L^{4}(D)$ with the definition
of $\delta$ provided in (7.5) and $\|u-u_{0}\|_{L^{2}(D)}\leq\delta$, we
obtain

$\displaystyle|(g(\xi)(u-u_{0})y,y)_{L^{2}(D)}|\leq
C_{H_{0}^{1};L^{4}}^{2}g_{\max}\|u-u_{0}\|_{L^{2}(D)}\|y\|_{H_{0}^{1}(D)}^{2}\leq(\kappa_{\min}/2)\|y\|_{H_{0}^{1}(D)}^{2}.$

Using $u_{0}(x)\geq 0$ and $g(\xi)(x)\geq 0$ for a.e. $x\in D$,
$(g(\xi)u_{0}y,y)_{L^{2}(D)}\geq 0$. It follows that

$\displaystyle\begin{aligned} &(\kappa(\xi)\nabla y,\nabla
y)_{L^{2}(D)^{d}}+(g(\xi)uy,y)_{L^{2}(D)}\\\ &\quad=(\kappa(\xi)\nabla
y,\nabla
y)_{L^{2}(D)^{d}}+(g(\xi)(u-u_{0})y,y)_{L^{2}(D)}+(g(\xi)u_{0}y,y)_{L^{2}(D)}\\\
&\quad\geq\kappa_{\min}\|y\|_{H_{0}^{1}(D)}^{2}-(\kappa_{\min}/2)\|y\|_{H_{0}^{1}(D)}^{2}=(\kappa_{\min}/2)\|y\|_{H_{0}^{1}(D)}^{2}.\end{aligned}$
(7.6)

The right-hand side in (7.2) defines a continuous bilinear form. Hence the
Lax–Milgram lemma ensures that the state equation (7.2) has a unique solution
$S(u,\xi)\in H_{0}^{1}(D)$ and yields the stability estimate

$\displaystyle\|S(u,\xi)\|_{H_{0}^{1}(D)}\leq(2/\kappa_{\min})C_{D}\|b(\xi)\|_{L^{2}(D)}\quad\text{for
all}\quad(u,\xi)\in U_{0}\times\Xi.$ (7.7)

We define the parameterized operator $E:H_{0}^{1}(D)\times
L^{2}(D)\times\Xi\to H^{-1}(D)$ by

$\displaystyle\langle
E(y,u,\xi),v\rangle_{H^{-1}(D),H_{0}^{1}(D)}=(\kappa(\xi)\nabla y,\nabla
v)_{L^{2}(D)^{d}}+(g(\xi)uy,v)_{L^{2}(D)}-(b(\xi),v)_{L^{2}(D)}.$

The mapping $E$ is well-defined and $E(\cdot,\cdot,\xi)$ is infinitely many
times continuously differentiable. For each $(y,u,\xi)\in H_{0}^{1}(D)\times
L^{2}(D)\times\Xi$ and $h$, $v\in H_{0}^{1}(D)$, we have

$\displaystyle\begin{aligned} \langle
E_{y}(y,u,\xi)h,v\rangle_{H^{-1}(D),H_{0}^{1}(D)}&=(\kappa(\xi)\nabla h,\nabla
v)_{L^{2}(D)^{d}}+(g(\xi)uh,v)_{L^{2}(D)}.\end{aligned}$

Together with (7.6), we obtain for all $(u,\xi)\in U_{0}\times\Xi$,

$\displaystyle\langle
E_{y}(S(u,\xi),u,\xi)h,h\rangle_{H^{-1}(D),H_{0}^{1}(D)}\geq(\kappa_{\min}/2)\|h\|_{H_{0}^{1}(D)}^{2}\quad\text{for
all}\quad h\in H_{0}^{1}(D).$

The Lax–Milgram lemma ensures that $E_{y}(S(u,\xi),u,\xi)$ has a bounded
inverse and

$\displaystyle\|E_{y}(S(u,\xi),u,\xi)^{-1}\|_{\mathscr{L}(H_{0}^{1}(D),H^{-1}(D))}\leq(2/\kappa_{\min})\quad\text{for
all}\quad(u,\xi)\in U_{0}\times\Xi.$ (7.8)

For each $y$, $v\in H_{0}^{1}(D)$ and $u\in L^{2}(D)$, we can show that
$\langle E(y,u,\cdot),v\rangle_{H^{-1}(D),H_{0}^{1}(D)}$ is measurable, as
7.1.1, 7.1.2, and 7.1.3 hold. Since $H^{-1}(D)$ is separable, $E(y,u,\cdot)$
is measurable [39, Thm. 1.1.6]. Hence $S(u,\cdot)$ is measurable [4, Thm.
8.2.9]. Combined with 7.1.4, we find that $J(S(u,\cdot))$ is a random variable
for each $u\in U_{0}$.

###  7.2 PDE regularity

We show that the solution $S(u,\xi)$ to (7.2) is contained in $H^{2}(D)$ and
derive a stability estimate for each $(u,\xi)\in U_{\text{ad}}\times\Xi$. The
computations performed here form the basis of verifying 4.1.6.

Fix $(u,\xi)\in U_{\text{ad}}\times\Xi$. Hence $u\in L^{\infty}(D)$. To
establish an $H^{2}(D)$-stability estimate, we apply [77, Thm. 3.1] to the
solution $\widetilde{y}=\widetilde{y}(u,\xi)\in H_{0}^{1}(D)$ to

$\displaystyle(\kappa(\xi)\nabla\widetilde{y},\nabla
v)_{L^{2}(D)^{d}}=(b(\xi),v)_{L^{2}(D)}-(g(\xi)uS(u,\xi),v)_{L^{2}(D)}\quad\text{for
all}\quad v\in H_{0}^{1}(D).$

Since this equation has a unique solution $\widetilde{y}(u,\xi)$ and
$S(u,\xi)$ is a solution, we have $\widetilde{y}(u,\xi)=S(u,\xi)$. Defining
the random variable

$\displaystyle
C_{\kappa}(\xi)=\frac{\|\kappa(\xi)\|_{C^{0}(\bar{D})}\|\kappa(\xi)\|_{C^{1}(\bar{D})}^{2}}{\kappa_{\min}^{4}},$
(7.9)

and applying [77, Thm. 2.1], we find that $S(u,\xi)\in H^{2}(D)$ and

$\displaystyle\|S(u,\xi)\|_{H^{2}(D)}\leq
C_{H^{2}}C_{\kappa}(\xi)\Big{(}\|g(\xi)uS(u,\xi)\|_{L^{2}(D)}+\|b(\xi)\|_{L^{2}(D)}\Big{)},$

where $C_{H^{2}}=C_{H^{2}}(D)>0$ is a deterministic constant. Combined with
Hölder’s and Friedrichsínequalities, we obtain for all $(u,\xi)\in
U_{\text{ad}}\times\Xi$,

$\displaystyle\|S(u,\xi)\|_{H^{2}(D)}\leq
C_{H^{2}}C_{\kappa}(\xi)\big{(}C_{D}C_{\mathfrak{u}}g_{\max}\|S(u,\xi)\|_{H_{0}^{1}(D)}+\|b(\xi)\|_{L^{2}(D)}\big{)}.$
(7.10)

###  7.3 Gradient regularity

In this section, we show that 4.1.3, 4.1.4, 4.1.6, and 4.1.7 hold true. Using
the construction of $U_{0}$, we can define the objective function
$U_{0}\times\Xi\ni(u,\xi)\mapsto F_{\xi}(u)\in[0,\infty)$ by

$\displaystyle F_{\xi}(u)=J(S(u,\xi)).$

Assumption 7.1 ensures that $F_{\xi}$ is continuously differentiable for every
$\xi\in\Xi$. Hence 4.1.4 holds true [18, Prop. 2.3.6]. The adjoint approach
[37, sect. 1.6.2] yields for each $(u,\xi)\in U_{0}\times\Xi$,

$\displaystyle(\nabla
F_{\xi}(u),s)_{L^{2}(D)}=(g(\xi)S(u,\xi)z(u,\xi),s)_{L^{2}(D)}\quad\text{for
all}\quad s\in L^{2}(D),$ (7.11)

where for each $(u,\xi)\in U_{0}\times\Xi$, the adjoint state $z=z(u,\xi)\in
H_{0}^{1}(D)$ solves

$\displaystyle(\kappa(\xi)\nabla z,\nabla
v)_{L^{2}(D)^{d}}+(g(\xi)uz,v)_{L^{2}(D)}=-\langle\mathrm{D}J(S(u,\xi)),v\rangle_{H^{-1}(D),H_{0}^{1}(D)}$
(7.12)

for all $v\in H_{0}^{1}(D)$. Using (7.8) and (7.12), we obtain the stability
estimate

$\displaystyle\begin{aligned}
\|z(u,\xi)\|_{H_{0}^{1}(D)}&\leq(2/\kappa_{\min})\|\mathrm{D}J(S(u,\xi))\|_{H^{-1}(D)}\;\;\text{for
all}\;\;(u,\xi)\in U_{0}\times\Xi.\end{aligned}$

Combined with the stability estimate (7.7), 7.1.4, and Lemma 7.2, we have for
all $(u,\xi)\in U_{0}\times\Xi$,

$\displaystyle\begin{aligned}
\|z(u,\xi)\|_{H_{0}^{1}(D)}&\leq(2/\kappa_{\min})\big{(}2\ell\varrho((2/\kappa_{\min})C_{D}b_{\max})\big{)}^{1/2}.\end{aligned}$
(7.13)

We show that 4.1.3 is satisfied. Using the Hölder and Friedrichs inequalities,
the continuity of $H_{0}^{1}(D)\xhookrightarrow{}L^{4}(D)$, the stability
estimates (7.7) and (7.13), and the gradient formula (7.11), we have
$(u,\xi)\in U_{0}\times\Xi$,

$\displaystyle\|\nabla F_{\xi}(u)\|_{L^{2}(D)}$ $\displaystyle\leq
C_{H_{0}^{1};L^{4}}^{2}g_{\max}\|S(u,\xi)\|_{H_{0}^{1}(D)}\|z(u,\xi)\|_{H_{0}^{1}(D)}$
$\displaystyle\leq
C_{H_{0}^{1};L^{4}}^{2}g_{\max}(2/\kappa_{\min})^{2}C_{D}b_{\max}\big{(}2\ell\varrho((2/\kappa_{\min})C_{D}b_{\max})\big{)}^{1/2}.$

Right-hand side in the above equation is a deterministic constant. Hence 4.1.3
holds true.

The stability estimate (7.7) and 7.1.4 yield for all $(u,\xi)\in
U_{0}\times\Xi$,

$\displaystyle F_{\xi}(u)\leq\varrho((2/\kappa_{\min})C_{D}b_{\max}).$ (7.14)

Combined with the measurability of $J(S(u,\cdot))$ for each $u\in U_{0}$, we
find that 4.1.5 holds true.

We verify 4.1.6 by showing that for each $(u,\xi)\in U_{\text{ad}}\times\Xi$,

$\displaystyle\nabla F_{\xi}(u)=\iota_{1}m_{\xi}(u),\quad\text{where}\quad
m_{\xi}(u)=g(\xi)S(u,\xi)z(u,\xi).$ (7.15)

We define $M_{\xi}(u)=\\{\,m_{\xi}(u)\,\\}$, $V=H^{1}(D)$, and $B=\iota_{1}$.
Fix $(u,\xi)\in U_{\text{ad}}\times\Xi$. We have $S(u,\xi)\in H^{2}(D)$ (see
section 7.2 and (7.10)) and $z(u,\xi)\in H^{1}(D)$ (see (7.13)). Combined with
$g(\xi)\in C^{0,1}(\bar{D})$ and (7.4), we find that $m_{\xi}(u)\in H^{1}(D)$.
Hence (7.11) implies (7.15).

We show that 4.1.7 holds true using Lemma 4.2. Fix $(u,\xi)\in
U_{\text{ad}}\times\Xi$. Using (7.4) and (7.15), we have

$\displaystyle\|m_{\xi}(u)\|_{H^{1}(D)}\leq
C_{\mathrm{tri}}g_{\max}\|S(u,\xi)\|_{H^{2}(D)}\|z(u,\xi)\|_{H^{1}(D)}.$

Combined with the stability estimates (7.7), (7.10), and (7.13), and
$\|y\|_{H^{1}(D)}\leq(1+C_{D})\|y\|_{H_{0}^{1}(D)}$ valid for all $y\in
H_{0}^{1}(D)$, we find that

$\displaystyle\|m_{\xi}(u)\|_{H^{1}(D)}$ $\displaystyle\leq
C_{\mathrm{tri}}g_{\max}C_{H^{2}}C_{\kappa}(\xi)\big{(}C_{D}^{2}C_{\mathfrak{u}}g_{\max}(2/\kappa_{\min})b_{\max}+b_{\max}\big{)}$
$\displaystyle\quad\cdot(2/\kappa_{\min})C_{D}(1+C_{D})\big{(}2\ell\varrho((2/\kappa_{\min})C_{D}b_{\max})\big{)}^{1/2}.$

We define $\zeta$ as the right-hand side in the above equation. Using 7.1.2
and (7.9), we find that $\zeta$ is integrable. Hence Lemma 4.2 yields 4.1.7.

##  8 Discussion

A multitude of applications in science and engineering yield PDE-constrained
optimization problems under uncertainty. Motivated by such applications, we
have considered infinite-dimensional stochastic optimization problems. We
approximated the expectations in the stochastic programs using two approaches:
the SAA approach and using probability measures weakly converging to the
random element’s probability distribution. For both approximation approaches,
we established asymptotic consistency statements for C-stationary points under
Clarke regularity of integrands. We applied our framework to a risk-averse
semilinear PDE-constrained optimization problem with the average value-at-risk
as a risk measure, and to risk-neutral bilinear PDE-constrained optimization.

Our consistency analysis requires integrands be Clarke regular. For many
applications in PDE-constrained optimization, integrands are continuously
differentiable and hence Clarke regular. For nonregular integrands,
consistency analysis of SAA first-order stationary points is more demanding.
Using smoothing functions of integrands, the SAA approach has been analyzed in
[12, 83] as the sample size approaches infinity and smoothing parameters
converge to zero. We refer the reader to [69] for recent contributions to the
consistency analysis of SAA stationary points as applied to risk-neutral
optimization problems with integrands that lack Clarke regularity.

Our main result, Theorem 4.3, establishes the consistency of SAA C-stationary
points of infinite-dimensional risk-neutral optimization problems. While
applicable to a large set of applications, it may be desirable to establish
consistency results for sample-based approximations of chance-constrained
problems governed by PDEs with random inputs [15, 21, 79], and PDE-constrained
problems under uncertainty with state constraints [25, 47]. Further future
work includes extending the consistency analysis to a broader class of risk-
averse PDE-constrained optimization problems.

## Acknowledgments

JM is very grateful to Professor Alexander Shapiro for taking the time to
address several of my questions related to the SAA approach.

## References

  * [1] J. Allen, E. Young, P. Bortolotti, R. King, and G. Barter, Blade planform design optimization to enhance turbine wake control, Wind Energy, 25 (2022), pp. 811–830, https://doi.org/10.1002/we.2699.
  * [2] Z. Artstein and R. J.-B. Wets, Stability results for stochastic programs and sensors, allowing for discontinuous objective functions, SIAM J. Optim., 4 (1994), pp. 537–550, https://doi.org/10.1137/0804030.
  * [3] Z. Artstein and R. J.-B. Wets, Consistency of minimizers and the SLLN for stochastic programs, J. Convex Anal., 2 (1995), pp. 1–17.
  * [4] J.-P. Aubin and H. Frankowska, Set-Valued Analysis, Mod. Birkhäuser Class., Springer, Boston, 2009, https://doi.org/10.1007/978-0-8176-4848-0.
  * [5] R. Balaji and H. Xu, Approximating stationary points of stochastic optimization problems in Banach space, J. Math. Anal. Appl., 347 (2008), pp. 333–343, https://doi.org/10.1016/j.jmaa.2008.06.015.
  * [6] A. Behzadan and M. Holst, Multiplication in Sobolev spaces, revisited, Ark. Mat., 59 (2021), pp. 275–306, https://doi.org/10.4310/arkiv.2021.v59.n2.a2.
  * [7] J. F. Bonnans and A. Shapiro, Perturbation Analysis of Optimization Problems, Springer Ser. Oper. Res., Springer, New York, 2000, https://doi.org/10.1007/978-1-4612-1394-9.
  * [8] J. M. Borwein and A. S. Lewis, On the convergence of moment problems, Trans. Amer. Math. Soc., 325 (1991), pp. 249–271, https://doi.org/10.2307/2001670.
  * [9] J. M. Borwein and A. S. Lewis, Strong rotundity and optimization, SIAM J. Optim., 4 (1994), pp. 146–158, https://doi.org/10.1137/0804008.
  * [10] J. M. Borwein and J. D. Vanderwerff, Convex functions: Constructions, Characterizations and Counterexamples, Encyclopedia Math. Appl. 109, Cambridge University Press, Cambridge, 2010, https://doi.org/10.1017/CBO9781139087322.
  * [11] A. Borzì, E.-J. Park, and M. Vallejos Lass, Multigrid optimization methods for the optimal control of convection-diffusion problems with bilinear control, J. Optim. Theory Appl., 168 (2016), pp. 510–533, https://doi.org/10.1007/s10957-015-0791-z.
  * [12] J. V. Burke, X. Chen, and H. Sun, The subdifferential of measurable composite max integrands and smoothing approximation, Math. Program., 181 (2020), pp. 229–264, https://doi.org/10.1007/s10107-019-01441-9.
  * [13] E. Casas, R. Herzog, and G. Wachsmuth, Optimality conditions and error analysis of semilinear elliptic control problems with $L^{1}$ cost functional, SIAM J. Optim., 22 (2012), pp. 795–820, https://doi.org/10.1137/110834366.
  * [14] E. Casas, D. Wachsmuth, and G. Wachsmuth, Second-order analysis and numerical approximation for bang-bang bilinear control problems, SIAM J. Control Optim., 56 (2018), pp. 4203–4227, https://doi.org/10.1137/17M1139953.
  * [15] P. Chen and O. Ghattas, Taylor approximation for chance constrained optimization problems governed by partial differential equations with high-dimensional random parameters, SIAM/ASA J. Uncertain. Quantif., 9 (2021), pp. 1381–1410, https://doi.org/10.1137/20M1381381.
  * [16] P. Chen and J. O. Royset, Performance bounds for PDE-constrained optimization under uncertainty, preprint, https://arxiv.org/abs/2110.10269, 2021.
  * [17] X. Chen, R. J.-B. Wets, and Y. Zhang, Stochastic variational inequalities: residual minimization smoothing sample average approximations, SIAM J. Optim., 22 (2012), pp. 649–673, https://doi.org/10.1137/110825248.
  * [18] F. H. Clarke, Optimization and Nonsmooth Analysis, Classics Appl. Math. 5, SIAM, Philadelphia, PA, 1990, https://doi.org/10.1137/1.9781611971309.
  * [19] H. T. H. Diem, Consistency of statistical estimators of solutions to stochastic optimization problems, J. Global Optim., 83 (2022), pp. 825–842, https://doi.org/10.1007/s10898-022-01125-3.
  * [20] R. S. Falk, Approximation of a class of optimal control problems with order of convergence estimates, J. Math. Anal. Appl., 44 (1973), pp. 28–47, https://doi.org/10.1016/0022-247X(73)90022-X.
  * [21] M. H. Farshbaf-Shaker, M. Gugat, H. Heitsch, and R. Henrion, Optimal Neumann boundary control of a vibrating string with uncertain initial data and probabilistic terminal constraints, SIAM J. Control Optim., 58 (2020), pp. 2288–2311, https://doi.org/10.1137/19M1269944.
  * [22] E. A. Feinberg, P. O. Kasyanov, and J. O. Royset, Epi-convergence of expectation functions under varying measures and integrands, J. Convex Anal., 30 (2023), pp. 917–936.
  * [23] F. Fuica and E. Otárola, A posteriori error estimates for an optimal control problem with a bilinear state equation, J. Optim. Theory Appl., 194 (2022), pp. 543–569, https://doi.org/10.1007/s10957-022-02039-6.
  * [24] C. Geiersbach and T. Scarinci, Stochastic proximal gradient methods for nonconvex problems in Hilbert spaces, Comput. Optim. Appl., 78 (2021), pp. 705–740, https://doi.org/10.1007/s10589-020-00259-y.
  * [25] C. Geiersbach and W. Wollner, Optimality conditions for convex stochastic optimization problems in Banach spaces with almost sure state constraints, SIAM J. Optim., 31 (2021), pp. 2455–2480, https://doi.org/10.1137/20M1363558.
  * [26] E. Giné and R. Nickl, Mathematical Foundations of Infinite-Dimensional Statistical Models, Camb. Ser. Stat. Probab. Math. 40, Cambridge University Press, New York, 2016, https://doi.org/10.1017/CBO9781107337862.
  * [27] R. Glowinski, Y. Song, X. Yuan, and H. Yue, Bilinear optimal control of an advection-reaction-diffusion system, SIAM Rev., 64 (2022), pp. 392–421, https://doi.org/10.1137/21M1389778.
  * [28] Z. L. Goss, D. S. Coles, S. C. Kramer, and M. D. Piggott, Efficient economic optimisation of large-scale tidal stream arrays, Appl. Energy, 295 (2021), p. 116975, https://doi.org/10.1016/j.apenergy.2021.116975.
  * [29] P. Grisvard, Elliptic Problems in Nonsmooth Domains, Classics Appl. Math. 69, SIAM, Philadelphia, PA, 2011, https://doi.org/10.1137/1.9781611972030.
  * [30] J. Guan and M. Aral, Optimal remediation with well locations and pumping rates selected as continuous decision variables, Journal of Hydrology, 221 (1999), pp. 20–42, https://doi.org/10.1016/S0022-1694(99)00079-7.
  * [31] F. Guillen-Gonzalez, E. Mallea-Zepeda, and M. A. Rodriguez-Bellido, A regularity criterion for a 3D chemo-repulsion system and its application to a bilinear optimal control problem, SIAM J. Control Optim., 58 (2020), pp. 1457–1490, https://doi.org/10.1137/18M1209891.
  * [32] L. Hertlein, A.-T. Rauls, M. Ulbrich, and S. Ulbrich, An inexact bundle method and subgradient computations for optimal control of deterministic and stochastic obstacle problems, in Non-Smooth and Complementarity-Based Distributed Parameter Systems: Simulation and Hierarchical Optimization, M. Hintermüller, R. Herzog, C. Kanzow, M. Ulbrich, and S. Ulbrich, eds., Internat. Ser. Numer. Math. 172, Birkhäuser, Cham, 2022, pp. 467–497, https://doi.org/10.1007/978-3-030-79393-7_19.
  * [33] L. Hertlein and M. Ulbrich, An inexact bundle algorithm for nonconvex nonsmooth minimization in Hilbert space, SIAM J. Control Optim., 57 (2019), pp. 3137–3165, https://doi.org/10.1137/18M1221849.
  * [34] C. Hess and R. Seri, Generic consistency for approximate stochastic programming and statistical problems, SIAM J. Optim., 29 (2019), pp. 290–317, https://doi.org/10.1137/17M1156769.
  * [35] M. Hintermüller and T. Surowiec, A bundle-free implicit programming approach for a class of elliptic MPECs in function space, Math. Program., 160 (2016), pp. 271–305, https://doi.org/10.1007/s10107-016-0983-9.
  * [36] M. Hintermüller and M. Ulbrich, A mesh-independence result for semismooth Newton methods, Math. Program., 101 (2004), pp. 151–184, https://doi.org/10.1007/s10107-004-0540-9.
  * [37] M. Hinze, R. Pinnau, M. Ulbrich, and S. Ulbrich, Optimization with PDE Constraints, Math. Model. Theory Appl. 23, Springer, Dordrecht, 2009, https://doi.org/10.1007/978-1-4020-8839-1.
  * [38] M. Hoffhues, W. Römisch, and T. M. Surowiec, On quantitative stability in infinite-dimensional optimization under uncertainty, Optim. Lett., 15 (2021), pp. 2733–2756, https://doi.org/10.1007/s11590-021-01707-2.
  * [39] T. Hytönen, J. van Neerven, M. Veraar, and L. Weis, Analysis in Banach Spaces: Martingales and Littlewood-Paley Theory, Ergeb. Math. Grenzgeb. (3) 63, Springer, Cham, 2016, https://doi.org/10.1007/978-3-319-48520-1.
  * [40] P. Jarde and M. Ulbrich, Existence of minimizers for optical flow based optimal control problems under mild regularity assumptions, Control Cybernet., 48 (2019), pp. 261–306.
  * [41] M. Kahlbacher and S. Volkwein, POD a-posteriori error based inexact SQP method for bilinear elliptic optimal control problems, ESAIM Math. Model. Numer. Anal., 46 (2012), pp. 491–511, https://doi.org/10.1051/m2an/2011061.
  * [42] O. Kallenberg, Foundations of Modern Probability, Probab. Appl., Springer, New York, NY, 2nd ed., 2002, https://doi.org/10.1007/978-1-4757-4015-8.
  * [43] Y. L. Keung and J. Zou, An efficient linear solver for nonlinear parameter identification problems, SIAM J. Sci. Comput., 22 (2000), pp. 1511–1526, https://doi.org/10.1137/S1064827598346740.
  * [44] R. N. King, K. Dykes, P. Graf, and P. E. Hamlington, Optimization of wind plant layouts using an adjoint approach, Wind Energy Science, 2 (2017), pp. 115–131, https://doi.org/10.5194/wes-2-115-2017.
  * [45] A. J. Kleywegt, A. Shapiro, and T. Homem-de Mello, The sample average approximation method for stochastic discrete optimization, SIAM J. Optim., 12 (2002), pp. 479–502, https://doi.org/10.1137/S1052623499363220.
  * [46] M. Koivu, Variance reduction in sample approximations of stochastic programs, Math. Program., 103 (2005), pp. 463–485, https://doi.org/10.1007/s10107-004-0557-0.
  * [47] D. P. Kouri, M. Staudigl, and T. M. Surowiec, A relaxation-based probabilistic approach for PDE-constrained optimization under uncertainty with pointwise state constraints, Comput. Optim. Appl., 85 (2023), pp. 441–478, https://doi.org/10.1007/s10589-023-00461-8.
  * [48] D. P. Kouri and T. M. Surowiec, Risk-averse PDE-constrained optimization using the conditional value-at-risk, SIAM J. Optim., 26 (2016), pp. 365–396, https://doi.org/10.1137/140954556.
  * [49] D. P. Kouri and T. M. Surowiec, Existence and optimality conditions for risk-averse PDE-constrained optimization, SIAM/ASA J. Uncertain. Quantif., 6 (2018), pp. 787–815, https://doi.org/10.1137/16M1086613.
  * [50] D. P. Kouri and T. M. Surowiec, Risk-averse optimal control of semilinear elliptic PDEs, ESAIM Control. Optim. Calc. Var., 26 (2020), https://doi.org/10.1051/cocv/2019061.
  * [51] E. Kreyszig, Introductory Functional Analysis with Applications, John Wiley & Sons, New York, NY, 1978.
  * [52] A. Kröner and B. Vexler, A priori error estimates for elliptic optimal control problems with a bilinear state equation, J. Comput. Appl. Math., 230 (2009), pp. 781–802, https://doi.org/10.1016/j.cam.2009.01.023.
  * [53] K. Kunisch and X.-C. Tai, Sequential and parallel splitting methods for bilinear control problems in Hilbert spaces, SIAM J. Numer. Anal., 34 (1997), pp. 91–118, https://doi.org/10.1137/S0036142993255253.
  * [54] K. Kunisch and D. Walter, Semiglobal optimal feedback stabilization of autonomous systems via deep neural network approximation, ESAIM Control Optim. Calc. Var., 27 (2021), https://doi.org/10.1051/cocv/2021009.
  * [55] S. Lang, Real and Functional Analysis, Grad. Texts in Math. 142, Springer, Berlin, 3rd ed., 1993, https://doi.org/10.1007/978-1-4612-0897-6.
  * [56] H.-J. Langen, Convergence of dynamic programming models, Math. Oper. Res., 6 (1981), pp. 493–512, https://doi.org/10.1287/moor.6.4.493.
  * [57] A. Mang, A. Gholami, C. Davatzikos, and G. Biros, PDE-constrained optimization in medical image analysis, Optim. Eng., 19 (2018), pp. 765–812, https://doi.org/10.1007/s11081-018-9390-9.
  * [58] M. Martin, S. Krumscheid, and F. Nobile, Complexity analysis of stochastic gradient methods for PDE-constrained optimal control problems with uncertain parameters, ESAIM Math. Model. Numer. Anal., 55 (2021), pp. 1599–1633, https://doi.org/10.1051/m2an/2021025.
  * [59] J. Milz, Consistency of Monte Carlo estimators for risk-neutral PDE-constrained optimization, Appl. Math. Optim., 87 (2023), https://doi.org/10.1007/s00245-023-09967-3.
  * [60] J. Milz, Sample average approximations of strongly convex stochastic programs in Hilbert spaces, Optim. Lett., 17 (2023), pp. 471–492, https://doi.org/10.1007/s11590-022-01888-4.
  * [61] J. Milz, Reliable error estimates for optimal control of linear elliptic PDEs with random inputs, preprint, https://arxiv.org/abs/2206.09160, June 2022.
  * [62] J. Milz and T. M. Surowiec, Asymptotic consistency for nonconvex risk-averse stochastic optimization with infinite dimensional decision spaces, preprint, https://arxiv.org/abs/2207.14756, July 2022.
  * [63] J. Milz and M. Ulbrich, An approximation scheme for distributionally robust PDE-constrained optimization, SIAM J. Control Optim., 60 (2022), pp. 1410–1435, https://doi.org/10.1137/20M134664X.
  * [64] J. Milz and M. Ulbrich, Sample size estimates for risk-neutral semilinear PDE-constrained optimization, preprint, https://arxiv.org/abs/2207.14755, July 2022.
  * [65] R. Nickl, S. van de Geer, and S. Wang, Convergence rates for penalized least squares estimators in PDE constrained regression problems, SIAM/ASA J. Uncertain. Quantif., 8 (2020), pp. 374–413, https://doi.org/10.1137/18M1236137.
  * [66] V. I. Norkin, Random Lipschitz functions, Cybern. Syst. Anal., (1986), pp. 226–233, https://doi.org/10.1007/BF01074785.
  * [67] T. Pennanen and M. Koivu, Epi-convergent discretizations of stochastic programs via integration quadratures, Numer. Math., 100 (2005), pp. 141–163, https://doi.org/10.1007/s00211-004-0571-4.
  * [68] C. Phelps, J. Royset, and Q. Gong, Optimal control of uncertain systems using sample average approximations, SIAM J. Control Optim., 54 (2016), pp. 1–29, https://doi.org/10.1137/140983161.
  * [69] Z. Qi, Y. Cui, Y. Liu, and J.-S. Pang, Asymptotic properties of stationary solutions of coupled nonconvex nonsmooth empirical risk minimization, Math. Oper. Res., 47 (2022), pp. 2034–2064, https://doi.org/10.1287/moor.2021.1198.
  * [70] R. T. Rockafellar and R. J.-B. Wets, Variational Analysis, Grundlehren Math. Wiss. 317, Springer, Berlin, 2009, https://doi.org/10.1007/978-3-642-02431-3.
  * [71] W. Römisch and T. M. Surowiec, Asymptotic properties of Monte Carlo methods in elliptic PDE-constrained optimization under uncertainty, preprint, https://arxiv.org/abs/2106.06347, 2021.
  * [72] A. Shapiro, Monte Carlo Sampling Methods, in Stochastic Programming, Handbooks in Oper. Res. Manag. Sci. 10, Elsevier, 2003, pp. 353–425, https://doi.org/10.1016/S0927-0507(03)10006-0.
  * [73] A. Shapiro, D. Dentcheva, and A. Ruszczyński, Lectures on Stochastic Programming: Modeling and Theory, MOS-SIAM Ser. Optim., SIAM, Philadelphia, PA, 3rd ed., 2021, https://doi.org/10.1137/1.9781611976595.
  * [74] A. Shapiro and H. Xu, Uniform laws of large numbers for set-valued mappings and subdifferentials of random functions, J. Math. Anal. Appl., 325 (2007), pp. 1390–1399, https://doi.org/10.1016/j.jmaa.2006.02.078.
  * [75] S. Stojanovic, Optimal damping control and nonlinear elliptic systems, SIAM J. Control Optim., 29 (1991), pp. 594–608, https://doi.org/10.1137/0329033.
  * [76] T. M. Surowiec, Numerical optimization methods for the optimal control of elliptic variational inequalities, in Frontiers in PDE-Constrained Optimization, H. Antil, D. P. Kouri, M.-D. Lacasse, and D. Ridzal, eds., IMA Vol. Math. Appl. 163, Springer, New York, NY, 2018, pp. 123–170, https://doi.org/10.1007/978-1-4939-8636-1_4.
  * [77] A. L. Teckentrup, R. Scheichl, M. B. Giles, and E. Ullmann, Further analysis of multilevel Monte Carlo methods for elliptic PDEs with random coefficients, Numer. Math., 125 (2013), pp. 569–600, https://doi.org/10.1007/s00211-013-0546-4.
  * [78] P. Terán, On consistency of stationary points of stochastic optimization problems in a Banach space, J. Math. Anal. Appl., 363 (2010), pp. 569–578, https://doi.org/10.1016/j.jmaa.2009.07.062.
  * [79] S. Tong, A. Subramanyam, and V. Rao, Optimization under rare chance constraints, SIAM J. Optim., 32 (2022), pp. 930–958, https://doi.org/10.1137/20M1382490.
  * [80] M. Ulbrich, Semismooth Newton Methods for Variational Inequalities and Constrained Optimization Problems in Function Spaces, MOS-SIAM Ser. Optim., SIAM, Philadelphia, PA, 2011, https://doi.org/10.1137/1.9781611970692.
  * [81] M. Vallejos and A. Borzì, Multigrid optimization methods for linear and bilinear elliptic optimal control problems, Computing, 82 (2008), pp. 31–52, https://doi.org/10.1007/s00607-008-0261-7.
  * [82] M. Winkler, Error estimates for the finite element approximation of bilinear boundary control problems, Comput. Optim. Appl., 76 (2020), pp. 155–199, https://doi.org/10.1007/s10589-020-00171-5.
  * [83] H. Xu and D. Zhang, Smooth sample average approximation of stationary points in nonsmooth stochastic optimization and applications, Math. Program., 119 (2009), pp. 371–401, https://doi.org/10.1007/s10107-008-0214-0.
  * [84] D. Yang, Y. Chang, and W. Liu, A priori error estimate and superconvergence analysis for an optimal control problem of bilinear type, J. Comput. Math., 26 (2008), pp. 471–487, https://www.jstor.org/stable/43693457.
  * [85] E. Zeidler, Nonlinear Functional Analysis and its Applications _II/B_ : Nonlinear Monotone Operators, Springer, New York, 1990, https://doi.org/10.1007/978-1-4612-0981-2.
  * [86] C. Zălinescu, Convex Analysis in General Vector Spaces, World Scientific, River Edge, NJ, 2002, https://doi.org/10.1142/9789812777096.