# Representation formulas for maximal monotone operators of type (D) in Banach
spaces whose dual spaces are strictly convex

Nguyen B. Tran, Tran N. Nguyen, Huynh M. Hien
Department of Mathematics and Statistics,
Quy Nhon University,
170 An Duong Vuong, Quy Nhon, Vietnam
<EMAIL_ADDRESS>
###### Abstract

This work deals with a maximal monotone operator $A$ of type (D) in a Banach
space whose dual space is strictly convex. We establish some representations
for the value $Ax$ at a given point $x$ via its values at nearby points of
$x$. We show that the faces of $Ax$ are contained in the set of all weak∗
convergent limits of bounded nets of the operator at nearby points of $x$,
then we obtain a representation for $Ax$ by use of this set. In addition,
representations for the support function of $Ax$ based on the minimal-norm
selection of the operator in certain Banach spaces are given.

Keywords: Maximal monotone operator, monotone operator of type (D), minimal-
norm selection, w-Kadec-Klee-property, w∗-Kadec-Klee-property, strictly convex
space

MSC 2020: 47H05, 47H04, 47N10

##  1 Introduction

The theory of maximal monotone operators has had a great development for over
sixty years. The concept of maximal monotone operators is a generalization of
subdifferentials of lower semi-continuous convex functions. Maximal monotone
operators rapidly found uses for subgradients, optimization, algorithms,
financial mathematics, and much more. The closedness of graphs of maximal
monotone operators plays a significant role in the theory of variational
analysis. It is well-known that the graph of a maximal monotone operator is
norm$\times$weak∗ sequentially closed. However, the graph of such an operator
might be not norm$\times$weak∗ closed. This occurs due to the fact that a
weak∗ convergent net might be unbounded.

In 2003, Borwein et al. [3] gave an example showing that the subdifferential
of a lower semi-continuous function in a separable Hilbert space is not
norm$\times$weak∗ closed. In 2012, Borwein and Yao [4] provided an explicit
structure formula for maximal monotone operators whose domains have nonempty
interiors. Consequently, the graphs of these operators are norm$\times$weak∗
closed. The authors considered a maximal monotone operator in a subset of the
interior of its domain in which the operator is locally bounded. The support
function of the operator at a given point $x$ equals the support function in
the set consisting of the weak∗ convergent limits of the operator at nearby
points of $x$. Recently, Hantoute and Nguyen [11] considered maximal monotone
operators in Hilbert spaces whose domains may have empty interiors and
established a relation between the faces of its values at a given point $x$
and the set of strong convergent limits of the operator at nearby points of
$x$. Then, the authors provided an explicit representation for maximal
monotone operators. These results were extended to maximal monotone operators
in reflexive Banach spaces which are locally bounded together with their dual
spaces by Khanh and Nguyen [14]. Furthermore, based on the minimal-norm
selections, the authors discovered another formula for maximal monotone
operators in that class of spaces.

The main approach in [11, 14] is that the authors used Minty surjectivity
theorem in reflective spaces. A class of maximal monotone operators in
nonreflexive Banach spaces which share several properties with these operators
in reflexive Banach spaces was introduced by Gossez in 1971 and named ‘type
(D)’; see [9]. A typical property of such a maximal monotone operator is that
the range of the sum of it and the duality mapping is dense in the dual space.
In 1992, Fitzpatrick and Phelps [7] defined a class of maximal monotone
operators which includes the class of type (D) and was later called ‘type
(FP)’. Four years later, Simons [19] introduced the class (NI) which was
seemed to generalize the class of type (D). However, in 2010, Marques and
Svaiter [12] showed that these two classes are identical. One year later,
Bauschke et al. [2] discovered that these three of types coincide.

In this paper we consider a maximal monotone operator of type (D) in a
nonreflexive Banach space whose dual space is strictly convex and try to
extend the results in [11, 14]. We represent the value of the operator at a
point $x$ via the set consisting of weak∗ convergent limits of its bounded
nets at nearby points of $x$. We introduce the concepts ‘w-Kadec-Klee
property’ and ‘w∗-Kadec-Klee property’ for Banach spaces and their dual
spaces, respectively. These classes of spaces include locally uniformly convex
spaces. Respective results are given for the case that $X$ has the w-Kadec-
Klee property and $X^{*}$ has the w∗-Kadec-Klee property.

The paper is organized as follows. In Section 2 we introduce the necessary
background material which is well-known in principle [1, 16, 20]. Since the
field requires a substantial amount of notation and concepts, we nevertheless
provide some details. In Section 3 we consider a maximal monotone operator $A$
of type (D) in a nonreflexive Banach space whose dual space is strictly
convex. In Theorem 3.1 we represent the value $Ax$ via the values of $A$ at
nearby points of $x$. Theorem 3.3 deals with lower semicontinuous convex
functions. In the final section we give a representation formula for the
support function of the value of a maximal monotone operator in a nonreflexive
Banach space whose dual space has the w∗-Kadec-Klee property by means of its
minimal-norm selections.

##  2 Preliminaries

Throughout the present paper, $(X,\|\cdot\|)$ is a real Banach space and
$X^{*},X^{**}$ are its dual and bidual spaces, respectively. The spaces $X$
and $X^{*}$ are paired by $\langle\cdot,\cdot\rangle$. We denote by
$\rightarrow$, $\overset{\operatorname{w}}{\to}$, and
$\overset{\operatorname{w}^{*}}{\rightharpoonup}$ the norm convergence, weak
convergence, and weak∗ convergence of nets, respectively.

We first recall some geometric properties of real Banach spaces.

######  Definition 2.1.

Let $X$ be a real Banach space.

  * (i)

$X$ is called _strictly convex_ if for $x,y\in X$, $x\neq y$ with
$\|x\|=\|y\|=1$, then $\|x+y\|<2.$

  * (ii)

$X$ is called _locally uniformly convex_ if, for every $x\in X$ with $\|x\|=1$
and $\varepsilon>0$, there exists $\delta(x,\varepsilon)>0$ such that for any
$y\in X$ with $\|y\|=1$, $\|x-y\|>\varepsilon$, then $\|x+y\|<2-\delta.$

It is clear that a locally uniformly convex space is strictly convex. The
duality mappings of $X$ and $X^{*}$ are respectively defined by

$\displaystyle J(x)$ $\displaystyle:=\\{x^{*}\in
X^{*}:\,\frac{1}{2}\|y\|^{2}\geq\frac{1}{2}\|x\|^{2}+\langle
x^{*},y-x\rangle,\forall y\in X\\},$ (2.1) $\displaystyle J^{*}(x^{*})$
$\displaystyle:=\\{x^{**}\in X^{**}:\langle
x^{**},x^{*}\rangle=\|x^{*}\|^{2}=\|x^{**}\|^{2}\\}.$ (2.2)

For $x\in X$, the set $J(x)$ is nonempty and closed convex. If $X^{*}$ is
strictly convex, then $J(x)$ is a singleton.

Let $C$ be a nonempty subset of $X$. We denote by
$\overline{C},\operatorname{co}C,\operatorname{int}C$, and
$\operatorname{bd}C$ the _closure_ , _convex hull_ , _interior_ , and
_boundary_ of $C$, respectively. For $x\in\overline{C}$, the _normal cone_ of
$C$ at $x$ is defined by

$N_{C}(x):=\\{x^{*}\in X^{*}:\langle x^{*},y-x\rangle\leq 0,\forall y\in
C\\}.$

It is well-known that $N_{C}(x)$ is closed convex. The _support function_ of
$C$ is the function from $X^{*}$ to
$\overline{\mathbb{R}}:=\mathbb{R}\cup\\{+\infty\\}$ defined by

$\sigma_{C}(x^{*})=\underset{x\in C}{\sup}\,\langle x^{*},x\rangle.$

The support function $\sigma_{K}:X\to\overline{\mathbb{R}}$ of a nonempty
subset $K\subset X^{*}$ is defined analogously.

Let $f:X\to\overline{\mathbb{R}}$ be a convex function. The _domain_ of $f$,
denoted by $\operatorname{dom}f$, is the set of points $x\in X$ such that
$f(x)<+\infty$. We call $f$ _lower semicontinuous_ on $X$ if for all $x\in X$

$\underset{y\to x}{\lim\inf}f(y)\geq f(x).$

Let $f$ be a lower semicontinuous convex function on $X$,
$x\in\operatorname{dom}f$ and $\varepsilon\geq 0$. A linear continuous
function $x^{*}\in X^{*}$ is called a _$\varepsilon$ -subgradient_ of $f$ at
$x$ if

$f(y)\geq f(x)+\langle x^{*},y-x\rangle-\varepsilon,\,\forall y\in X.$

The set of all $\varepsilon$-subgradients of $f$ at $x$ is denoted by
$\partial_{\varepsilon}f(x)$ and is called the $\varepsilon$-subdifferential
of $f$ at $x$. For the case of $f(x)=+\infty$ we write
$\partial_{\varepsilon}f(x)=\varnothing.$

As usual, $\partial_{0}f(x)$ is denoted again by $\partial f(x)$ and is called
the subdifferential of $f$ at $x$. In the case of $f(x)=\frac{1}{2}\|x\|^{2}$
for $x\in X$, we denote by $J_{\varepsilon}(x)$ the
$\varepsilon$-subdifferential of $f$ at $x$. From (2.1), $J(x)=\partial f(x)$.

Let $A:X\rightrightarrows X^{*}$ be a set-valued operator. The _domain_ ,
_graph_ , and _range_ of $A$ are given, respectively, by

$\displaystyle{D}(A)$ $\displaystyle:=\\{x\in X:Ax\neq\varnothing\\},$
$\displaystyle{Gr}(A)$ $\displaystyle:=\\{(x,x^{*})\in X\times X^{*}:x^{*}\in
Ax\\},$ $\displaystyle{R}(A)$ $\displaystyle:=\underset{x\in X}{\bigcup}Ax.$

######  Definition 2.2.

(i) An operator $A$ is called _monotone_ if

$\langle x^{*}-y^{*},x-y\rangle\geq 0,\mbox{ for
}(x,x^{*}),(y,y^{*})\in{Gr}(A).$

(ii) A monotone operator $A$ is called _maximal monotone_ if ${Gr}(A)$ is not
properly contained in the graph of any other monotone operators.

If $A$ is a maximal monotone operator, then for every $x\in X,$ $Ax$ is
closed, convex (see [1, Proposition 2.1]) and satisfies the equality

$Ax=Ax+N_{\overline{{D}(A)}}(x).$ (2.3)

Let $x\in{D}(A)$. The _minimal-norm selection_ of $Ax$ is the set

$A^{\circ}x:=\\{x^{*}\in Ax:\|x^{*}\|=\underset{y^{*}\in
Ax}{\inf}\|y^{*}\|\\}.$

If $X^{*}$ is strictly convex, then $A^{\circ}$ is a singleton.

We call a net $(x_{i})_{i\in I}$ in $X$ _eventually bounded_ if there exists
$M>0$ and $i_{0}\in I$ such that

$\|x_{i}\|<M,\,\,\forall i\succeq_{I}i_{0}.$

According to [4, Fact 3.5], if $(x_{i},x^{*}_{i})_{i\in I}$ is a net in
${Gr}(A)$ such that $x_{i}\to x$,
$x^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*}$, and
$(x^{*}_{i})_{i\in I}$ is eventually bounded, then $(x,x^{*})\in{Gr}(A)$.

The next concept was introduced by Gossez ([10]).

######  Definition 2.3.

Let $X$ be a Banach space and $A:X\rightrightarrows X^{*}$ be a maximal
monotone operator. The Gossez’s monotone closure of $A$ is defined by

$\overline{A}^{g}=\Big{\\{}(x^{**},x^{*})\in X^{**}\times X^{*}:\langle
x^{*}-y^{*},x^{**}-y\rangle\geq 0,\forall(y,y^{*})\in{Gr}(A)\Big{\\}}.$

The operator $A$ is called of _type (D)_ if for any
$(x^{**},x^{*})\in\overline{A}^{g}$ there exists a bounded net
$(x_{i},x^{*}_{i})_{i\in I}$ in ${Gr}(A)$ such that $x_{i}$ converges to
$x^{**}$ in $\tau(X^{**},X^{*})$ and $x^{*}_{i}$ strongly converges to
$x^{*}$.

The following theorem is a characterization of maximal monotone operators of
type (D).

######  Theorem 2.1 ([12, Theorem 4.4]).

Let $X$ be a Banach space and $A:X\rightrightarrows X^{*}$ be a maximal
monotone operator. Then $A$ is of type (D) if and only if

${R}(\lambda
J_{\varepsilon}(\cdot-x)+A)=X^{*},\forall\lambda>0,\varepsilon>0,\forall x\in
X.$

Finally, let $F:X\rightrightarrows X^{*}$ be a set-valued mapping. For $x\in
X$ we define some types of convergent limits

$\begin{split}\operatorname{w}^{*}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Fy:=\Big{\\{}x^{*}&\in X^{*}:\exists\text{ a net
}(x_{i},x^{*}_{i})_{i\in I}\in{Gr}(F)\text{ with }\\\ &x_{i}\neq x,\forall
i\in I,x_{i}\to
x,\,\,x^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*}\Big{\\}},\\\
\operatorname{bw}^{*}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Fy:=\Big{\\{}x^{*}&\in X^{*}:\exists\text{ a net
}(x_{i},x^{*}_{i})_{i\in I}\in{Gr}(F)\text{ and }M>0\text{ with }\\\
&x_{i}\neq x,\forall i\in I,x_{i}\to x,\|x^{*}_{i}\|\leq M,\,\forall i\in
I,\,\,x^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*}\Big{\\}},\\\
{\operatorname{s}-}\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Fy:=\Big{\\{}x^{*}&\in X^{*}:\exists\text{ a net
}(x_{i},x^{*}_{i})_{i\in I}\in{Gr}(F)\text{ with }x_{i}\neq x,\forall i\in
I,\\\ &x_{i}\to x,\,x^{*}_{i}\to x^{*}\Big{\\}}.\end{split}$

##  3 Representation formulas for maximal monotone operators of type (D)

In this section we consider a maximal monotone operator $A$ of type (D) in a
Banach space $X$ whose dual space is strictly convex. We represent the value
$Ax$ via the values at nearby points of $x$. We define new properties for $X$
and $X^{*}$, which are generalizations of the Kadec-Klee property, named the
w-Kadec-Klee and $\operatorname{w}^{*}$-Kadec-Klee, respectively. A respective
representation was obtained in the case that $X^{*}$ has the
$\operatorname{w}^{*}$-Kadec-Klee property.

We first observe a property of the minimal-norm selection.

######  Lemma 3.1.

Let $A:X\rightrightarrows X^{*}$ be a maximal monotone operator. Then the
minimal-norm $A^{\circ}x$ is nonempty for all $x\in{D}(A).$

###### Proof.

Let $x\in{D}(A)$. Then $\alpha:=\inf_{x^{*}\in Ax}\|x^{*}\|<+\infty$. For
every $n\in\mathbb{N}$, we take $x^{*}_{n}\in Ax$ such that

$\|x^{*}_{n}\|\leq\alpha+\frac{1}{n}.$ (3.1)

By Banach-Alaoglu theorem, there exist a subnet $(x^{*}_{i})_{i\in I}$ of
$(x^{*}_{n})_{n\in\mathbb{N}}$ and $x^{*}\in X^{*}$ such that
$x^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*}.$ It follows
from inequality (3.1) that
$\|x^{*}\|\leq\displaystyle\liminf_{i}\|x^{*}_{i}\|\leq\alpha.$ On the other
hand, also from inequality (3.1), the subnet $(x^{*}_{i})_{i\in I}$ is
eventually bounded. Owing to [3, Section 2], we have $x^{*}\in Ax$. This
implies $\|x^{*}\|\geq\alpha$ and hence $\|x^{*}\|=\alpha$, so we get
$x^{*}\in A^{\circ}x$, proving the lemma. ∎

The following properties of $J_{\varepsilon}$ are useful later.

######  Lemma 3.2.

Let $x\in X$ and $\varepsilon\geq 0,\lambda>0$. Then

  * (i)

$J_{\varepsilon}(-x)=-J_{\varepsilon}(x),\,\,\lambda
J_{\varepsilon}(x)=J_{\lambda^{2}\varepsilon}(\lambda x);$

  * (ii)

$\underset{x^{*}\in
J_{\varepsilon}(x)}{\sup}|\|x\|-\|x^{*}\||\leq\sqrt{2\varepsilon};$

  * (iii)

$\underset{x^{*}\in J_{\varepsilon}(x)}{\sup}|\langle
x^{*},x\rangle-\|x\|^{2}|\leq\sqrt{\varepsilon}(1+\frac{1}{2}\|x\|^{2}).$

###### Proof.

(i) Let $x\in X,\varepsilon\geq 0,\lambda>0$ and $x^{*}\in
J_{\varepsilon}(-x).$ It follows from the definition of $J_{\varepsilon}$ that

$\frac{1}{2}\|y\|^{2}+\varepsilon\geq\frac{1}{2}\|x\|^{2}+\langle
x^{*},y+x\rangle,\,\forall y\in X.$ (3.2)

This implies

$\frac{1}{2}\|y\|^{2}+\varepsilon\geq\frac{1}{2}\|x\|^{2}+\langle-x^{*},y-x\rangle,\forall
y\in X,$

which gives $-x^{*}\in J_{\varepsilon}(x)$. Hence, we have the inclusion

$J_{\varepsilon}(-x)\subset-J_{\varepsilon}(x),\forall x\in X.$

Switching the roles of $x$ and $-x$, we also get $-J_{\varepsilon}(x)\subset
J_{\varepsilon}(-x)$, implying the former. For the latter, multiplying both
side of (3.2) by $\lambda^{2}$, we get

$\frac{1}{2}\|\lambda y\|^{2}+\lambda^{2}\varepsilon\geq\frac{1}{2}\|\lambda
x\|^{2}+\langle\lambda x^{*},\lambda y-\lambda x\rangle,\forall y\in X.$

This yields

$\frac{1}{2}\|y\|^{2}+\lambda^{2}\varepsilon\geq\frac{1}{2}\|\lambda
x\|^{2}+\langle\lambda x^{*},y-\lambda x\rangle,\forall y\in X.$ (3.3)

Then $\lambda x^{*}\in J_{\lambda^{2}\varepsilon}(\lambda x)$ implies that

$\lambda J_{\varepsilon}(x)\subset J_{\lambda^{2}\varepsilon}(\lambda
x),\,\,\forall x\in X,\forall\varepsilon\geq 0,\forall\lambda>0.$ (3.4)

This leads to

$\frac{1}{\lambda}J_{\lambda^{2}\varepsilon}(\lambda x)\subset
J_{(\frac{1}{\lambda})^{2}(\lambda^{2}\varepsilon)}(\frac{1}{\lambda}\lambda
x)=J_{\varepsilon}(x),\,\,\forall x\in X,\forall\varepsilon\geq
0,\forall\lambda>0.$ (3.5)

Combining (3.4) and (3.5), we get the latter.

(ii) This follows from

$|\|x^{*}\|-\|x\||\leq\sqrt{2\varepsilon}$

due to [13].

(iii) This is obvious for $\varepsilon=0.$ For $\varepsilon>0$ the definition
of $J_{\varepsilon}$ implies that

$\frac{1}{2}\|(1+t)x\|^{2}\geq\frac{1}{2}\|x\|^{2}+t\langle
x^{*},x\rangle-\varepsilon,\,\,\forall t\in\mathbb{R}.$

This is equivalent to

$t(\langle
x^{*},x\rangle-\|x\|^{2})\leq\varepsilon+\frac{t^{2}}{2}\|x\|^{2},\,\,\forall
t\in\mathbb{R}.$

In particular, for $t=\sqrt{\varepsilon}$ we get

$|\langle
x^{*},x\rangle-\|x\|^{2}|\leq\sqrt{\varepsilon}\left(1+\frac{1}{2}\|x\|^{2}\right),$

which proves (iii). ∎

The following lemma is an extension of [1, Proposition 2.2] on the Yosida
approximation of maximal monotone operators in reflexive Banach spaces.

######  Lemma 3.3.

Let $X$ be a Banach space whose dual space is strictly convex,
$A:X\rightrightarrows X^{*}$ be a maximal monotone operator of type (D). For
every $x\in{D}(A)$ and a sequence $\lambda_{n}\downarrow 0$ there exist a
subnet $(\lambda_{i})_{i\in I}$ of $(\lambda_{n})_{n\in\mathbb{N}}$, a net
$(x_{i})_{i\in I}\subset{D}(A)$ converging to $x$, a net $(z^{*}_{i})_{i\in
I}\subset X^{*}$ with $z^{*}_{i}\in J_{\lambda^{6}_{i}}(x-x_{i})$ such that
$(\lambda^{-1}_{i}z^{*}_{i})_{i\in I}$ is bounded and

$\lambda^{-1}_{i}z^{*}_{i}\in
Ax_{i},\,\,\,\,\|\lambda^{-1}_{i}z^{*}_{i}\|\to\|A^{\circ}x\|,\,\,\,\lambda_{i}^{-1}z^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}A^{\circ}x.$
(3.6)

###### Proof.

Let $x\in{D}(A)$ and a sequence $\lambda_{n}\downarrow 0$. By Theorem 2.1, for
every $n\in\mathbb{N}$ we can find $x_{n}\in X$ such that

$0\in\lambda^{-1}_{n}J_{\lambda^{6}_{n}}(x_{n}-x)+Ax_{n}.$

By Lemma 3.2 (i), there exists a sequence $(z^{*}_{n})_{n\in\mathbb{N}}\subset
X^{*}$ such that $z^{*}_{n}\in J_{\lambda^{6}_{n}}(x-x_{n})$ and

$\lambda^{-1}_{n}z^{*}_{n}\in Ax_{n},\,\,\forall n\in\mathbb{N}.$ (3.7)

Due to the monotonicity of $A$ and Lemma 3.1, the set $A^{\circ}(x)$ is
nonempty and

$\langle\lambda^{-1}_{n}z^{*}_{n}-A^{\circ}x,x-x_{n}\rangle\leq 0,$ (3.8)

which implies that

$\lambda^{-1}_{n}\langle z^{*}_{n},x-x_{n}\rangle\leq\langle
A^{\circ}x,x-x_{n}\rangle\leq\|A^{\circ}x\|\left\|x_{n}-x\right\|.$ (3.9)

Since $z^{*}_{n}\in J_{\lambda^{6}_{n}}(x-x_{n})$, Lemma 3.2 (iii) gives us

$\langle
z^{*}_{n},x-x_{n}\rangle\geq\|x-x_{n}\|^{2}-\lambda^{3}_{n}\left(1+\frac{1}{2}\|x-x_{n}\|^{2}\right)\geq\|x-x_{n}\|^{2}-\lambda^{3}_{n}(1+\|x-x_{n}\|^{2}).$

It follows from (3.9) that

$\lambda^{-1}_{n}\Big{(}\|x-x_{n}\|^{2}-\lambda^{3}_{n}(1+\|x-x_{n}\|^{2})\Big{)}\leq\|A^{\circ}x\|\left\|x-x_{n}\right\|.$

Multiplying both sides by $\lambda^{-1}_{n}$ and then adding $\lambda_{n}$, we
get

$(1-\lambda^{3}_{n})(\lambda^{-1}_{n}\|x-x_{n}\|)^{2}\leq\|A^{\circ}x\|(\lambda^{-1}_{n}\|x-x_{n}\|)+\lambda_{n},$

which yields

$\lambda^{-1}_{n}\|x-x_{n}\|\leq\frac{\|A^{\circ}x\|+\sqrt{\|A^{\circ}x\|^{2}+4\lambda_{n}(1-\lambda^{3}_{n})}}{2(1-\lambda^{3}_{n})}$
(3.10)

for sufficiently large $n$, and hence

$\underset{n\to\infty}{\lim\sup}\,\,\lambda^{-1}_{n}\|x-x_{n}\|\leq\|A^{\circ}x\|.$
(3.11)

Furthermore, by Lemma 3.2 (ii),

$\|z^{*}_{n}\|\leq\|x-x_{n}\|+\sqrt{2}\lambda^{3}_{n},\,\,\forall
n\in\mathbb{N}.$

Together with (3.11), we get

$\underset{n\to\infty}{\lim\sup}\,\,\|\lambda^{-1}_{n}z^{*}_{n}\|\leq\underset{n\to\infty}{\lim\sup}\,\,(\lambda^{-1}_{n}\|x-x_{n}\|+\sqrt{2}\lambda^{2}_{n})\leq\|A^{\circ}x\|.$
(3.12)

This shows that $(\lambda^{-1}_{n}z^{*}_{n})_{n\in\mathbb{N}}$ is bounded.
Using Banach-Alaoglu theorem, there exist $y^{*}\in X^{*}$ and a subnet
$(\lambda^{-1}_{i}z^{*}_{i})_{i\in I}$ of
$(\lambda^{-1}_{n}z^{*}_{n})_{n\in\mathbb{N}}$ satisfying

$\lambda^{-1}_{i}z^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}y^{*}.$
(3.13)

It follows from (3.12) that

$\|y^{*}\|\leq\underset{i}{\lim\inf}\,\|\lambda^{-1}_{i}z^{*}_{i}\|\leq\underset{i}{\lim\sup}\,\|\lambda^{-1}_{i}z^{*}_{i}\|\leq\|A^{\circ}x\|.$
(3.14)

On the other hand, owing to (3.10), $x_{n}\to x\text{ as }n\to\infty$, and
hence $x_{i}\to x.$ It follows from (3.12) and (3.13) that
$(\lambda^{-1}_{i}z^{*}_{i})_{i\in I}$ is bounded and

$\lambda^{-1}_{i}z^{*}_{i}\in
Ax_{i},\lambda^{-1}_{i}z^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}y^{*},\,\,x_{i}\to
x.$ (3.15)

Then $y^{*}\in Ax$ by the maximal monotonicity of $A$. From (3.14), we get
$y^{*}=A^{\circ}x$ since $A^{\circ}x$ is a singleton set due to the strict
convexity of $X^{*}$. Hence, we obtain from (3.14) that
$\|\lambda^{-1}_{i}z^{*}_{i}\|\to\|A^{\circ}x\|$, completing the proof. ∎

Recall that if $A$ is a maximal monotone operator, then $Ax$ is convex and
closed in $X^{*}$ for $x\in{D}(A)$. In order to represent $Ax$ by means of the
values at nearby points of $x$, we establish a relation between the values at
nearby points of $x$ and the faces $A(x;v)$ of $Ax$ with respect to vector
$v\in X\setminus\\{0\\},$ which is defined by

$A(x;v):=\Big{\\{}y^{*}\in Ax:\langle y^{*},v\rangle=\underset{x^{*}\in
Ax}{\sup}\langle x^{*},v\rangle\Big{\\}}.$

This relation depends on the properties of $X$. Next we introduce a new class
of Banach spaces.

######  Definition 3.1.

Let $X$ be a Banach space.

  * (a)

$X$ is said to have the w-Kadec-Klee property if for any net $(x_{i})_{i\in
I}$, $x_{i}\overset{\operatorname{w}}{\to}x$ and $\|x_{i}\|\to\|x\|$, then
$x_{i}\to x$.

  * (b)

$X^{*}$ is said to have the $w^{*}$-Kadec-Klee property if for any net
$(x^{*}_{i})_{i\in I}$,
$x^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*}$ and
$\|x^{*}_{i}\|\to\|x^{*}\|$, then $x^{*}_{i}\to x^{*}$.

The following lemma shows that the class of Banach spaces with $w^{*}$-Kadec-
Klee property includes locally uniformly convex Banach spaces.

######  Lemma 3.4.

Let $X$ be a Banach space. Then,

(a) If $X$ is locally uniformly convex, then $X$ has w-Kadec-Klee property;

(b) If $X^{*}$ is locally uniformly convex, then $X^{*}$ has $w^{*}$-Kadec-
Klee property.

See [21, Proposition 3.7.6] for a proof of (a). The proof of (b) is similar.

The next result discovers a relation between the faces of $Ax$ and the set
consisting of the weak∗ convergent limits of bounded nets of $A$ at nearby
points of $x$. In what follows $j_{\lambda}(w)$ means an element of
$J_{\lambda}(w)$.

######  Proposition 3.1.

Let $X$ be a Banach space whose dual space is strictly convex and let
$A:X\rightrightarrows X^{*}$ be a maximal monotone operator of type (D). Then
for $x\in{D}(A)$ and $v\in X\setminus\\{0\\}$

$A(x;v)\subset\operatorname{bw}^{*}{-}\underset{\lambda\to
0\atop\underset{j_{\lambda}(w)\overset{\operatorname{w}^{*}}{\rightharpoonup}J(v)}{\|j_{\lambda}(w)\|\to\|J(v)\|}}{\operatorname{Lim}\sup}\,\,A(x+\lambda
w).$ (3.16)

If in addition that $X^{*}$ has the $w^{*}$-Kadec-Klee property, then for
$x\in{D}(A)$ and $v\in X\setminus\\{0\\}$

$A(x;v)\subset\operatorname{s}-\underset{\lambda\to 0\atop j_{\lambda}(w)\to
J(v)}{\operatorname{Lim}\sup}A(x+\lambda w).$ (3.17)

###### Proof.

Let $x\in{D}(A)$, $v\in X\setminus\\{0\\}$ and $x^{*}\in A(x;v).$ Since
$X^{*}$ is strictly convex, $J(v)$ is a singleton set. We consider a maximal
monotone operator defined by

$By:=Ay-J(v)-x^{*},\,\,\forall\,y\in X.$

It is obvious that ${D}(B)={D}(A)$, and hence $x\in{D}(B)$. We first claim
that

$B^{\circ}x=-J(v).$ (3.18)

Indeed, observe that

$-J(v)=x^{*}-J(v)-x^{*}\in Ax-J(v)-x^{*}=Bx,$ (3.19)

Moreover, for $y^{*}\in Ax$, $\langle x^{*}-y^{*},v\rangle\geq 0$ since
$x^{*}\in A(x;v)$. This implies

$\begin{split}\|-J(v)\|&=\|v\|^{-1}\langle J(v),v\rangle\\\
&\leq\|v\|^{-1}\langle J(v)+x^{*}-y^{*},v\rangle\\\
&\leq\|v\|^{-1}\|J(v)+x^{*}-y^{*}\|\|v\|\\\
&=\|y^{*}-J(v)-x^{*}\|.\end{split}$

Now (3.18) follows from (3.19).

Next, we apply Lemma 3.3 to have nets $(\lambda_{i})_{i\in I}\to 0$,
$(x_{i})_{i\in I}\subset{D}(B)$ with $x_{i}\to x$ and $(z^{*}_{i})_{i\in
I}\subset X^{*}$ with $z^{*}_{i}\in J_{\lambda^{6}_{i}}(x-x_{i})$ such that
$(\lambda^{-1}_{i}z^{*}_{i})_{i\in I}$ is bounded and

$\lambda^{-1}_{i}z^{*}_{i}\in
Bx_{i},\,\,\,\,\lambda^{-1}_{i}z^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}B^{\circ}x=-J(v),\|\lambda^{-1}_{i}z^{*}_{i}\|\to\|J(v)\|.$

Define $w_{i}:=\lambda^{-1}_{i}(x_{i}-x),i\in I$. We then have

$(\lambda^{-1}_{i}z^{*}_{i}+J(v)+x^{*})_{i\in I}\,\,\text{ is bounded},$
$\lambda^{-1}_{i}z^{*}_{i}+J(v)+x^{*}\in
Bx_{i}+J(v)+x^{*}=Ax_{i}=A(x+\lambda_{i}w_{i}),$ (3.20)
$\lambda^{-1}_{i}z^{*}_{i}+J(v)+x^{*}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*},$
(3.21)
$-\lambda_{i}^{-1}z^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}J(v),\|-\lambda_{i}^{-1}z^{*}_{i}\|\to\|J(v)\|.$
(3.22)

It follows from Lemma 3.2 (i) that

$-\lambda^{-1}_{i}z^{*}_{i}\in
J_{\lambda^{4}_{i}}(\lambda^{-1}_{i}(x_{i}-x))\subset
J_{\lambda_{i}}(\lambda^{-1}_{i}(x_{i}-x))=J_{\lambda_{i}}(w_{i}).$ (3.23)

Thus

$x^{*}\in{\rm bw}^{*}-\underset{t\to
0\atop\underset{j_{t}(w)\overset{\operatorname{w}^{*}}{\rightharpoonup}J(v)}{\|j_{t}(w)\|\to\|J(v)\|}}{\operatorname{Lim}\sup}\,\,A(x+tw),$

and hence (3.16) holds.

Now, suppose in addition that $X^{*}$ has the $\operatorname{w}^{*}$-Kadec-
Klee property and $x^{*}\in A(x;v)$ with $x\in{D}(A)$ and $v\neq 0$. By the
above result, we find nets $(\lambda_{i})_{i\in I}\subset\mathbb{R}_{+},$
$(x_{i})_{i\in I}\subset{D}(A)$, $(w_{i})_{i\in I}\subset X$ and
$(z^{*}_{i})_{i\in I}\in X^{*}$ satisfying (3.20)-(3.22). It follows from
(3.22) that $-\lambda_{i}^{-1}z^{*}_{i}\to J(v)$, and hence
$\lambda^{-1}_{i}z^{*}_{i}+J(v)+x^{*}\to x^{*}.$ This leads to

$x^{*}\subset\operatorname{s}-\underset{t\to 0\atop j_{t}(w)\to
J(v)}{\operatorname{Lim}\sup}A(x+tw),$

completing the proof. ∎

######  Corollary 3.1.

Let $X$ be a Banach space whose dual space is strictly convex and let
$A:X\rightrightarrows X^{*}$ be a maximal monotone operator of type (D). Then,
for $x\in{D}(A)$ and $v\in X\setminus\\{0\\}$

$A(x;v)\subset{\rm bw}^{*}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay.$ (3.24)

If in addition that $X^{*}$ has the ${w}^{*}$-Kadec-Klee property, then for
$x\in{D}(A)$ and $v\in X\setminus\\{0\\},$

$A(x;v)\subset\operatorname{s}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay.$ (3.25)

###### Proof.

Let $x\in{D}(A)$, $v\in X\setminus\\{0\\}$ and $x^{*}\in A(x;v)$. By (3.16),
there exist nets $(\lambda_{i})_{i\in I}\to 0,$ $(w_{i})_{i\in I}\subset X,$
$(z^{*}_{i})_{i\in I}\subset X^{*}$ with $z^{*}_{i}\in
J_{\lambda_{i}}(w_{i})$,
$z^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}J(v),$
$\|z^{*}_{i}\|\to\|J(v)\|$, $(x^{*}_{i})_{i\in I}\subset X^{*}$ with
$x^{*}_{i}\in A(x+\lambda_{i}w_{i})$, such that

$(x^{*}_{i})_{i\in I}\text{ is bounded and
}x^{*}_{i}\overset{\operatorname{w}^{*}}{\rightharpoonup}x^{*}.$

Next we verify that $x+\lambda_{i}w_{i}\to x$, which would imply

$x^{*}\in{\rm bw}^{*}-\underset{y\to x,y\neq x}{\operatorname{Lim}\sup}\,Ay$

and hence (3.24) holds. By Lemma 3.2 (ii) and $z_{i}\in
J_{\lambda_{i}}(w_{i}),$ we have
$\|w_{i}\|\leq\|z^{*}_{i}\|+\sqrt{2\lambda_{i}}$ for $i\in I$. Therefore

$\underset{i\in I}{\limsup}\|w_{i}\|\leq\underset{i\in
I}{\sup}(\|z^{*}_{i}\|+\sqrt{2\lambda_{i}})=\|J(v)\|=\|v\|$

implies $x+\lambda_{i}w_{i}\to x.$ The same argument applies to yield (3.25).
∎

Now we are in a position to derive the first representation for the value of a
maximal monotone operator of type (D) at a point in its domain.

######  Theorem 3.1.

Let $X$ be a Banach space whose dual space is strictly convex, and let
$A:X\rightrightarrows X^{*}$ be a maximal monotone operator of type (D). Then
for $x\in X$

$Ax=\operatorname{co}\overline{\\{\operatorname{bw}^{*}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\}}+N_{\overline{{D}(A)}}(x).$ (3.26)

If in addition that $X^{*}$ has the ${w}^{*}$-Kadec-Klee property, then for
$x\in X$

$Ax=\operatorname{co}\\{\operatorname{s}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\}+N_{\overline{{D}(A)}}(x).$ (3.27)

###### Proof.

Let $x\in X$ and

$K:=\operatorname{bw}^{*}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay.$

Owing to the maximal monotonicity of $A$, $K\subset Ax$ and the fact that $Ax$
is closed convex and satisfies $Ax=Ax+N_{\overline{{D}(A)}}(x)$, we deduce the
inclusion

$\operatorname{co}\overline{\\{\operatorname{bw}^{*}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\}}+N_{\overline{{D}(A)}}(x)\subset Ax.$

To justify the reverse inclusion, we will show the following two inclusions

$Ax\subset\operatorname{co}(\operatorname{bd}Ax)+N_{\overline{{D}(A)}}(x)$
(3.28)

and

$\operatorname{bd}Ax\subset\overline{K}.$ (3.29)

For the case $x\notin{D}(A)$, one has $K=Ax=\operatorname{bd}Ax=\varnothing$
and so the above inclusions are trivial. Let $x\in{D}(A).$ To clarify (3.28),
we pick any $x^{*}\in Ax$. If $x^{*}\in\operatorname{bd}Ax$, then

$x^{*}\in\operatorname{bd}Ax\subset\operatorname{co}(\operatorname{bd}Ax)+N_{\overline{{D}(A)}}(x).$
(3.30)

If $x^{*}\in\operatorname{int}Ax$, we take $x^{*}_{0}\in\operatorname{bd}Ax$
and define

$\bar{\rho}:=\sup\big{\\{}\rho>0:x^{*}+t(x^{*}-x^{*}_{0})\subset Ax,\forall
t\in[0,\rho)\big{\\}}>0.$

We consider two cases of $\bar{\rho}$.

Case 1: $\bar{\rho}=+\infty$. It means that $x^{*}+t(x^{*}-x^{*}_{0})\in Ax$
for every $t\geq 0$. Hence, for $(y,y^{*})\in{Gr}(A)$ by the monotonicity of
$A$, we have

$\langle x^{*}+t(x^{*}-x^{*}_{0})-y^{*},y-x\rangle\leq 0,\forall t\geq 0.$

This implies $\langle x^{*}-x^{*}_{0},y-x\rangle\leq 0$. Since this holds for
any $y\in{D}(A),$ we get $x^{*}-x^{*}_{0}\in N_{\overline{{D}(A)}}(x)$, which
implies that

$x^{*}\in
x^{*}_{0}+N_{\overline{{D}(A)}}(x)\subset\operatorname{co}(\operatorname{bd}Ax)+N_{\overline{{D}(A)}}(x).$

Case 2: $\bar{\rho}<+\infty.$ Since $Ax$ is closed,
$z^{*}:=x^{*}+\bar{\rho}(x^{*}-x^{*}_{0})\in Ax.$ Furthermore,
$z^{*}\in\operatorname{bd}Ax$ by the definition of $\bar{\rho}$. Therefore

$x^{*}=\frac{1}{1+\bar{\rho}}z^{*}+\frac{\bar{\rho}}{1+\bar{\rho}}x^{*}_{0}\in\operatorname{co}(\operatorname{bd}Ax)\subset\operatorname{co}(\operatorname{bd}Ax)+N_{\overline{{D}(A)}}(x).$

Combining the results above, we conclude that (3.28) holds.

Finally, let $x^{*}\in\operatorname{bd}Ax$. According to [15, Theorem 1],
since $Ax$ is weak∗ closed convex, there exists a sequence
$(x^{*}_{n})_{n\in\mathbb{N}}\subset\operatorname{bd}Ax$ such that
$x^{*}_{n}\to x^{*}$, where $x^{*}_{n}$ are weak∗ support points of $Ax$,
i.e., for each $x^{*}_{n}$, there exists $v_{n}\in X\setminus\\{0\\}$ such
that

$\langle x^{*}_{n},v_{n}\rangle=\sigma_{Ax}(v_{n}).$

The equality means that $x^{*}_{n}\in A(x;v_{n})$ for $n\in\mathbb{N}$. By
Corollary 3.1, $x^{*}_{n}\in K$, which implies $x^{*}\in\overline{K}$. Hence
we get (3.29) and the proof of (3.26) is complete.

Now suppose in addition that $X^{*}$ has the $\operatorname{w}^{*}$-Kadec-Klee
property. The proof of (3.27) is similar to that of (3.26), where $K$ is
replaced by the closed set $K^{\prime}:=\operatorname{s}-\underset{y\to
x,y\neq x}{\operatorname{Lim}\sup}\,Ay.$ ∎

######  Remark 3.1.

With the setting in the previous theorem, for $x\in X$

$Ax=\operatorname{co}\,\overline{\\{\operatorname{bw}^{*}-\underset{y\to
x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\}}+N_{\overline{{D}(A)}}(x)\subset\operatorname{co}\overline{\\{\operatorname{w}^{*}-\underset{y\to
x,y\neq x}{\operatorname{Lim}\sup}\,Ay\\}}+N_{\overline{{D}(A)}}(x).$ (3.31)

It is worth noting that in the case that $X$ is a Hilbert space, the above
inclusion is still strict; see [3, Example 1]. Furthermore, the inclusion
becomes equality if the nets are replaced by sequences, and by [11, Theorem
3.3]

$Ax=\operatorname{co}{\\{\operatorname{ss}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\}}+N_{\overline{{D}(A)}}(x),$

where

$\displaystyle{\operatorname{ss}-}\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay=\Big{\\{}x^{*}\in X^{*}:$
$\displaystyle\exists\,(x_{n},x^{*}_{n})_{n\in\mathbb{N}}\in{Gr}(A)\text{ with
}x_{n}\neq x,\forall n\in\mathbb{N},$ $\displaystyle x_{n}\to x,\,x^{*}_{n}\to
x^{*}\Big{\\}}.$

$\Diamond$

The next theorem is a representation for maximal monotone operators in
reflexive spaces.

######  Theorem 3.2.

Let $X$ be a reflexive Banach space and let $A:X\rightrightarrows X^{*}$ be a
maximal monotone operator. Then for $x\in X$

$Ax=\operatorname{co}\\{\operatorname{s}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\}+N_{\overline{{D}(A)}}(x).$

###### Proof.

Let $x\in X$. We first observe that the maximal monotonicity of $A$ and the
terms $\operatorname{co}\\{\operatorname{s}-\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,Ay\\},N_{\overline{{D}(A)}}(x)$ are not affected
by equivalently renorming of $X$ and $X^{*}.$ By Troyanski’s renorming theorem
([8, Theorem 5.192]), $X$ admits an equivalent norm so that both $X$ and
$X^{*}$ are locally uniformly convex. Therefore, we may assume that both $X$
and $X^{*}$ are locally uniformly convex. Then $X^{*}$ has the
$\operatorname{w}^{*}$-Kadec-Klee property due to Lemma 3.4. Since the space
$X$ is reflexive, the operator $A$ is of type (D), owing to [18, Proposition
1]. Using the second part of Theorem 3.1, we derive the conclusion of the
theorem. ∎

The next result is a representation formula for the subdifferentials of a
lower semicontinuous convex function.

######  Theorem 3.3.

Let $X$ be a Banach space whose dual space is strictly convex and let
$f:X\to\overline{\mathbb{R}}$ be a lower semicontinuous function. Then for
$x\in X$

$\partial
f(x)=\operatorname{co}\overline{\\{\operatorname{bw}^{*}-\,\,\underset{y\to
x,y\neq x}{\operatorname{Lim}\sup}\,\partial
f(y)\\}}+N_{\overline{\operatorname{dom}f}}(x).$ (3.32)

If in addition that $X^{*}$ has the ${w}^{*}$-Kadec-Klee property, then for
$x\in X$

$\partial f(x)=\operatorname{co}\\{\operatorname{s}-\,\,\underset{y\to x,y\neq
x}{\operatorname{Lim}\sup}\,\partial
f(y)\\}+N_{\overline{\operatorname{dom}f}}(x).$ (3.33)

###### Proof.

Using [10, Lemma 1] and [17, Corollary 2] one has

${R}(\lambda J_{\varepsilon}(\cdot-x)+\partial f)=X^{*},\forall x\in
X,\forall\lambda,\varepsilon>0.$

Now the theorem follows from Theorem 3.1. ∎

######  Remark 3.2.

Let $X$ be a Banach space. If the closed unit ball of $X^{*}$ is
$\rm{weak}^{*}$ sequentially compact, then the nets in Lemma 3.3 can be
replaced by sequences. As a consequence, all results in Proposition 3.1,
theorems 3.1, 3.2, 3.3 and Corollary 3.1 are also true for sequences. In
particular, these results are true for the case that $X$ is a weak Asplund
space; see [6, p.239].

$\Diamond$

##  4 A representation formula for the support functions of the values of
maximal monotone operators

In this section we consider a maximal monotone operator $A$ of type (D) in a
Banach space which has the w-Kadec-Klee property and whose dual space has the
w∗-Kadec-Klee property. We provide an explicit formula for the support
function of $Ax$ by use of the minimal-norm selection of $Ay$ for nearby
points $y$ of $x$.

We first present a technical result for the theorem below.

######  Lemma 4.1.

Let $X$ be a Banach space such that $X^{*},X^{**}$ are strictly convex and $X$
has the w-Kadec-Klee property, $X^{*}$ has the $w^{*}$-Kadec-Klee property.
Let $v\in X\setminus\\{0\\}$, $(t_{i})_{i\in I}\subset\mathbb{R}_{+}$ with
$t_{i}\to 0$ and $(w_{i})_{i\in I}\subset X$. If there exists a net
$(z^{*}_{i})_{i\in I}\subset X^{*}$ with $z^{*}_{i}\in J_{t_{i}}(w_{i})$,
$z^{*}_{i}\to J(v)$, then there exists a subnet of $(w_{i})_{i\in I}$
converging to $v$.

###### Proof.

Assume that $(z^{*}_{i})_{i\in I}\subset X^{*}$ with $z^{*}_{i}\in
J_{t_{i}}(w_{i})$ converging to $J(v)$. According to [5, p.608], for every
$i\in I$, there exists $v_{i}\in X$ such that

$\|w_{i}-v_{i}\|\leq\sqrt{t_{i}},\|z^{*}_{i}-J(v_{i})\|\leq\sqrt{t_{i}}.$
(4.1)

This implies $J(v_{i})\to J(v),$ and hence $\|v_{i}\|\to\|v\|$, which shows
that $(v_{i})_{i\in I}\subset X\subset X^{**}$ is eventually bounded. Using
Banach-Alaoglu theorem, by passing to a subnet, if necessary, that
$v_{i}\overset{\operatorname{w}}{\to}z^{**}\in X^{**}$. Furthermore, since
$X^{**}$ is strictly convex, we have $v_{i}=J^{*}(J(v_{i}))$ and
$v=J^{*}(J(v))$. By the maximal monotonicity of $J^{*}$, we deduce
$z^{**}=J^{*}(J(v))$ and hence

$\|v_{i}\|\to\|v\|,\,\,\text{ and
}\,\,v_{i}\overset{\operatorname{w}}{\rightarrow}v.$

Owing to the assumption that $X$ has the $\operatorname{w}$-Kadec-Klee
property, we get $v_{i}\to v$. It follows from (4.1) that $w_{i}\to v$, which
completes the proof. ∎

The following theorem represents the support function of the value $Ax$ via
its minimal-norm selections.

######  Theorem 4.1.

Let $X$ be a Banach space such that $X^{*},X^{**}$ are strictly convex and $X$
has the w-Kadec-Klee property, $X^{*}$ has the $w^{*}$-Kadec-Klee property.
Let $A:X\rightrightarrows X^{*}$ be a maximal monotone operator of type (D).
Then for $x\in{D}(A),\,v\in X\setminus\\{0\\}$

$\sigma_{Ax}(v)=\underset{t\downarrow 0\atop w\to v}{\lim\inf}\langle
A^{\circ}(x+tw),w\rangle.$ (4.2)

###### Proof.

Let $x\in{D}(A)$ and $v\in X\setminus\\{0\\}$. We first show that

$\sigma_{Ax}(v)\leq\underset{t\downarrow 0\atop w\to v}{\lim\inf}\langle
A^{\circ}(x+tw),w\rangle.$ (4.3)

Indeed, by the monotonicity of $A$, for $x^{*}\in Ax$ we have

$\langle x^{*},w\rangle=\frac{1}{t}\langle x^{*},(x+tw)-x\rangle\leq\langle
A^{\circ}(x+tw),w\rangle$

for all $t\geq 0,w\in X\setminus\\{0\\}$, with the convention $\langle
A^{\circ}(x+tw),w\rangle=+\infty$ if $x+tw\notin{D}(A)$. It follows that

$\langle x^{*},v\rangle\leq\underset{t\downarrow 0\atop w\to
v}{\lim\inf}\langle A^{\circ}(x+tw),w\rangle.$

Since this holds for any $x^{*}\in Ax$, we get inequality (4.3). It remains to
show

$\sigma_{Ax}(v)\geq\underset{t\downarrow 0\atop w\to v}{\lim\inf}\,\langle
A^{\circ}(x+tw),w\rangle.$ (4.4)

This obviously holds for $\sigma_{Ax}(v)=+\infty$. If
$\sigma_{Ax}(v)<+\infty$, then
$v\in\operatorname{dom}(\sigma_{Ax})\subset\overline{{D}(\partial\sigma_{Ax})}$.
We consider two cases of $v$ as follows.

Case 1: $v\in D(\partial\sigma_{Ax})$. Pick any
$x^{*}\in\partial\sigma_{Ax}(v).$ Involving (4.2) that
$\partial\sigma_{Ax}(v)=A(x;v)$, we have $x^{*}\in A(x;v)$. According to
Proposition 3.1, we can find nets $(t_{i})_{i\in I}\subset\mathbb{R}_{+}$,
$(w_{i})_{i\in I}\subset X$ and $(z^{*}_{i})_{i\in I}\subset
X^{*},(x^{*}_{i})_{i\in I}\subset X^{*}$ with $t_{i}\downarrow 0,$
$z^{*}_{i}\in J_{t_{i}}(w_{i})$, and $x^{*}_{i}\in A(x+t_{i}w_{i})$ such that

$z^{*}_{i}\to J(v),\,\,\,\,x^{*}_{i}\to x^{*}.$ (4.5)

On one hand, since $x^{*}_{i}\to x^{*}$, $(x^{*}_{i})_{i\in I}$ is eventually
bounded. Hence $(A^{\circ}(x+t_{i}w_{i}))_{i}$ is also eventually bounded due
to $\|A^{\circ}(x+t_{i}w_{i})\|\leq\|x^{*}_{i}\|$ for all $i\in I$. By Banach-
Alaoglu theorem, passing to the limit $i\to\infty$ along a subnet, one has

$A^{\circ}(x+t_{i}w_{i})\overset{\operatorname{w}^{*}}{\rightharpoonup}z^{*}$
(4.6)

for some $z^{*}\in X^{*}$. Note that $(w_{i})_{i}$ is eventually bounded since
$\|w_{i}\|\leq\|z^{*}_{i}\|+\sqrt{2t_{i}},\forall i\in I$ and
$\|z^{*}_{i}\|\to\|J(v)\|=\|v\|$. This gives $x+t_{i}w_{i}\to x$. It follows
from the maximal monotonicity of $A$, the eventual boundedness of
$(A^{\circ}(x+t_{i}w_{i}))_{i}$ and (4.6) that $z^{*}\in Ax.$ This yields

$\sigma_{Ax}(v)\geq\langle z^{*},v\rangle.$ (4.7)

On the other hand, using Lemma 4.1, we may suppose, by passing to a subnet if
necessary, that $w_{i}\to v.$ Owing to the fact that
$(A^{\circ}(x+t_{i}w_{i}))_{i\in I}$ is eventually bounded and (4.6), we get

$\langle A^{\circ}(x+t_{i}w_{i}),w_{i}\rangle\to\langle z^{*},v\rangle.$

Together with (4.7), we derive

$\sigma_{Ax}(v)\geq\underset{i}{\lim}\,\langle
A^{\circ}(x+t_{i}w_{i}),w_{i}\rangle\geq\underset{t\to 0\atop j_{t}(w)\to
J(v)}{\lim\inf}\langle A^{\circ}(x+tw),w\rangle.$ (4.8)

It follows from Lemma 4.1 that

$\underset{t\to 0\atop j_{t}(w)\to J(v)}{\lim\inf}\langle
A^{\circ}(x+tw),w\rangle\geq\underset{t\to 0\atop w\to v}{\lim\inf}\langle
A^{\circ}(x+tw),w\rangle.$ (4.9)

Combining (4.8) and (4.9), we obtain (4.4).

Case 2:
$v\in\operatorname{dom}\sigma_{Ax}\setminus\operatorname{D}(\partial\sigma_{Ax})$.
This means that $\partial\sigma_{Ax}(v)=\varnothing.$ According to [5, Theorem
2], there exists $v_{n}\to v$ such that $\sigma_{Ax}(v_{n})\to\sigma_{Ax}(v)$
and $\partial\sigma_{Ax}(v_{n})\neq\varnothing.$ From Case 1, for every
$n\in\mathbb{N}$

$\sigma_{Ax}(v_{n})=\underset{t\downarrow 0,\atop w\to v_{n}}{\lim\inf}\langle
A^{\circ}(x+tw),w\rangle.$

Hence, for every $n$ there exist $t_{n}>0,w_{n}\in X$ such that

$t_{n}\leq\frac{1}{n},\,\,\|w_{n}-v_{n}\|\leq\frac{1}{n},\,\,|\langle
A^{\circ}(x+t_{n}w_{n}),w_{n}\rangle-\sigma_{Ax}(v_{n})|\leq\frac{1}{n}.$

This implies that $t_{n}\to 0,w_{n}\to v$ and
$\underset{n\to\infty}{\lim}\langle
A^{\circ}(x+t_{n}w_{n}),w_{n}\rangle=\sigma_{Ax}(v)$. Hence we get (4.4). The
proof is complete. ∎

The next result follows directly from Lemma 3.4 and the previous theorem.

######  Corollary 4.1.

Let $X$ be a Banach space such that $X,X^{*}$ are locally uniformly convex and
$X^{**}$ is strictly convex and let $A:X\rightrightarrows X^{*}$ be a maximal
monotone operator of type (D). Then for $x\in{D}(A)$ and $v\in
X\setminus\\{0\\}$

$\sigma_{Ax}(v)=\underset{t\downarrow 0,\atop w\to v}{\lim\inf}\,\langle
A^{\circ}(x+tw),w\rangle.$ (4.10)

######  Corollary 4.2.

Let $X$ be a Banach space such that $X^{*},X^{**}$ are strictly convex and $X$
has the w-Kadec-Klee property, $X^{*}$ has the $w^{*}$-Kadec-Klee property.
Let $A_{1},A_{2}:X\rightrightarrows X^{*}$ be maximal monotone operators of
type (D). If ${D}(A_{1})={D}(A_{2})=:D$ and $A^{\circ}_{1}x\in A_{2}x$ for all
$x\in D,$ then $A_{1}=A_{2}.$

###### Proof.

We first show that for $x\in D$ and $v\in X\setminus\\{0\\}$

$\sigma_{A_{1}x}(v)\geq\sigma_{A_{2}x}(v),$ (4.11)

which would imply $A_{2}x\subset A_{1}x$ for $x\in{D}$. This is trivial for
$\sigma_{A_{1}x}=+\infty$. We consider $\sigma_{A_{1}x}(v)<+\infty$. According
to Theorem 4.1, there exist nets $(t_{i})_{i\in
I}\subset\mathbb{R}_{+},(w_{i})_{i\in I}\in X$ with $t_{i}\to 0$ and $w_{i}\to
v$ such that

$\sigma_{A_{1}x}(v)=\underset{i}{\lim}\langle
A^{\circ}_{1}(x+t_{i}w_{i}),w_{i}\rangle.$ (4.12)

By assumption, $A^{\circ}_{1}(x+t_{i}w_{i})\subset A_{2}(x+t_{i}w_{i})$ for
all $i\in I$. The monotonicity of $A_{2}$ implies that

$\sigma_{A_{2}x}(w_{i})\leq\langle A^{\circ}_{1}(x+t_{i}w_{i}),w_{i}\rangle\ \
\mbox{for all }i\in I.$

Combining this with the lower semicontinuity of $\sigma_{A_{2}x}$ and (4.12),
we obtain

$\sigma_{A_{2}x}(v)\leq\underset{i}{\lim}\,\sigma_{A_{2}x}(w_{i})\leq\underset{i}{\lim}\langle
A^{\circ}_{1}(x+t_{i}w_{i}),w_{i}\rangle\leq\sigma_{A_{1}x}(v),$

which means that (4.11) holds. This yields $A^{\circ}_{2}x\in A_{2}x\subset
A_{1}x$ for all $x\in D.$ Similarly, we have $A_{1}x\subset A_{2}x$ for every
$x\in D$. Therefore $A_{1}=A_{2}$, as was to be shown.∎

Acknowledgements: A part of this paper was done during the first author’s stay
at Vietnam Institute for Advanced Study in Mathematics (VIASM). He would like
to thank VIASM for its wonderful working condition. This work was supported by
Vietnam National Program for the Development of Mathematics until 2030
(project code: B2022-CTT-07).

## References

  * [1] V. Barbu, Nonlinear Differential Equations of Monotone Types in Banach Spaces, Monogr. Math., Springer, New York 2010.
  * [2] H.H. Bauschke, J.M. Borwein, X. Wang, L. Yao, Every maximally monotone operator of Fitzpatrick-Phelps type is actually of dense type, Optim. Lett.6 (8) (2012), 1875–1881.
  * [3] J.M. Borwein, S. Fitzpatrick, R. Girgensohn, Subdifferentials whose graphs are not norm $\times$ weak∗ closed, Canad. Math. Bull. 46 (4) (2003) 538–545.
  * [4] J.M. Borwein, L. Yao, Structure theory for maximally monotone operators with points of continuity, J. Optim. Theory Appl. 157 (2013), 1–24.
  * [5] A. Brøndsted, R.T. Rockafellar, On the subdifferentiability of convex functions, Proc. Amer. Math. Soc. 16 (1965), 605–611.
  * [6] J. Diestel, Sequences and Series in Banach Spaces, Springer, New York, 1984.
  * [7] S. Fitzpatrick, R.R. Phelps, Bounded approximants to monotone operators on Banach spaces, Ann. Inst. H. Poincaré C Anal. Non Linéaire, 9 (5) (1992), 573–595.
  * [8] L. Gasiński, N.S. Papageorgiou, Exercises in Analysis. Part 1, Problem Books in Mathematics, Springer, Cham, 2014.
  * [9] J.P. Gossez, Opérateurs monotones non linéaires dans les espaces de Banach non réflexifs, J. Math. Anal. Appl. 34 (1971), 371–395.
  * [10] J.P. Gossez, On the extensions to the bidual of a maximal monotone operator, Proc. Amer. Math. Soc. 62 (1) (1977) 67–71.
  * [11] A. Hantoute, B.T. Nguyen, Boundary of maximal monotone operators values, Appl. Math. Optim. 82 (1) (2020) 225–243.
  * [12] A.M. Marques, B.F. Svaiter, On Gossez type (D) maximal monotone operator, J. Convex Anal. 17 (3-4) (2010) 1077–1088.
  * [13] A.M. Marques, B.F. Svaiter, On the surjectivity properties of perturbations of maximal monotone operators in nonreflexive Banach spaces, J. Convex Anal. 18 (1) (2011) 209–226.
  * [14] B.T. Nguyen, P.D. Khanh, Faces and support functions for the values of maximal monotone operators, J. Optim. Theory Appl. 186 (3) (2020) 843–863.
  * [15] R.R. Phelps, Weak∗ support points of convex sets in $E^{\ast}$, Proc. Amer. Math. Soc. 99 (2) (1987), 319–322.
  * [16] R.R. Phelps, Convex Functions, Monotone Operators and Differentiability. Second ed., Lecture Notes in Math., vol. 1364, Springer-Verlag, Berlin, 1993.
  * [17] A. Răşcanu, On the maximal monotonicity of subdifferential operators, Rev. Roumaine Math. Pures Appl. 66 (1) (2021), 237–242.
  * [18] R.T. Rockafellar, On the maximality of sums of nonlinear monotone operators, Trans. Amer. Math. Soc. 149 (1970), 75–88.
  * [19] S. Simons, The range of a monotone operator, J. Math. Anal. Appl., 199 (1) (1996) 176–201.
  * [20] S. Simons, From Hahn-Banach to Monotonicity, Second edition. Lecture Notes in Mathematics, 1693. Springer, New York, 2008.
  * [21] C. Zălinescu, Convex Analysis in General Vector Spaces, World Scientific Publishing Co., Inc., River Edge, NJ, 2002.


We discuss an
actor-critic-type reinforcement learning algorithm for solving the
problem of hedging a portfolio of financial instruments
such as securities and over-the-counter derivatives
using purely historic data.

Essentially we present a practical implementation for solving a problem of the structure

\[
\mbox{current state}
\,\big)
\supstack!=
\sup_{\mbox{action $\ma$}}:\
U\!\left[\ \mbox{discount factor $\cdot$} \
\mbox{future state}(\ma)
\,\big) + \mbox{rewards}(\ma)
\ \right]
\]

where $U$ represents a risk-adjusted return metric.

The key characteristics of our approach are: the ability to hedge with derivatives such as forwards, swaps, futures, options;
incorporation of trading frictions such as trading cost and liquidity constraints;
applicability to any reasonable portfolio of financial instruments; realistic, continuous state and action spaces;
and formal risk-adjusted return objectives. We do not impose a boundary condition on some future maturity.

Most importantly,
the trained model provides an optimal hedge for arbitrary initial portfolios and
market states without the need for re-training.

We also prove existence of finite solutions to our Bellman equation,
and show the relation
to our vanilla Deep Hedging approach [1]

The views expressed on this note are the author’s personal views and should not be attributed to any other person, including that of their employers.

§ INTRODUCTION

This note discusses a model-free, data-driven
method of managing a portfolio of financial instruments such
as stock, FX, securities and derivatives
with reinforcement learning "AI" methods.
It is a dynamic programming “Bellman" version of our
Deep Hedging approach [1].

The key characteristics of our Deep Hedging framework above
several other proposed methods for hedging a portfolio of financial instruments are: its ability to hedge with derivatives such as forwards, swaps, futures, options;
incorporation of trading frictions such as trading cost and liquidity constraints;
applicability for any reasonable portfolio of financial instruments; realistic, continuous state and action spaces;
formal risk-adjusted return objectives. However, our original approach
solved this problem for a given initial portfolio and market state.
That means it needs to be re-trained, say, daily to reflect changes in our
trading universe or the market.
The method
proposed here, on the other hand, attempts to solve the optimal hedging problem over an
for any portfolio and market state which is close to past experiences.
It does not impose a boundary restriction for some terminal maturity.

Our main Bellman equation (<ref>) has the structural form

\[
\mbox{current state}
\,\big)
\supstack!=
\sup_{\mbox{action $\ma$}}:\
U\!\left[\ \mbox{discount factor $\cdot$} \
\mbox{future state}(\ma)
\,\big) + \mbox{rewards}(\ma)
\ \right]
\]

where $U$ represents a risk-adjusted return metric.

The work presented here is an extension of
our patent application [4].
The main contribution is to provide
a numerical implementation method for the practical problem of being able to represent
arbitrary portfolios of derivatives as states using purely historic data. We also clarify conditions
under which the corresponding Bellman equations are well-posed and
admit unique finite solutions.

§.§.§ Quant Finance 2.0

The motivation for the work presented in this article – and of the Deep Hedging framework in
general – is to build financial risk management models which “learn" to trade from historic
data and experiences. Today, portfolios of derivatives, securities and other instruments
are managed using the traditional quantitative finance engineering paradigm borne out of the seminal work by
Black, Scholes & Merton. However, practical experience on any trading desk is that
such models do not perform sufficiently well to be automated directly. To start with, they
do not take into account trading frictions such as cost and liquidity constraints. Even beyond
that they suffer from the underlying engineering approach which prioritizes a focus on interpolating
hedging instruments such as forwards, options, swaps over realistic market dynamics.
It is an indication of the state of affairs that standard text books on financial engineering
in quantitative finance do not discuss real data out-of-sample performance of the models proposed.

As a result,
the prices and risk management signals (“greeks") are overly simplistic and do not capture
important real-life dynamics. A typical trader will therefore need to overwrite the prices and
trading signals
from such as standard models with their own heuristics.

Our Deep Hedging framework takes a different approach and focuses on robust performance
under real-life dynamics, enabed by the use of modern machine learning techniques.
The current article is the closest attempting to mimicing a trader's real life behaviour
in that here we will give an AI the same historic “experience" a real trader would have.
Of course, our model will still be limited by the coverage of historic scenarios used
to train it. Hence, human oversight is still required to cater for abrupt changes in
market scenarios or starkly adverse risk scenarios.

The website  gives an overview over available material on the
wider topic.

§.§.§ Related Works

There a few related works concerning the use of machine learning methods for
managing portfolios of financial instruments which include derivatives, starting with our
own [1]. There, we solved the optimal trading problem for a fixed initial portfolio with a given terminal maturity
and a fixed initial market state using periodic policy search, a method akin to “American Monte Carlo".

In [7] the authors discuss the use of Bellman methods for this task, namely  using DQN
and a number of similar methods. They also use risk-adjusted returns in the form of a mean-variance
objective. However, in their work the state and action spaces are
finite which are not realistic in practise. Moreover, their parametrization of the derivative
portfolio is limited to single vanilla options. They also do not cover derivatives as hedging instruments. The method
relies on a
fixed terminal maturity.

In [10] the authors also develop a discrete state approach,
where the problem is solved for each derivative position separately. The authors focus in
their first work on vanilla options and minimize the terminal variance of the delta-hedged position.
The maturity of the problem is fixed.
In their later [11] the authors present methods to smooth the state space.
In neither account are derivatives as hedging instruments supported.

A forthcoming ICML contribution <cit.> we solve the Bellman equation
associated with the Deep Hedging problem for a fixed maturity, using continuous states
and the entropy as risk-adjusted return.

There is a larger literature on the application of AI methods for managing portfolio risks in the context
of perpetual assets such as stock and FX portfolios which might reasonably be approximated
by normal assets. See the summary [13] for an overview, where they also cover
the related topic of trade execution with AI methods.

Underlying our work is the use of dynamic risk measures, a topic
with a wide literature. We refer the interested reader to [8] and [18]
among many others.

§ DEEP BELLMAN HEDGING

In this note we will use a notation much more similar to standard
reinforcement learning literature, chiefly [20]. That means
in particular that we will formulate our approach essentially
as a continuous state Markov Decision Process (MDP) problem. We will make a decision
from some point in time to another. That would typically be intraday or from day to day.
To simplify our discussion we will assume we are making a decision “today"
and then again “tomorrow". Variables which are valid tomorrow
will be indicated by a $'$. We will strive to use bold letters for
vectors. A product of two vectors is element wise, while “$\cdot$"
represents the dot product. We will use small letters
for instances of data, and capital letters for random variables.

We denote by $\mm$ the market state today. The market contains all information available to us today such as
current market prices, time, past prices, bid/asks, social media feeds and the like. The set of all market states is
denoted by $\calM\subset \R^N$.
All quantities observed today are a function of the market state.[
Mathematically, we say that $\mm$ generates today's $\sig$-algebra]
Past market states are also known today.[This means
the sequence of market states generates a filtration.
The market tomorrow is a random variable $\mM'$
whose distribution is assumed to  only depend on $\mm$,
and not on our trading activity.[
See the lecture notes [2] for an example
of incorporating market impact.
In terms of notation, think $\mm\equiv \mm_t$ and
$\mM'\equiv \mM_{t+1}$.
The expectation operator of a function $f$ of $\mM'$ conditional
on $\mm$ is written as $\E[f(\mM')|m]:=\int f(\mm') \P[d\mm'|\mm]$.

We will trade financial instruments such as securities, OTC derivatives
or currencies. We will loosely refer to them as “derivatives"
as the most general term, even if we explicitly include primary
asset such as stocks and currencies. We use $\calX$ to refer to the
space of these	instruments.
For $x\in\calX$ we denote by $r(x,\mm)\in\R$ the  cashflows arising from holding $x$ today, aggregated
into our accounting currency.[This implies
implies that spot-FX transactions
are frictionless.]
Cashflows here cover everything from expiry settlements, coupons, dividends, to payments
arising from borrowing or lending an asset.
For a vector $\mx$ of instruments we use $
\mr(\mx;\mm)$ to denote the vector of their cashflows.

An instrument changes with the passage of time: an instrument $x\in\calX$ today
becomes $x'\in\calX$ tomorrow, representing only cashflows from tomorrow onwards.
If the expiry of the instrument is today, then $x'=0$.

Every instrument $x$ we may trade has a book value in our accounting currency
which we denote by $B(x,\mm)$.
The book value of a financial instrument is its official
mark-to-market, computed
using the prevailing market data $\mm$. This could be a simple closing price, a weighted
mid-price, or the result of running a
more complex standard derivative model.
Following our notation $B(x',\mM')$ denotes the book value of the instrument tomorrow.
We use $\mB(\mx,\mm)$ for the vector
of book values if $\mx$ is a vector of instruments. We like
to stress that contrary to our other work [3] here
the book value is with respect  only to
today's and future
cashflows, not past cashflows.

In order to take into account the
value of money across time, we will also assume are given a bank
account – usually called the numeraire - which charges the
same overnight interest rate for
credits and deposits.
The respective one-day discount factor from tomorrow to today
is denoted by $\beta(\mm)$
and we will assume that there is some
$\beta^*$ such that $\beta(\mm)\leq\beta^*<1$.
Contrary to [1] we do not assume that
cashflows are
discounted using the numeraire.

The discounted profit-and-loss (P&L) for a given instrument $x\in\calX$
is the random variable
dB(x,,') :=  β()  B(x',') - B(x,) _ +  r(x,) _  .
If $\mx\in\calX^n$ is a vector, then $d\mB(\mx,\mm,\mM')$ denotes the vector of P&Ls.

§.§.§ Trading

A trader is in charge of a portfolio
$z\in\calX$ – also called “book" – of financial
instruments such as currencies, securities and over-the-counter (OTC) derivatives.
We call the combined $\ms:=(z,\mm)$ our state today
which takes values in $\ms\in\calS:=\calX\times \calM$.
We will switch in our notation between writing functions in both variables
$(z,\mm)$ and only in $(\ms)$ depending on context.

In order to risk manage her portfolio, the trader has access to $n$
liquid hedging instruments $\mh\equiv \mh(\mm)\in\calX^n$ in each
time step.
These are any liquid instruments such
such as forwards, options, swaps etc.
Across different market states they will usually not be the contractually same fixed-strike fixed-maturity instruments:
instead, they will
usually be defined relative the prevailing market
in terms of time-to-maturities
and strikes relative to at-the-money. See [1]
for details.

The action of buying[Selling amounts
to purchasing a negative quantity.] $\ma\in\R^n$
units of our hedging instruments
will incur transaction cost $c(\ma;z,\mm)$ on top of the book value.
cost as function of $\ma$ is assumed to be normalized to $c(0;\ms)=0$, non-negative, and convex.[Convexity excludes fixed transaction cost.]
The convex set of admissible actions is given
as $\calA(z,\mm):=\{\ma\in\R^n: c(\ma;z,\mm) < \infty\}$.
Making cost dependent on both the current portfolio
and the market  allows modelling
trading restrictions based on our current position such as short-sell constraints,
or restrictions based on risk exposure.

Given $z$ and an action $\ma$ today the new portfolio tomorrow is $z'+\ma \cdot \mh' \in \calX$.

A trading policy $\pi$ is a function
which determines the next action based on our current state,
i.e. simply $\ma = \pi(z,\mm)$. We use $\calP:=\{ \pi: \calX\times \calM \rightarrow \calA(z,\mm) \}$ to refer to the convex set of
all admissible policies.

A trader will usually manage her book by referring to the change in book values
plus any other cashflows, most notably cashflows and the cost of hedging.
The associated reward  for taking an action $\ma$ per time step is given as
R(;z,, ') :=  dB(z,,') _

·d(,,') _

c(;z,) _

For the rewards of a policy $\pi$ we will use the convention

\[
R(\pi;z,\mm, \mM') := R(\pi(z,\mm);\,z,\mm, \mM') \ .
\]

The new joint portfolio tomorrow is given by
z_' := z' + ·'  .
The new state tomorrow is a random variable depending on our action which we write as

\[
\mS'_\ma := (z_\ma',\mM') \ .
\]

§.§ The Bellman Equation for Monetary Utilities

Standard reinforcement learning as discussed for example
in [20] usually aims to maximize
the discounted expected future rewards of running a given policy.
Essentially, the optimal value function $V^*$ is stipulated to satisfy
a Bellman equation of the form

\[
\,\big)
\supstack!=
\sup_{\ma\in \calA(z,\mm)}: \
\E\!\left[\  \beta(\mm) \
z' +\ma\cdot \mh';\,\mM'
\,\big) + R(\ma;z,\mm, \mM')
\,\big|\,\mm
\ \right]  \ .
\]

Instead of using the expectation
it is more natural in finance to
choose an operator $U$ which takes into account risk aversion:
roughly means that if two events have the same expected outcome, then we prefer the
one with the lower uncertainty.

The Deep Bellman Hedging problem is finding a value function $V^*$ which satisfies

\begin{equation}\label{eq:bell_U}
\left\{
\begin{array}{lcl}
\,\big)
& \supstack!= &
(TV^*)(z,\mm) \\
\\
(Tf)(z,\mm) & := &
\sup_{\ma\in \calA(z,\mm)}:\
U\!\left[\ \beta(\mm) \
z' +\ma\cdot \mh';\,\mM'
\,\big) + R(\ma;z,\mm, \mM')
\,\big|\,\mm
\ \right]  \ .
\end{array}
\right.
\end{equation}

(The action $\ma$ in above $\sup$ operator is a function of the state $\ms=(z,\mm)$.)

We may therefore write the operator as an optimization over policies
$\pi$ as follows:

\begin{equation}\label{eq:bell_U_pi}
\,\big)
\equiv
\sup_{\pi(z,\ms)\in \calA(z,\mm)}:
U\!\left[\ \beta(\mm) \
z' +\pi(z,\mm)\cdot \mh';\,\mM'
\,\big) + R(\ma;z,\mm, \mM')
\,\big|\,\mm
\ \right]  \ .
\end{equation}

We would like to stress that the value function here represents the “excess value"
of a portfolio over its book value: if $V^*$ were zero, that would mean the optimal risk-adjusted
value for a portfolio were given as its book value. Remark <ref> makes this statement explicit.

There are many different reasonable
risk-adjusted return metrics $U$ used in finance, most notably mean-volatility, mean-variance
and their downside versions. We refer the reader to the seminal [16].
Mean-volatility in particular remains a popular choice for many practical applications.
However, it is well known that mean-volatility, mean-variance
and their downside variants are not monotone,
which means that even if $f(\ms)\geq g(\ms)$
for all states $\ms$ it is not guaranteed that $U[ f(\mS' )\geq U[ g(\mS' )]$, c.f. [6]. The lack of monotonicity
means that standard convergence proofs for the Bellman
equation do not apply; see section <ref>.

We will here take a  more formal route and focus on
monetary utilities.
A functional $U$ is a monetary utility here if it is normalized to $U(0)=0$, monotone increasing (more is better),[
If $f\geq g$ then $U[f(\mS')]\geq U[g(\mS')]$.
concave (diversification works)[
For $X=f(\mS'),Y=g(\mS')$ and $\alpha\in[0,1]$ we have $U[\alpha X + (1-\alpha)Y] \geq \alpha U[X]
+ (1-\alpha) U[Y]$.
] and cash-invariant. The latter
means that  $U[ f(\mS',\ms) + y(\ms) |\ms] = U[f(\mS',\ms)|\ms] + y(\ms)$ for any function $y(\ms)$.
The intuition behind this property is if we add a cash amount $y$
to our portfolio, then its monetary utility increases by this amount.[We have shown in [6] that cash-invariance is equivalent
to being able to write-off parts of our portfolio for the worst
possible outcome.] An important implication of cash-invariance is that our optimal actions
do not depend on our current wealth. We will also assume that our monetary utilities
are risk-averse with respect to $\P$ in the sense that $\E[ f(\mS') ]=U[\E[ f(\mS') ]] \geq U[ f(\mS') ]$.[
We note that concavity of $U$ does not imply risk aversion w.r.t. $\P$ in general. As an example chose a measure $\Q\not\approx \P$
and set $U(f(\mS')) := \E_\Q[ f(\mS') ]$.

The negative
of a monetary utility is  called a convex risk measure, c.f. [9]. See also [8] on the topic of dynamic
and time-consistent risk measures.

As in [3] we will focus on monetary utilities given
as optimized certainty equivalents (OCE) of a utility function,
introduced by [5].

Let $u:\R\rightarrow\R$ be a $C^1$ utility function which
means it is
monotone increasing
and concave. We  normalize it to $u(0)=0$ and $u'(0)=1$.
The respective OCE monetary utility is then defined by

\[
U\!\left[\  f(\mS') \ \big|\ \ms\ \right]
\sup_{y(\ms)\in\R} \E\!\left[\ u\left(\ f(\mS')
+ y(\ms)\, \right)\ \big|\ \ms\ \right] - y(\ms)
\]

The function $y$ in our definition will be modelled as a neural network.

Examples of OCE utility functions are

* Expectation (risk-neutral):
u(x) :=  x

* Worst Case:
u(x) :=  \inf x

* CVaR or Expected Short Fall:
u(x) := (1+\lambda)\ \min\{0,X\}

* Entropy:
u(x) := (1-e^{-\lambda x})/\lambda
in which case $U[ f(\mS')|\ms] = - \frac 1\lambda
\E[ \exp(-\lambda f(\mS')) | \ms ]$. The entropy
reduces to mean-variance if
the variables concerned are normal. It has many
other desirable properties, but it also
penalizes losses rather harshly: an unhedgable short position
in a Black&Scholes stock has negative infinite entropy.

* Truncated Entropy: to avoid the harsh penalties for short
positions imposed by the exponential utility we might instead use
u(x) :=
( 1- e^{-\lambda x} ) /\lambda \1[ x > 0 ]
( x - \half \lambda x^2 ) \1[x<0]$.

* Vicky: the following functional was proposed in [12]:
u(x) := \frac1\lambda \left( 1 + \lambda x - \sqrt{ 1 + \lambda^2 x^2 } \right)

* Normalized quadratic utility:
u(x) := - \half \lambda (x-\mbox{$\frac1\lambda$})^2\1[x<\mbox{$\frac1\lambda$}] + \frac1{2 \lambda}

We call a monetary utility coherent
if $U[ n(\ms)\,f(\mS' ) | \ms ] = n(\ms) U[ f(\mS') | \ms ]$.
An OCE monetary utility is coherent if $u(n\,x) = n\,u(nx)$.
Coherence is not a particularly natural property: it says
that the risk-adjusted value of a position growths linearly with position
size. Usually, we would assume that risk increases superlinearly.
The practical relevance of this property for us is that if $U$
is coherent, then we can move the discount factor $\beta$
in and out of our monetary utility: $\beta(\ms)\, U[ f(\mS') | \ms] =
U[ \beta(\ms)\,f(\mS')|\ms]$.

We say $U$ is
if iterative application lead to the same monetary utility in the sense that $U[ U[ f(\mS'') | \mS' ] | \ms ] = U( f(\mS'' | \ms )$.  The only
time-consistent OCE monetary utilities are the entropy and the
expectation, c.f. [14].

We may now present the first key result of this article: recall the
definition of our rewards (<ref>).
We say that rewards are  finite if
sup_∈(z,)   U[ R(;,,') |  ] <∞ .
for all $z\in\calX,\mm\in\calM$.
This can be achieved for example if $\calA(z,\mm)$ is bounded.

Assume rewards are finite and that $U$ is an OCE monetary utility.

Then the Bellman
equation (<ref>)
has a unique finite solution.

The proof can be found in section <ref>. It relies
on monotonicity
and cash-invariance of the monetary utility, which means that while it
does not apply to mean-variance, mean-volatility and related operators,
it does apply to $U$ being VaR, e.g. the percentile function
$U(X) := \P[X \leq \cdot]^{-1}( 1-\alpha )$ where $\alpha\in[0,1)$ is a confidence interval.
This is a good alternative to mean-volatility as it has a similar interpretation,
as long as VaR's lack of concavity is not a concern.

Our definition of our rewards (<ref>) as the full mark-to-market of
the hedged portfolio is in so far unusual as the reward term contains
future variables, namely the book value of the hedged book tomorrow.

A more classic approach would be to let the rewards represent
only actual cashflows, e.g.
R̃(,z,):=  r(z,) _

- ·(,) - c(,z,) _

The numerical challenge with this formulation is that cashflows are
relatively rare for most hedging instruments: if we trade a 1M vanilla
option, then it only has one cashflow at maturity – if it ends up in the money.
That means that learning the value of future cashflows is harder
when we train with only daily cashflows. Hence, it is numerically
more efficient to solve for the difference between the optimal value
function and the book value of a portfolio.

Theoretically, though, the two are equivalent:
let $\tilde V^*(z,\mm) := V^*(z,\mm) + B(z,\mm)$. Then $\tilde V^*$
solves the Bellman equation
Ṽ^*(z,) != sup_∈():
Ṽ^*( z' +·', )
+ R̃(,z,)
|  ]  .

It is straight forward to formally extend (<ref>) to multiple
time steps. Let $\mS^{(1)}:=\ms$ and $\mS^{(i+1)}=\mS^{(i)}{}'$.
We use the same numbering for other variables.
the discount factors
$\beta_i := \prod_{e=1}^i \beta(\mM^{(e-1)})$ and set

\begin{equation}\label{eq:bell_U_multi}
(T_n f)(z,\mm) :=
\sup_{\pi}:\
U\!\left[\ \beta_n\
z^{(n)} + \mA^{(n)}\cdot \mH^{(n)};\,\mM^{(n+1)}
\,\big) + \sum_{i=1}^n
\beta_{i-1}\,
\ \big|\ \mM^{(1)}=\mm
\ \right]
\end{equation}

where we used $
\mA^{(n)} := \pi(z^{(n)}, \mM^{(n)})$.
Our indexing scheme means that $T_1 = T$.

It is straight forward to
amend the proof of theorem <ref> to
show that if any statistical arbitrage is finite,
then the associated equation $T_n f = f$ also has a unique finite
solution $V^*_n$.

However, the resulting value functions are not usually consistent: for $n>1$ in general $V^*_n \not= T^n V^*$ unless
$U$ is the expectation.[
We show the claim for $n=2$. Let $R^{(n)}:=R(\mA^{(n)},\mS^{(n)},\mM^{(n+1)} )$.
We use the state notation $\ms=(z,\mm)$.
=     sup_π: U[  β(^(1))			{
β(^(2))								f(  ⋯, ^(3) )
| ^(2)
} + R^(1)

(*)=     sup_π: U[  β(^(1))			{
β(^(2))								f( ⋯, ^(3) )
| ^(2)
} + R^(1)

(**)≥/=     sup_π: U[  							 U[
β(^(1)) β(^(2))								f( ⋯, ^(3) )
+ β(^(1)) R^(1)
| ^(2)

(***)=     sup_πU[ ⋯|  ]
=  T_2 f()  .
The equation $(*)$ is true if $\pi$ is found over the entire set of measurable
functions, and if the state contains all previous state information.
Moreover, $(**)$ is an equality for the expectation
or any other coherent monetary utility. For all others
convexity and $U(0)=0$ imply the stated inequality.
The final equality $(**)$ is only true of $U$ is time-consistent
which means either the entropy or the expectation.

§ NUMERICAL IMPLEMENTATION

We now present an algorithm which will iteratively approach an optimal solution of our Bellman equation (<ref>).
This is an extension over the entropy case presented in [4].

Recall that $V^*(z,\mm)=0$ is the optimal solution of the book value of a portfolio
is indeed its value function. We therefore use as initial guess $V^{(0)}(\,z,\mm\, ) := 0$
for all portfolios and states $\ms=(z,\ms)$.[
It is not a good idea to initialize a network with zero to achieve this as all gradients will look rather the same. Assume $\calN(\theta;x)$ is a neural network initialized by random
weights $\theta_0$. Then use
the Buehler-zero network
$N(\theta;x) := \calN(\theta;x) -
\calN(\theta_0;x)$.

Then we solve iteratively for each $n$ the following scheme $(n-1)\rightarrow n$:

* Actor: given $V^{(n-1)}$ we wish to find an optimal neural network
policy $\pi^{(n)}:\calX\times \calM\rightarrow \R^n$ which maximizes for all states $\ms=(z,\mm)\in\calS$ the expression

\begin{equation}\label{eq:dhb_it_1}
\sup_{\pi}:\  U\!\left[  \,   \beta(\mm)\, V^{(n-1)}\big(\,
z'  + \pi(z,\mm)\cdot \mh';\, \mM '\,
\big)+
\Big|\,\mm\,
\right]
\ .
\end{equation}

We recall that $c(\ma;z,\mm)=\infty$
whenever $\ma\not\in\calA(z,\mm)$. Hence, any finite solution $\pi^{(n)}$
will be a member of $\calP$.

In the case of our OCE
monetary utility, we will need to find both a network $\pi^{(n)}$
and a network $y^{(n)}$ to jointly maximize for all states $\ms=(z,\mm)$

\begin{equation}
\sup_{\pi,y}: \  \E\!\left[\,\beta (\mm)\,
u\!\left(  \,    V^{(n-1)}(\, z'  + \pi(z,\mm)\cdot \mh';\, \mM'\, ) + y(z,\mm)
\right) - y(z,\mm)        +  R(\pi;z,\mm,\mM')
\,\Big|\,\mm\,
\right] \ .
\end{equation}

Following [20] will approach this by stipulating
that we have a density $\Q$ over
all samples from $\calS=\calX\times\calM$. In practise, $\calS$
is a finite sample set, and $\Q$ could be
a uniform distribution. We note that under $\Q$ the current portfolio
and the current market are random variables $Z$ and $\mM$, respectively.
The associated unconditional expectation is denoted by
$\E[ \cdot ] = \int \Q[d\ms]\, \E[\cdot|\ms]$.

We then solve

\begin{equation}\label{eq:dhb_it_1n}
\sup_{\pi,y}: \ \E\!\left[\
u\!\left(  \,  \beta (\mM)\  V^{(n-1)}(\, Z + \pi(Z,\mM)\cdot \mH';\, \mM'\, ) + y(Z,\mM)
\right)
- y(Z,\mM)  +
\right]  \ .
\end{equation}

Under $\Q$
the current market state, the portfolio and the hedging
instrument representation
are random variables, hence we have referred to them
with capital letters.

The choice of $\Q$ is not trivial: it is meant to represent the
probability of possible portfolio and market state conditions.

* Critic (Interpolation): as next step, we estimate a new value function $V^{(n)}$
given $\pi^{(n)}$ and $y^{(n)}$.
This means fitting a neural network $V^{(n)}$ such that

\begin{equation}\label{eq:dhb_it_2}
V^{(n)}( z,\mm ) \equiv ( TV^{(n-1})( z, \mm )
\end{equation}

We note that solving (<ref>) numerically
with packages like TensorFlow or PyTorch will also yield
samples $TV^{(n-1)}(z,\mm)$ for all $(z,\mm)\in\calS$. Assuming this is the case we
may find network weights for $V^{(n)}$ by solving an interpolation problem of the form

\[
\inf_V: \E\!\left[
\
d\left( - V( Z,\mM ) + (TV^{(n-1)})( Z, \mM )
\right)\
\right]
\]

for some distance $d$
over our discrete sample space. An example is $d(\cdot)=|\cdot|$.
Instead of using neural networks for the last step we may also
consider classic interpolation techniques such as kernel interpolators.

This scheme is reasonably
intuitive as it iteratively improves the
estimation of the monetary utility $V^{(n)}$ and the optimal
action $a^{(n)}$. There is a question on how many training epochs
to use when solving, in each step, for the action and the value function.
In [20] there is a suggestion that using just one step is sufficient.
The authors call this the actor-critic method. There are several
other discussions on the viability of such methods, see also [17]
and the references therein.

In some applications we may not be able to
use samples of $TV^{(n-1)}$ to solve (<ref>),
but make use of trained $a^{(n)}$ and $y^{(n)}$ directly.
We therefore may solve

\begin{equation}\label{eq:dhb_it_2n}
\inf_V: \E\left[ \left(
- V(\,Z;\,\mM\,) +
\E\! \!\left[\,  \beta (\mM)\ u\!\left(
V^{(n-1)}(\,\cdots; \mM'\, ) + y^{(n)}(Z,\mM) \right) - y^{(n)}(Z,\mM)
\,\Big|\,Z,\mM\,
\right]
\right)^2 \right] \ .
\end{equation}

The nested expectation is numerically sub-optimal. In order
to address this, we solve instead the unconditional

\begin{equation}\label{eq:dhb_it_2n2}
\inf_V: \E\left[ \left(
- V(\,Z;\,\mM\,) +
\beta (\mM)\ u\!\left(
V^{(n-1)}(\, \cdots, \mM'\, ) + y^{(n)}(Z,\mM) \right) - y^{(n)}(Z,\mM)
\right)^2 \right) \ .
\end{equation}

which has the same gradient in $V$, and therefore the same optimal solution.[

Assume that $V(\ms) \equiv V(\theta;\ms)$ where $\theta$ are our network parameters.
Denote by $\partial_i$ the derivative with respect to the $i$th parameter.
Our equation then has the form $\inf_\theta f(\theta)$ where

\[
f(\theta) := \E\!\left[ ( V(\theta;\mS)
+ \E[ h(\mS') | \mS ] +g(\mS) )^2 \right]
\]

The gradient is

\[
\partial_{\theta_i} f'(\theta) = 2\, \E\!\left[ \partial_i V(\theta;\mS) ( V(\theta;\mS) + \E[ h(\mS') | \mS ]
+ g(\mS) ) \right]
2\, \E\!\left[ \partial_i V(\theta;\mS) ( V(\theta;\mS) + h(\mS')
+ g(\mS) ) \right]
\]

Therefore $f$ has the same gradient as $
\theta \mapsto
\E\!\left[ ( V(\theta;\mS) + h(\mS')
+ g(\mS) )^2 \right]

§.§ Representing Portfolios

The most obvious challenge when applying the approach presented in section <ref>
is the need to represent our portfolio is some numerically efficient way.
The following is an extension of the patent [4]
where we proposed using a more cumbersome signature representation of our trader instruments
a'la [15].

Assume that we are given historic market data $\mm_t$
at time points $\tau_0,\ldots,\tau_N$.
Further assume that at each point $\tau_j$ we had in our book instruments
$\mx^t=(x^{t,1},\ldots, x^{t,m_t})$ with $x^{t,i}\in\calX$.

As $\mx$ were actual historic instruments,
we have for each $x^{t,i}$
a vector $\mf^{t,i}_t \in \R^F$
of historic risk metrics computed in $t$, such as the
book value, a range of greeks, scenarios and other
calculations made in $\tau_t$ to assist humans in their risk 		management decisions.
We assume that those metrics        $\mf^t_t=(\mf^{t,1}_t,\ldots,
\mf^{t,m_t}_t)$
are also available for the same instruments
at the next time step $\tau_{t+1}$, denoted by $\mf^t_{t+1}$.
Instrument which expire between $\tau_t$ and $\tau_{t+1}$
will have their book value and
all greeks and scenario values set to zero.

It is a reasonable
assumption that those metrics $\mf$  have decent predictive power for
the behaviour of our instruments; after all this is what
human traders use to do drive their risk management decisions. Hence we will use them as instrument
features. We will here only consider linear features such that
for any weight vector $\mw \in \R^{m_t}$ the feature vector (the greeks, scenarios etc)
of the weighted instrument $\mw \cdot \mx^t$ is correctly given
as $\mw\cdot \mf^t$, so there is no need to recompute it
later.[We note that this linearity is satisfied for all common risk metric calculations
except VaR and counterparty credit calculations.]
We have referred to such a representation in [4] as
Finite Markov Representation, or short FMR.

We further denote by $r^i_t$
the historic aggregated cashflows of $x^{t,i}$
over the period $[\tau_t,\tau_{t+1})$, all in our accounting currency.
We set $\mr_t := (r^1_t,\ldots,r^{m_t}_t)$. The aggregated cashflows
of a weighted instrument $\mw \cdot \mx^{(t)}$ are $\mw\cdot \mr_t$.
Similarly, we use $\mB^t_u=(\mB^{t:1}_u,\ldots,\mB^{i,m_t}_u)$
to refer to the book values of our instruments in $u\in\{t,t+1\}$,

We also assume that we have for all our hedging instruments  access to
their respective feature vectors $\mf^{h:t}_t$ for both $\tau_t$
and $\tau_{t+1}$.
It is important to recall that the greeks $\mf^{h:t,i}_{t+1}$
refer to the features of the $i$th hedging instrument traded
at $\tau_t$, but computed at $\tau_{t+1}$. That means in particular
as the instrument definition changes between time steps.
We also denote by $\mb^{h:t}_u$ the book values of our hedging instruments
for $u\in\{t,t+1\}$.

In addition to our instrument features, we also assume that we chose a
reasonable subset of market features at each time step $\tau_t$.
We continue to use the symbol $\mm$ for those features even though in practise
we will not use the entire available state vector.

We will now generate random scenarios as follows

* Randomly choose $t\in\{0,\ldots,N-1\}$, which determines
the market states $\mm:=\mm_t$
and $\mm':=\mm_{t+1}$.

* Identify the hedging instruments $\mh$ with their
finite Markov representation

\[
\begin{array}{llcl}
\mbox{Terminal FMR of hedging instruments}  & \mh'       & :=&  \mf^{h:t}_{t+1} \\
\mbox{Book values for our hedging instruments} & \mB(\mh,\mm) & :=&  \mb^{h:t}_t \\
& \mB(\mh,\mm') & :=&  \mb^{h:t}_{t+1} \\
\mbox{Cashflows of our hedging instruments} & \mr(\mh,\mm) & :=&  r^{h:t}_t \\
\mbox{Cost}       & c(\ma;z,\mm) & \leftarrow & \ms_t, \mf^h_t
\end{array}
\]

The concrete implementation of the last line depends on the specifics of
the cost function. For example, proportional transaction cost on net
traded feature exposure
are implemented using a weight vector $\mgamma\in\R^F$
and setting $c(\ma;z,\mm) := |\ma \cdot (\mgamma\,\mf^t_t) |$.

* Choose a random weight vector $\mw\in\R^{m_t}$
and define a sample portfolio as $z:=\mw \cdot \mx$ with

\[
\begin{array}{llcl}
\mbox{Initial and terminal FMR of the portfolio} & z    & :=&  \mw\cdot \mf^t_t \\
& z' & :=&  \mw\cdot \mf^t_{t+1} \\
\mbox{Book value of our portfolio} & B(z,\mm) & :=&   \mw\cdot \mb^t_t\\
& B(z,\mm') & :=&  \mw\cdot \mb^t_{t+1} \\
\mbox{Cashflows of the portfolio} & r(z,\mm) & :=&  \mw\cdot \mx_t \ .
\end{array}
\]

The construction of a reasonable randomization of the weight
vector is important: if the samples are too different from likely
portfolios, then the resulting model will underperform.
However, if only historic portfolios are used, then the model
is less able to learn handling deviations. More importantly, though,
generating portfolios increases sample size.

This approach allows us training our actor-critic model with real data scenarios without the
need of a market simulator. Indeed, the use of a market simulator is difficult for this
particular set up as it would require computing book values and greeks for simulated data for a large
number of made up derivative instruments. An open research topic is whether it is feasible to write
a simulator for the data sets generated above, e.g. synthetically generating returns
of market data jointly with the feature vectors of instruments.

§ RELATION TO VANILLA DEEP HEDGING

We will now discuss the relation of
equation (<ref>)
to the solution of a corresponding vanilla Deep Hedging

We start by stating our original Deep Hedging problem [1]
adapting the notation used here so far.
We fix some initial time $t=0$
with market state $\mm\equiv \mm_0 \equiv \mM_0$.
Subsequent market states are denoted by $\mM_{t+1} := \mM_t'$.
We use a similar notation for all other variables. We use $\calT(\mM_t) := t$
as the operator which extracts from a state $\mM_t$ current calendar time.
We also define  the stochastic discount factor to zero
as $\beta_t:=\beta(\mM_{t-1}) \beta_{t-1}$ starting with $\beta_0:=1$.
We note that $\beta_1 = \beta(\mm)$ and $\beta_{\calT(\mM_t)} = \beta(\mM_{t-1}) \beta_{\calT(\mM_{t-1})}$.

For this part we will need to assume that
every hedging instrument has a time-to-maturity less than $\tau^*$ in the
sense that if we by $\ma\cdot \mh^t$ at time $t$, then all the book value and all cashflows
from the portfolio beyond $t+\tau^*$  are zero. This assumption excludes perpetual assets
such as shares or currencies. We will need to trade those with futures or
forwards in the current setup.

Assume then that we are starting with an initial portfolio $z$ and follow a trading
policy $\pi$. Accordingly, $Z_0 := z$ and $Z_{t+1} := Z_t' +\mA_t \cdot \mh^t{}'$
where $\mA_t := \pi(Z_t,\mM_t)$. We also use $\mS_t:=(Z_t,\mM_t)$ where convenient. Assume the portfolio has
maturity $T^*$ beyond which all cashflows are zero.

The total gains from trading $\pi$ starting in $z$ are given as

\begin{eqnarray}
G^\pi(z)&  := & \sum_{t=0}^\infty \beta_t\label{eq:discRDH}
R(\mA_t;Z_t,\mM_t,\mM_{t+1} )\\
& = &  	\nonumber
\underbrace{
- \mB(z,\ms_0)
\sum_{t=0}^{T^*}
\beta_t r(z,\mM_t)
}_{\mbox{P\&L from $z$}}
\\
&&						 + \nonumber
\sum_{t=0}^{\infty}
\beta_t
\Big( \underbrace{
- \mA_t\cdot \mB( \mh^t, \mM_t )
- c(\mA_t,\mS_t)
\mbox{Cost of trading}\\
\mbox{$\mA_t\cdot \mh^t$ in~$t$}
\end{array}}
\underbrace{
\mA_t\cdot
\sum_{u=t}^{t+\tau^*}
\frac{\beta_u}{\beta_t}
\mbox{All future rewards}\\
\mbox{from trading $\mA_t\cdot \mh^t$ in~$t$}
\end{array}}
\Big)
\end{eqnarray}

We now introduce the  discounted cashflow rewards

\[
\hat R(\ma;z,\mm) := \beta_{\calT(\mm)}\, \tilde R(\ma;z,\mm)  \supstack{\eqref{discrewards}}=  \beta_{\calT(\mm)}\, \left(\
r(z+\ma \cdot \mh,\mm) - \ma \cdot B(\mh,\mm) - c(\ma,z,\mm)
\ \right)
\]

such that

\[
G^\pi(z) = \sum_{t=0}^\infty \hat R(\mA_t;Z_t,\mM_t)  \ .
\]

We say that the market has only finite statistical arbitrage if we cannot make an infinite amount of money by trading
from an empty portfolio in our hedging instruments. Formally,

\[
\infty > \gamma := \sup_\pi \E\!\left[\, G^\pi(0) \,\right]  \ .
\]

If the market has only finite statistical arbitrage, then $U[\,G^\pi(z)\,]\leq \E[\,G^0(z)\,] + \gamma<\infty$ for all integrable $z\in\calX$ and
all trading policies $\pi\in\calP$.[
Since $U$ is risk-averse we have
$U[G^\pi(z)]=U[\,\sum_{t=0}^\infty \beta_t
R(\mA_t,\mS_t ) ] \leq \E[ \sum_{t=0}^\infty \beta_t
R(\mA_t,\mS_t ) ] = \E[\,G^0(z)\,] + \gamma< \infty$.]

That means the following definition makes sense:

The value function of the Vanilla Deep Hedging problem for an infinite trading
horizon expressed as in [1] in units of the underlying numeraire is given as
the finite
Û^*(z,) :=
U[					∑_t=0^∞R̂(π;Z_t,_t )
| Z_0=z,_0= ]
The actual cash value function is given as
U^*(z,) := Û^*(z,)  / β_τ(_0)

Simple arithmetic[
We may write out
U^*(z,)     =     sup_π:
U[					∑_t=0^∞R̂(π;Z_t,_t )
| Z_0=z,_0= ]

sup_∈ 	:
U[U[					∑_t=1^∞R̂(π;Z_t,_t )
| Z_1,_1 ] + R̂(,Z_0,_0) | Z_0=z,_0= ]

sup_∈ 	:
U[U^*( Z_1,_1) + R̂(,Z_0,_0) | Z_0=z,_0= ]
] shows that the value function for Deep Hedging solves
a Bellman equation:

Assume that the market is strictly free of statistical arbitrage, and that
$U$ is time-consistent (i.e. it is the entropy or the expectation). Then the  value function  $\hat U^*$ relative
to the underlying numeraire satisfies
the discounted dynamic programming equation

\begin{equation}\label{eq:bell_U_DH}
\left\{
\begin{array}{lcl}
\hat  U^*\big(\,
\,\big)
& \supstack!= &
(\hat T \hat U^*)(z,\mm) \\
\\
(\hat Tf)(z,\mm) & := &
\sup_{\ma\in \calA(z,\mm)}:\
z' +\ma\cdot \mh';\,\mM'
\,\big)
\,\big|\,\mm
\ \right] + \hat R(\ma;z,\mm) \ .
\end{array}
\right.
\end{equation}

for discounted rewards $\hat R$.

The cash value function $U^*(z,\mm):=\hat  U^*(z,\mm)/\beta_{\calT(\mm)}$ satisfies

\begin{equation}\label{eq:bell_U_DH_df}
\left\{
\begin{array}{lcl}
\,\big)
& \supstack!= &
(\tilde T U^*)(z,\mm) \\
\\
(\tilde Tf)(z,\mm) & := &
\sup_{\ma\in \calA(z,\mm)}:\
\frac1{\beta_{\calT(\mm)}}\  U\!\left[\ \beta_{\calT(\mM')}
z' +\ma\cdot \mh';\,\mM'
\,\big)
\,\big|\,\mm
\ \right] + \tilde R(\ma;z,\mm) \ .
\end{array}
\right.
\end{equation}

(note the presence of the non-discounted rewards $\tilde R$ instead of $\hat R$).

The first observation we make is that (<ref>) only reduces to our original (<ref>)
if $U$ is coherent. In this case, we can move $\frac1{\beta_{\calT(\mm)}}$ inside $U$.
However, we have already seen that the value function of the vanilla Deep Hedging problem only
solves (<ref>) if $U$ is the entropy or the expectation. Since the entropy
is not coherent, this means

The cash value function $U^*$ is only a solution to the Deep Bellman Hedging problem (<ref>)  if $U=\E$.

§.§ Solutions to the Vanilla Deep Hedging Bellman Equation

As in theorem <ref>, we now reverse the situation and ask under which circumstances (<ref>)
has a finite solution: recall that $\beta_{\calT(\mM_t)} = \beta(\mM_{t-1}) \beta_{\calT(\mM_{t-1})}$

Assume that rewards $\tilde R$ are finite and that $U$ is a monetary utility.

Then, the Vanilla Deep Hedging Bellman Equation (<ref>) has a unique finite solution.

If $U$ is the entropy or the expectation, then this optimal solution coincides with the
cash value function (<ref>) of the Vanilla Deep Hedging problem.

The proof for the existence of a unique finite solution is presented in section <ref>.

In light of this result it is evident that both the Deep Bellman Hedging equation (<ref>) and
the Vanilla Deep Hedging Bellman equation (<ref>)
are reasonable candidates for solving the generic hedging problem. However, the latter requires
us to essentially fix an initial point point $t=0$, upon which the numeraire $\beta_t$ is based.
In order to maintain consistence accross time, that initial time point would need to be kept
constant and therefore in the distant past.
We therefore recommend
using (<ref>) as presented.

§ EXISTENCE OF A  UNIQUE FINITE SOLUTION FOR DEEP BELLMAN HEDGING

We will now prove with theorem <ref>
convergence of our Deep Bellman Hedging
equations. This is easiest understood when the space $\calZ$ of
future cashflows is parameterized in $\R^{|\calZ|}$
with a finite Markov representation. However, in more generality
we may assume that $\calZ$ represents the set of suitably
adapted stochastic processes with values in $\R$.
Therefore, we may
just assume that $(\calS,\Q), \calS=\calZ\times\calM$
is a measure space. In the following we will
consider the function space $F$ by the $\Q$-equivalence
classes of functions $f:\calS\rightarrow \R$.

Let as before
(Tf)(z,) := sup_∈(z,):
U[ β()  f(
z' +·', ' ) + R(,z,,')                 |]
for $\beta(\mm)\leq\beta^*<1$.
Then the Bellman equation $f=Tf$ has a unique, finite

We will demonstrate the proof for bounded value
functions. See [19] on how
to extend results of convergence of Bellman operators
to the unbounded case. The below mimics the spirit
of the proof of the classic Banach contraction theorem.

equip $F$ with the supremum norm. We wish to
show that for $\|f\|_\infty<\infty$ we have $\| Tf \|<\infty$
We have

\[
Tf \leq T \| f \| = T0 + \| f \|
\]

because of monotonicity and cash-invariance.
Using (<ref>) we find that $T0<\infty$.

We will now show that $Tf$ is a contraction
for bounded $f:\calS\rightarrow \R$,
i.e. $\| Tf - Tg \|\leq \beta^* \| f - g \|$
for our $\beta^*<1$. Note that $f(\ms)-g(\ms)\leq \| f - g \|$.
Monotonicity and cash invariance of the operator $T$ yield

\[
(Tf)(\ms) \leq T\left( g + \| f - g \| \right)(x) \leq (Tg)(\ms) + \beta^*\| f-g \|
\]

\[
(Tg)(\ms) \leq T\left( f + \| f - g \| \right)(x) \leq (Tf)(\ms) + \beta^*\| f-g \| \ .
\]

Jointly this gives

\[
\| Tf - Tg \| \leq \beta^*\| f-g \| \ .
\]

Applying the Banach fixed-point theorem then yields the desired result.[
For illustration, we sketch a simple proof:
Chose $f_0$ and
let $f_n := Tf_{n-1}$
such that $f_n = T^n f_0$.
We know that $\| Tf_1 - Tf_0 \|
\leq \beta^* \| f_1 - f_0 \|
and therefore iteratively
$\| Tf_n - Tf_{n-1} \|
\leq \beta^*{}^n \| f_n - f_{n-1} \|
Triangle inequality implies $\| Tf_n - Tf_m \|
\leq \sum_{i=m+1}^n
\| Tf_i - Tf_{i-1} \|
\leq \| f_1 - f_0 \| \sum_{i=m+1}^n \beta^*{}^i
\downarrow 0$. This means $Tf_n$ is a Cauchy
sequence and therefore converges to a unique point
$f_n\rightarrow f$.

To show that $f$ is a fixed point
note that $\| Tf - f | \leq \| Tf - f_n \|
+ \| f_n - f \|
\leq \beta^* \| f - f_{n-1} \|  + \| f_n - f \|  \downarrow 0$.

§.§ Vanilla Deep Hedging Bellman Equation

We now focus on (<ref>) i.e. the operator

\[
(\tilde Tf)(z,\mm)  :=
\sup_{\ma\in \calA(z,\mm)}:\
\frac1{\beta_{\calT(\mm)}}\  U\!\left[\ \beta_{\calT(\mM')}
z' +\ma\cdot \mh';\,\mM'
\,\big)
\,\big|\,\mm
\ \right] + \tilde R(\ma;z,\mm) \ .
\]

With our previous assumptions is clear that $\tilde T0 < \infty$.
We now show that $\tilde T$ admits to a discounted form of cash invariance in the sense that

\[
T (f(\cdot)+y(z,\mm))(z,\mm) = (Tf)(z,\mm) + \beta(\mm) y(z,\mm)
\]

To this end, recall that $\beta_{\calT(\mM')} = \beta(\mm) \beta_{\calT(\mm)}$. Hence,
( T(  f(⋯) + y(z,)  ) )(z,)
sup_∈(z,):           1/β_()  U[ β() β_()
( f(⋯) + y(z,) )
|  ] + R̃(;z,)

(Tf)(z,) + β() y(z,)  .
The last equality follows from cash-invariance of the operator $U$.

Hence, just as in the preceding proof,

\[
(\tilde Tf)(\ms) \leq \tilde T\left( g + \| f - g \| \right)(x) \leq (\tilde Tg)(\ms) + \beta^*\| f-g \|
\]

which shows that $\tilde T$ is also a contraction. Applying the Banach fixed-point theorem
yields the existence of a unique solution.

§.§ Disclaimer

Opinions and estimates constitute our judgement as of the date of this Material, are for informational
purposes only and are subject to change without notice. It is not a research report and is not intended
as such. Past performance is not indicative of future results. This Material is not the product of J.P.
Morgan’s Research Department and therefore, has not been prepared in accordance with legal requirements
to promote the independence of research, including but not limited to, the prohibition on the dealing ahead
of the dissemination of investment research. This Material is not intended as research, a recommendation,
advice, offer or solicitation for the purchase or sale of any financial product or service, or to be used in
any way for evaluating the merits of participating in any transaction. Please consult your own advisors
regarding legal, tax, accounting or any other aspects including suitability implications for your particular
circumstances. J.P. Morgan disclaims any responsibility or liability whatsoever for the quality, accuracy or
completeness of the information herein, and for any reliance on, or use of this material in any way.
Important disclosures at: www.jpmorgan.com/disclosure

[1]
H. Buehler, L. Gonon, J. Teichmann, and B. Wood.
Deep hedging.
Quantitative Finance, 0(0):1–21, 2019.

[2]
Hans Buehler and Blanka Horvath.
Lecture notes learning to trade iii: Advanced topics and open
research questions, 2022.

[3]
Hans Buehler, Phillip Murray, Mikko S. Pakkanen, and Ben Wood.
Deep hedging: Learning to remove the drift.
Risk, March 2022.

[4]
Hans Buehler, Louis Moussu, and Ben Wood.
Method for optimizing a hedging strategy for portfolio management us
patent 16/744514, 2020.

[5]
Aharon Ben-Tal and Marc Teboulle.
An old-new concept of convex risk measures: The optimized certainty
Mathematical Finance, 17(3):449–476, July 2007.

[6]
H. Buehler.
Statistical hedging.
SSRN, 2017.

[7]
Jiayi Du, Muyang Jin, Petter N. Kolm, Gordon Ritter, Yixuan Wang, and Bofei
Deep reinforcement learning for option replication and hedging.
The Journal of Financial Data Science, 2(4):44–57, 2020.

[8]
Kai Detlefsen and Giacomo Scandolo.
Conditional and dynamic convex risk measures.
Finance and Stochastics, 9:539–561, 02 2005.

[9]
Hans Föllmer and Alexander Schied.
Stochastic Finance: An Introduction in Discrete Time.
De Gruyter, 2016.

[10]
Igor Halperin.
Qlbs: Q-learner in the black-scholes(-merton) worlds.
SSRN Electronic Journal, 12 2017.

[11]
Igor Halperin.
The qlbs q-learner goes nuqlear: fitted q iteration, inverse rl, and
option portfolios.
Quantitative Finance, 19:1543 – 1553, 2019.

[12]
V. Henderson and D. Hobson.
Utility indifference pricing: An overview, 01 2009.

[13]
Petter N. Kolm and Gordon Ritter.
Modern perspectives on reinforcement learning in finance.
Econometrics: Mathematical Methods & Programming eJournal,

[14]
Michael Kupper and Walter Schachermayer.
Representation results for law invariant time consistent functions.
Mathematics and Financial Economics, 2(3):189–210, 2009.

[15]
Terry Lyons, Sina Nejad, and Imanol Perez Arribas.
Non-parametric pricing and hedging of exotic derivatives.
Applied Mathematical Finance, 27:457 – 494, 2019.

[16]
Harry Markowitz.
Portfolio selection.
The Journal of Finance, 7(1):77–91, 1952.

[17]
Volodymyr Mnih, Adrià Puigdomènech Badia, Mehdi Mirza, Alex Graves,
Timothy P. Lillicrap, Tim Harley, David Silver, and Koray Kavukcuoglu.
Asynchronous methods for deep reinforcement learning, 2016.

[18]
Teemu Pennanen and Irina Penner.
Hedging of claims with physical delivery under convex transaction
SIAM Journal on Financial Mathematics, 1(1):158–178, 2010.

[19]
Juan Pablo Rincón-Zapatero and Carlos Rodríguez-Palmero.
Existence and uniqueness of solutions to the bellman equation in the
unbounded case.
Econometrica, 71(5):1519–1555, 2003.

[20]
Richard S. Sutton and Andrew G. Barto.
Reinforcement Learning: An Introduction.
The MIT Press, second edition, 2018.