# Mathematical Foundations of Regression Methods for Approximating the Forward
Initial Margin

Lucia Cipolina-Kun  University of Bristol. School of Computer Science,
Electrical and Electronic and Engineering Mathematics  Simone Caenazzo
Riskcare Ltd., London  Ksenia Ponomareva  Riskcare Ltd., London

###### Abstract

††The authors would like to thank Eusebio Gardella and Bertrand Nortier for
their contribution to the final manuscript.

In recent years, forward initial margin has attracted the attention of
practitioners and academics. From a computational point of view, it poses a
challenging problem as it requires the implementation of a nested Monte Carlo
simulation. Abundant literature has been published on approximation methods
aiming to reduce the dimensionality of the problem, the most popular ones
being the family of regression methods.

This paper describes the mathematical foundations, on which these regression
approximation methods lie. Mathematical rigor is introduced to show that, in
essence, all methods are just different variations of approximating the
conditional expectation function. In short, all methods are performing
orthogonal projections on Hilbert spaces, while simply choosing a different
functional form to numerically estimate the conditional expectation.

The most popular methods in the literature so far are covered here. These are
polynomial approximations, Kernel regressions and Neural Networks.

##  1 Introduction

Initial margin (IM) has become a topic of high relevance for the financial
industry in recent years. The relevance stems from its wide implications to
the way business will be conducted in the financial industry and in addition,
for the need of approximation methods to calculate it.

The exchange of initial margin is not new in the financial industry, however,
after regulations passed in 2016 by Financial Regulatory Authorities [5], a
wider set of financial institutions are required to post significant amounts
of initial margin to each other.

The initial margin amount is calculated at the netting set level following a
set of rules depending on whether the trade has been done bilaterally [13] or
facing a Clearing House [7]. The calculation is performed on a daily basis and
carried on throughout the life of the trade.

Since each netting set consumes initial margin throughout its entire life,
financial institutions need resources to fund this amount in the future, given
that the margin received from its counterparty cannot be rehypothecated. To
estimate the total need for funds, each counterparty needs to perform a
forecast of the initial margin amount up to the time to maturity of the
netting set.

Forecasting the total forward initial margin amount is a computationally
challenging problem. An exact initial margin calculation requires a full
implementation of a nested Monte Carlo simulation, which is computationally
onerous given the real-time demand for the IM numbers as part of day-to-day
business activities.

###  1.1 Literature Review

As a response to this computational challenge, practitioners and academics
have proposed several approximation methods to reduce the dimensionality of
the problem, at the cost of losing a tolerable degree of accuracy. The most
popular of these approaches have been the regression methods, since they offer
simplicity and re usability of the Bank’s legacy Monte Carlo engines, paired
with good results. Inspired by early works of Longstaff-Schwartz [16], several
banks have implemented some version of the polynomial regression as proposed
by [1], [8] and [10], or the Kernel regressions proposed by [2], [10] and
[11]. Most recently, estimation by Neural Networks has been proposed by [12]
and [18].

These methods have not been developed exclusively to tackle the initial margin
problem. These are well-known regression methodologies and have been used
extensively in finance and engineering. As such, the current literature on
initial margin have focused on the implementation and numerical validation of
the regression methods from a practical point of view. The authors’ main focus
have been to show, the applicability and performance of the different
methodologies when compared to a brute-force benchmark, without going deep
into mathematical formalities.

This paper aims to fill the gap between theory and practice. The underlying
mathematical framework required to analyze the initial margin computation
problem from a theoretical standpoint is presented here.

###  1.2 Organization

The rest of this paper is organized as follows. Section 2 translates the
practitioner’s definition of initial margin into a formal probability setting.
Section 3-4 develop the assumptions needed to fit the initial margin problem
into a classical problem of orthogonal projections into Hilbert spaces. The
last section concludes by linking back the formal theory presented with the
practitioner’s suggested approaches.

##  2 Description of the Initial Margin Problem

###  2.1 Definition of Forward Initial Margin

Following the definition in [6], the aim of a Bank is to calculate the total
cost associated with the posting of initial margin to a counterparty over the
life of a netting set. Since this is a quantity that evolves in time, it is
necessary to consider a forward diffusion model, together with a probability
space and the relevant discounting and funding rates into the equation.

Given a filtered probability space
$(\Omega,\mathcal{F},\left\\{{\mathcal{F}}_{{t}}\right\\}_{{t\geq
0}},\mathbb{P})$ where as usual, $\Omega$ is the set of all possible sample
paths $\omega$, $\mathcal{F}$ is the sigma-
algebra,$\left\\{{\mathcal{F}}_{{t}}\right\\}$ is the filtration of events at
time $t$ (i.e. the natural filtration to which risk factors are adapted to)
and $\mathbb{P}$ the probability measure defined on the elements of
$\mathcal{F}$. The total cost of posting initial margin over the entire life
of a netting set is referred as the Margin Valuation Adjustment as defined
below.

$MVA_{C}(t)=\mathbb{E}_{t}\left[\int_{t}^{T}((1-R_{C})\lambda(u)-S_{I}(u))e^{\int_{t}^{u}(r(s)+\lambda_{B}(s)+\lambda_{C}(s))ds}\mathbb{E}_{u}\left[IM_{C}(u)\right]du\right]$

where:

  * •

$\mathbb{E}_{t}$ denotes an expectation conditional to information at time
$t$, i.e.
$\mathbb{E}_{t}\left[\dots\right]=\mathbb{E}\left[\dots\mid\mathcal{F}_{t}\right]$

  * •

$R_{C}(t)$ is the recovery rate at time $t$ of the counterparty related to
netting set $C$;

  * •

$T$ is the final maturity of netting set $C$;

  * •

$\lambda(t)$ is the Bank’s effective financing rate (i.e. borrowing spread) at
time $t$;

  * •

$S_{I}$ is the spread received on initial margin;

  * •

$r(t)$ is the risk free rate (e.g. an OIS rate) at time $t$;

  * •

$\lambda_{B}(t),\lambda_{C}(t)$ are the survival probabilities at time $t$ of
the Bank and the counterparty related to netting set $C$, respectively;

  * •

$IM_{C}(t)$ is the initial margin posted by the bank at time $t$ against
netting set $C$.

Note that the expectation $\mathbb{E}_{t}$ is taken on the whole integral.
This is because certain simulation setups, like the one in [21] and [22],
feature explicitly modeled credit factors that lead to stochastic credit
spreads and survival probabilities. If the simulation setup instead considers
deterministic credit factors, the formula can be simplified as follows:

$\displaystyle MVA_{C}(t)$ $\displaystyle=$
$\displaystyle\int_{t}^{T}((1-R_{C})\lambda(u)$ $\displaystyle-$
$\displaystyle
S_{I}(u))e^{\int_{t}^{u}(\lambda_{B}(s)+\lambda_{C}(s))ds}\mathbb{E}_{t}\left[\mathbb{E}_{u}\left[IM_{C}(u)\right]e^{\int_{t}^{u}r(s)ds}\right]du$

Here our quantity of interest is $\mathbb{E}_{t}\left[IM(t)\right]$. This is
the expectation of the initial margin at time $t$, taken across all random
paths $\omega$, conditional on the realization of the risk factors at time
$t$. The initial margin is a path-wise random variable defined as 111This is
the simplest definition of initial margin, for alternative definitions see [3]

$P(\Delta_{t}\leq IM(\omega,t)|\mathcal{F}_{t})=p$ (1)

Where:

  * •

$p$ denotes the desired quantile (usually the $99\%$ quantile);

  * •

$\Delta(\omega,t,\delta_{IM})$ is defined (in its simplest form222This is the
simplest definition of netting set PnL, for further definitions see [3].) as
the change, conditional to counterparty default, in the netting set’s value
(i.e. profit and loss (PnL)) over a time interval $(t,t+\delta_{IM}]$:

$\Delta(\omega,t,\delta_{IM})=V(\omega,t+\delta_{IM})-V(\omega,t)$

Note that the Initial Margin as defined above is specifically the _posted_ IM,
as the quantile is calculated on the upper-tail of the PnL distribution. In
order to calculate the _received_ IM, one can instead compute the $1\%$
quantile on the same distribution.

The following sections describe the computational methods developed in the
current literature to estimate this quantity.

###  2.2 The Brute Force Approach to Calculate Initial Margin

The most exact method to calculate the initial margin requires a brute force
implementation of a nested Monte Carlo simulation. Since this approach is
computationally untractable, in practice it is only used as a ground truth
benchmark to assess the performance of the approximation methodologies.

Figure 1 exemplifies the situation. For a given point in time, $t_{1}$, an
outer Monte Carlo simulation for the values of $V(\omega,t_{1})$ and an inner
Monte Carlo simulation of the forward Probability Density Function (PDF) of
$V(\omega,t_{1}+\delta_{IM})\mid\mathcal{F}_{t_{1}})$ have been performed. The
difference in the value of the netting set between $t_{1}$ and
$t_{1}+\delta_{IM}$ time steps is $\Delta(\omega,t_{1},\delta_{IM})$. Since
the PDF distribution of $(V(\omega,t_{1}+\delta_{IM})\mid\mathcal{F}_{t_{1}})$
is known, it is possible to obtain the PDF distribution of
$\Delta(\omega,t_{1},\delta_{IM})$. Then, one should take the
$\mathcal{Q}_{99}$ to obtain the initial margin.

Figure 1: A simulation tree with two branches and two time steps

The initial margin at every time step $t$ is the $99^{th}$ percentile of the
inner PDF scenarios. Both processes (inner and outer) are usually simulated
using the same stochastic differential equation with a sufficiently large
number of scenarios to ensure convergence. The number of scenarios become
particularly important for the inner scenarios as the $Q_{99}$ is a tail
statistic, therefore the number of scenarios on the tail should be
sufficiently large.

As every nested Monte Carlo problem, the brute force calculation of initial
margin is computationally onerous given that the number of operations increase
exponentially. This type of complexity is not possible to afford in real time,
e.g. in a live trading scenario, thus the reason behind the development of
approximation methods.

##  3 Regression Methods for the Approximation of Initial Margin

The regression proposed by Longstaff-Schwartz in [16] is the traditional
framework used in finance to reduce dimensions in nested Monte Carlo problems.
In short, the authors propose to reduce the number of machine operations by
simulating only the outer scenarios and fitting a regression to forecast the
inner ones. Their regression uses a polynomial basis as the functional choice
for the regression.

The use of regressions is justified since the authors’ mathematical problem is
the estimation of a conditional expectation on the inner scenarios, which is
usually approximated with a linear regression. Initial margin computation
problem posed in this paper requires approximation of a quantile. It will be
shown that the general Longstaff-Schwartz framework can be applied to this
problem as well.

Figure 2 provides a graphical description of the approach followed to
approximate a nested Monte Carlo problem. The main idea is to simulate only
the outer scenarios and assume a parametrized PDF for the inner ones. Under
this setting, one can use a regression to estimate the required quantile.

Figure 2: Simplification of a nested Monte Carlo by applying distributional
assumptions

From equation 1, the inner scenarios are used to calculate the PDF of the
change in the portfolio value, conditional on the information provided on each
filtration set $\mathcal{F}_{t}$. From this PDF, one can obtain the $Q_{99}$
that makes the initial margin amount. However, when only the outer scenarios
are simulated, there is only a single point of the empirical distribution per
branch left, which is not enough to build an entire PDF. To overcome this
limitation, the relevant literature propose to assume a parametric
distribution for the rest of the unknown inner scenarios. The majority of
authors propose modelling the quantity $[V(\omega,t+\delta_{IM})-V(\omega,t)]$
with a local Normal distribution, since it can be conveniently fitted by only
two parameters, the mean and the variance. This choice is justified under the
assumption that the simulated $V(\omega,t)$ come from a Normal SDE and the
time lapse between observations $\delta_{IM}$ is small. Other authors like [4]
propose a fat-tailed $\mathcal{T}$ distribution.

If it can be assumed that the inner scenarios follow a local Normal
distribution, the only task left is the calculation of the distribution’s
expectation and variance to then obtain the initial margin as:

$(IM(\omega,t)\mid\mathcal{F}_{t})=\sigma(\omega,t)\cdot\Phi^{-1}(99\%)\mid\mathcal{F}_{t}$

Where for simplicity and without loss of generalization, we have assumed that
the data is centered and thus the expectation is zero. Under this assumption,
the term $\sigma(\omega,t)$ can be written as:

$(\sigma(\omega,t)\mid\mathcal{F}_{t})=\sqrt{\mathbb{E}\left[\Delta^{2}(\omega,t,\delta_{IM})\mid\mathcal{F}_{t}\right]}$
(2)

Equation 2 shows that, when restated in this way, a quantile estimation
problem has been transformed into the conditional expectation problem, which
ends up in the same setting as the Longstaff-Schwartz model. The next sections
study different approximations of
$\mathbb{E}\left[\Delta^{2}(\omega,t,\delta_{IM})\mid\mathcal{F}_{t}\right]$,
which is the conditional expectation of the netting set’s PnL under the
information given at time $t$.

##  4 Conditional Expectations as Orthogonal Projections on Hilbert Spaces

###  4.1 Conditional Expectations as Orhtogonal Projections

The task is to find the best (i.e. unbiased) estimator of the conditional
expectation in equation 2 at every time step $t$. The proposal is to
approximate the conditional expectation by regressing
$\Delta^{2}(\omega,t,\delta_{IM})$ against a set of basis functions. However,
this use of regressions to approximate conditional expectations needs to be
formally justified.

The intuition behind the connection between conditional expectations and
regressions is to frame conditional expectations as functions mapping a set of
vectors in $\mathcal{L}^{2}$ onto a given set of vectors in the same space.
The the particular form of these functions is an arbitrary choice, and the
coefficients are calibrated following the geometric properties of
$\mathcal{L}^{2}$ spaces.

###  4.2 Properties of Conditional Expectations in Hilbert Spaces

The connection between conditional expectations and regressions is proved in
two steps. Firstly, the theoretical framework is described in order to
characterize conditional expectations as orthogonal projections on Hilbert
spaces, and, secondly, orthogonal projections are connected to regressions in
an empirical step.

The conditional expectation is expressed as a function of a set of vectors
$x_{t}$ obtained from the filtration,

$\mathbb{E}\left[\Delta^{2}(\omega,t,\delta_{IM})\mid\mathcal{F}_{t}\right]=m(x_{t})$
(3)

where $x_{t}$ a short for $span(x_{1},...,x_{T})$, an
$\mathcal{F}_{t}$-adapted process and $m(x_{t})$ is a stochastic function with
certain properties described in the following sections.

In equation 3, the set of $(x_{1},...,x_{T})$ regressors have been fixed by
conditioning on information obtained from the filtration at time $t$, and
therefore, $\mathbb{E}\left[\Delta^{2}(\omega,t,\delta_{IM})\right]$ is
projected on a fixed space. In this setting, a conditional expectation is a
function of the vectors $(x_{1},...,x_{T})$ and not just a random variable, as
it is typically defined in option pricing. By interpreting conditional
expectations as functions, the properties inherited from the functional space
they belong to can be used. In particular, the properties regarding orthogonal
projections and decomposition into orthogonal bases from $\mathcal{L}^{2}$ .

In order to set up an appropriate functional space, the space of random
bounded functions is considered on the filtered probability space
$\mathcal{L}^{2}$$(\Omega,\mathcal{F},\left\\{{\mathcal{F}}_{{t}}\right\\}_{{t\geq
0}},\mathbb{P})$. Thus, only square-integrable functions, satisfying
$\mathbb{E}[y^{2}]<\infty$ for every $y\in\mathcal{L}^{2}$ (i.e. finite second
moment), are considered. As such, the inner product of two elements $y_{1}$
and $y_{2}$ is $\mathbb{E}[y_{1}.y_{2}]$, with the induced norm $\parallel
y\parallel=\sqrt{\mathbb{E}[y^{2}]}$.

Following the definition 4.1 below, under $\mathcal{L}^{2}$ spaces conditional
expectations are equivalent to linear maps that perform orthogonal projections
between a space and a subspace 333While conditional expectations are well
defined in $\mathcal{L}^{1}$, this is not an inner product space, only the
$\mathcal{L}^{2}$ space defines an inner product that can be used to meet
orthogonality requirement..

######  Definition 4.1.

Given a $\sigma$-algebra $\mathcal{F}$, there is a closed subspace
$\mathcal{L}^{2}$ $(\Omega,X,\mathbb{P})$ $\subset\mathcal{L}^{2}$
$(\Omega,\mathcal{F},\mathbb{P})$, each equipped with its filtration. Suppose
that $y\in\mathcal{L}^{2}$ $(\Omega,\mathcal{F},\mathbb{P})$, the conditional
expectation $\mathbb{E}[y\mid X]$ is defined to be the orthogonal projection
of $y$ onto the closed subspace $\mathcal{L}^{2}(\Omega,X,\mathbb{P})$.

The next step is to find a functional form for the projection function and
this is done by choosing an orthogonal basis for the $span(x_{1},...,x_{T})$.
While the basis vectors can be chosen arbitrarily, the vector weights (i.e.
coefficients) must be chosen so that the resulting projection vector is
orthogonal to the original space444The linear combination is guaranteed to
exist since every Hilbert space admits an orthonormal basis (Zorn’s Lemma) and
it is unique (by the theorem of characterization of orthogonal projections
(see [15]). Moreover, if the space projected onto is separable, the
combination of basis functions needed to obtain this projected vector is
finite. .

The challenge under a Monte Carlo simulation setting is to find a finite set
of suitable basis functions to represent the projected vectors. The following
sections expand on the most common empirical methodologies used for this mean.

##  5 Approximation Methodologies for the Estimation of Conditional
Expectations

This section deals with the practical implementation of the theory above, in
particular, the choice of regressors and basis functions. The described theory
of Hilbert spaces can be considered as the population version, where
conditional expectations are equivalent to orthogonal projections, while the
simulated vectors are the sample version. The spaces are now Euclidean (i.e.
finite) and conditional expectations are estimated with regressions.

###  5.1 The Choice of Regressors

Under a Monte Carlo simulation, the independent $X_{t}$ and dependent
variables $Y_{t}$ are empirically generated following an arbitrary choice of
SDE. In particular, the discrete time filtration for both is obtained.

The key art of estimating regressions is the choice of the regressors $X_{t}$.
The output of the Monte Carlo simulation contains several variables that can
be used as regressors. As explained in the previous sections, in theory it is
desirable to find an orthogonal basis to decompose the projected vectors, in
practice, the choice of regressors is an empirical call. However, some minimal
conditions must be met.

###  5.2 Markovian Regressors

In choosing the regressors, there should be a sound relationship between the
projected vector $Y_{t}$ and the projection vectors $(X_{t})_{0\leq t\leq T}$
obtained from the filtration. The second aspect to determine is the relevance
of past data as predictors of the dependent variable, $Y_{t}$. This is, how
much of the past history coming from the filtration is relevant. For Markov
chains, only the most recent values of the state variables are necessary,
while for non Markov, past realizations of the filtrations should be included
as regressors.

If the variables forming the filtration are Markovian, then the conditional
expectation can be estimated as:

$\mathbb{E}[Y_{t+1}\mid\mathcal{F}_{t}]=\mathbb{E}[Y_{t+1}\mid X_{t}]$ (4)

If the $(X_{t})_{0\leq t\leq T}$ process has stochastic independent
increments, then it is a Markov process ([15]).

###  5.3 The Choice of the Basis Functions

The choice of basis functions depends on the projection subspace
$span(x_{1},...,x_{T})$ and mostly, on the empirical relationship between the
independent and dependent variables $X_{t}$ and $Y_{t}$.

The underlying idea behind the different approximation methods is to find an
approximation for $\mathbb{E}[Y_{t+\delta}\arrowvert X=X_{t}]=m(X_{t})$ that
solves the below minimization problem:

Let $Y_{t+\delta}$ be a random vector in a Hilbert space and $(X_{t})_{0\leq
t\leq T}$ a vector in a closed Hilbert subspace. Additionally, assume
$m(X_{t})$ is a given a function of $span(X_{t})$. Solve the minimization
problem below to find $\hat{\beta}\subset\mathcal{R}^{n}$ such that:

$\underset{\hat{\beta}}{inf}\;\mathbb{E}[(Y_{t+\delta}-m(X_{t},\hat{\beta}))^{2}]$
(5)

where the function $m(\cdot)$ can be decomposed into basis functions as
explained in Section 4.2.

###  5.4 Empirical Relationship Between the Dependent and Independent
Variables

The choice of functional form to represent the relationship between the
vectors $Y_{t}$ and $X_{t}$ is an empirical call, mostly based on data
analysis. The question is, which would be the best choice of basis functions
such that $m(X)\colon X\to Y$ agrees with our simulation data?

In practice, under a nested Monte Carlo simulation, basis functions are
selected arbitrarily so that the resulting functional form of the conditional
expectation function represents appropriately the relationship between the
vector $(Y_{t})_{n}$ and the filtration vector $(X_{t})_{n}$, where $n$ is the
dimension of the simulated vectors at time $t$.

From the Monte Carlo simulation engine, one can easily obtain the simulated
variables $V(\omega,t)$ and $V(\omega,t+\delta_{IM})$ from which
$\Delta^{2}(\omega,t,\delta_{IM})$ can be calculated at every $t$. The natural
choice used in the literature is to choose $V(\omega,t)$ as the regressor for
$\Delta^{2}(\omega,t,\delta_{IM})$. This assumes that $V(\omega,t)$ is a good
predictor for $\Delta^{2}(\omega,t,\delta_{IM})$ as in equation 4. In what
follows, the following terms will be used:

$X_{t}={V(\omega,t)}_{n}=[V(\omega_{1},t),V(\omega_{2},t),...,V(\omega_{n},t)]^{T}$

$Y_{t+\delta}={\Delta^{2}(\omega,t,\delta_{IM})}_{n}=\left({V(\omega,t+\delta_{IM})}_{n}-{V(\omega,t)}_{n}\right)^{2}$

where $n$ is the number of simulated scenarios, $\omega$.

In order to choose the functional relationship between $V(\omega,t)$ and
$\Delta^{2}(\omega,t,\delta_{IM})$ one must remember from the previous section
that the only available data are the simulations from the outer scenarios, and
from this information it is necessary to infer the relationship between the
$V(\omega,t)$ and $\Delta^{2}(\omega,t,\delta_{IM})$ on the inner scenarios.
The assumption is that the functional relationship between the $V(\cdot)$ and
$\Delta^{2}(\cdot)$ in the inner and outer scenarios is the same. In other
words, it is assumed that the model we adjust with data coming from the outer
scenarios simulation can be used to infer the inner scenarios. This assumption
is based on the fact that the same SDE is used to simulate both the inner and
outer scenarios (provided observer is at a given $t$).

From the graphs in [10] we can study the relationship between $V(\cdot)$ and
$\Delta^{2}(\cdot)$ for different type of regression models. From the cloud of
points, the functional form is rather un-conclusive.

Figure 3:  A comparison of scatter plots of portfolio value vs simulated
initial margin and PnL for initial margin models. Taken from [10].

Lastly, a usual simplification is to assume that the set of basis functions
does not change over time, but weights do. Hence, after a functional form of
the conditional expectation has been chosen, the problem is to estimate the
weights at every simulation time step $t$, conditional on the information
given by the filtration up to that time.

###  5.5 Necessary conditions on MtM processes

In earlier sections it has been stated that the analysis should be restricted
to square integrable functions with finite second moment (variance) - in the
available literature this point is assumed to hold without proof, especially
with respect to the squared PnL random variable $Y_{t}$. Now formal derivation
of two conditions that a generic Mark-to-Market (MtM) random variable has to
satisfy in order for $Y_{t}$ to be square integrable will be provided. It will
be assumed that for the MtM to be a real random variable, as must be the case
for the financial setting we are analysing.

####  5.5.1 Theorems

This section begins with stating of the theorems on square-integrable
variables that will be used in the rest of the derivations.

######  Theorem 5.1.

If a real random variable $X$ has finite mean and variance, it is square
integrable:

$\mathbb{E}[X]<\infty,Var[X]<\infty\Longrightarrow\mathbb{E}[\lvert
X\rvert^{2}]<\infty$

###### Proof.

The variance of $X$ is by definition:

$Var[X]=\mathbb{E}[X^{2}]-\mathbb{E}^{2}[X]$

If $Var[X]<\infty$, then all terms in the RHS must be finite. It is known that
$\mathbb{E}[X]<\infty$, therefore $\mathbb{E}^{2}[X]<\infty$. It must follow
that $\mathbb{E}[X^{2}]<\infty$ and therefore $\mathbb{E}[\lvert
X\rvert^{2}]<\infty$.

∎

######  Remark.

Note that $\mathcal{L}^{2}$ is formally defined as the space of square
integrable functions. Since a space of random variables is considered, from
now on these two concepts will be used interchangeably.

######  Theorem 5.2.

Sums or scalar multiplications of square integrable variables are also square
integrable.

###### Proof.

The theorem for a generic sum $Z=X+Y$ of two square integrable random
variables $X$ and $Y$ will be proved. Let’s express the variance of $Z$:

$Var(Z)=\mathbb{E}[(X+Y)^{2}]-\mathbb{E}^{2}[X+Y]\\\
=\mathbb{E}[X^{2}]+\mathbb{E}[Y^{2}]-\mathbb{E}^{2}[X]-\mathbb{E}^{2}[Y]-2\mathbb{E}[X]\mathbb{E}[Y]+2\mathbb{E}[XY]$

In the above, the first two terms are finite as $X$ and $Y$ are square-
integrable. The third and fourth terms are also finite as $X$ and $Y$ must
have finite mean. Finally, the Cauchy-Schwarz inequality can be used to infer
that the last term must also be finite, as its absolute value is bounded by
the square root of the product of two finite terms:

$\lvert\mathbb{E}[XY]\rvert\leq\sqrt{\mathbb{E}[X^{2}]\mathbb{E}[Y^{2}]}$

Since the variance is finite, we can refer to Theorem 5.1 and conclude that
$Z$ must also be square integrable. The result is trivially extendable, via
induction arguments, to multiple sums of square integrable variables as well
as scalar multiplications thereof. ∎

####  5.5.2 Derivation of conditions

Recalling that the generic form of the squared PnL random variable
$Y_{t+\delta_{IM}}$ is:

$Y_{t+\delta_{IM}}={\Delta^{2}(\omega,t,\delta_{IM})}_{n}=\left({V(\omega,t+\delta_{IM})}_{n}-{V(\omega,t)}_{n}\right)^{2}\\\
={V^{2}(\omega,t+\delta_{IM})}_{n}-2{V(\omega,t+\delta_{IM})}_{n}{V(\omega,t)}_{n}+{V^{2}(\omega,t)}_{n}$

It can be observed that, without loss of generality,
$Y_{t}\sim\mathcal{O}({V^{2}(\omega,t)}_{n})$. By virtue of Theorem 5.2 it can
be concluded that if the generic squared MtM random variable
${V^{2}(\omega,t)}_{n}$ is square-integrable, so will be $Y_{t}$. In turn, by
virtue of Theorem 5.1, this only leaves the need to state the conditions on
${V(\omega,t)}_{n}$ for which the variance of ${V^{2}(\omega,t)}_{n}$ is
finite. The variance of ${V^{2}(\omega,t)}_{n}$ is:

$Var({V^{2}(\omega,t)}_{n})=\mathbb{E}[{V^{4}(\omega,t)}_{n}]-\mathbb{E}^{2}[{V^{2}(\omega,t)}_{n}]$

This variance is finite i.f.f. the two terms on the RHS are finite, yielding
the following two conditions on ${V(\omega,t)}_{n}$:

######  Proposition 5.1.

${V(\omega,t)}_{n}$ must have finite mean and variance. This ensures that
${V^{2}(\omega,t)}_{n}$ is square integrable and therefore
$\mathbb{E}^{2}[{V^{2}(\omega,t)}_{n}]<\infty$.

######  Proposition 5.2.

${V(\omega,t)}_{n}$ must have finite fourth moment. This ensures that
$\mathbb{E}[{V^{4}(\omega,t)}_{n}]<\infty$.

###  5.6 Analytical Approximations for the Conditional Expectation Function

Several methods can be used to estimate the conditional expectation function
$\mathbb{E}[Y_{t+\delta}\arrowvert X=X_{t}]=m(X_{t})$, depending on the
properties of the underlying data. A good summary of the available
methodologies for nested Monte Carlo problems can be found in [9]. The next
sections discuss the most popular approaches used in the context of initial
margin.

####  5.6.1 Algorithms Based on Linear Maps

In the context of initial margin estimation, linear regression methods are
proposed by [1], [8] and [10]. The problem setting is as described in equation
5. In the case where the Hilbert subspace $\mathcal{F}_{t}$ is a linear vector
space, the estimate can be computed solving a linear equation system. Linear
maps include the ordinary least square regression and polynomial functions.

The projection is done onto a linear independent set of random variables
$(X_{t})$.

where $m(X,\hat{\beta}))$ is a linear map of the form:

$m(X_{t},\hat{\beta})=\sum_{i=0}^{P}\hat{\beta}_{i}.\phi_{i}(X_{t})=\mathbb{B}^{T}.\phi_{i}(X_{t})$
(6)

where $\phi_{i}(X_{t})$ are the $P$ set of linear basis functions arbitrarily
chosen.

A linear problem like this can be solved by inverting matrices. If $Y$ and $X$
are jointly Gaussian distributed, then the setting is of the well known Normal
Equations. For example, in [1] $P=2$ and $\phi_{i}(X_{t})=X_{t}^{i}$.

In the non-linear methods below, the vectors spaces (and thus the function
$m(X,\hat{\beta})$) are non-linear and thus, the Least Ordinary Square
approach does not work.

####  5.6.2 Algorithms Based on Non Parametric Regressions

Among the vast amount of non-parametric algorithms available, kernel
regressions have been the most popular ones. In particular the Naradaya-Watson
kernel [17] has been used by [2], [10] and [11] in the context of initial
margin estimation.

In principle, Naradaya-Watson proposed an algorithm to approximate the
empirical joint density distribution of two random variables. This is, instead
of estimating a conditional expectation directly, the method estimates the
entire joint pdf distribution,from which the conditional expectation is
derived. The method calculates a locally weighted average of the regressors
given a choice of a kernel function. The weights provide a smoothing feature
to the fit as they decrease smoothly to zero with distance from the given
point. The derivation of the Naradaya-Watson pdf estimator can be found in
[14]. Under this method, the conditional expectation function can be
approximated as 555”Kernel” functions can be seen as data ”filters” or
”envelopes”, in the sense that they define proximity between points. See [14]
for a derivation.

$\mathbb{E}[Y\arrowvert
X=x]=\frac{\sum_{i=1}^{N}K_{h}(x,x_{i})y_{i}}{\sum_{j=1}^{N}K_{h}(x,x_{j})}$
(7)

where $h$ is the bandwidth parameter that controls the degree of smoothing,
$x$ is the centroid, $x_{i},y_{i}$ are elements of the vectors $Y_{t},X_{t}$
and $K(.)$ is a Kernel function satisfying: :

  1. 1.

symmetry: $k(-u)=k(u)$

  2. 2.

positive semi-definiteness: $k(u)\geq 0\,\forall u$

  3. 3.

$\int_{-\infty}^{+\infty}k(u)du=1$

While equation 7 is non-linear for the regressors, it can be shown that the NW
estimator is a particular case of the local estimation problem. The function
above can be re-expressed as being locally linear (i.e. piece-wise linear) on
a certain set of $\beta$ parameters. Moreover, it will be shown that the
conditional expectation estimator proposed by kernel regressions can be framed
as a particular case of the orthogonal projection problem in equation 5 with a
modified objective function and a weighted norm.

The local minimization problem works by setting a small neighborhood of a
given centroid $x$ and assuming a parametric function within the selected
region. The process is equivalent to partitioning the support of the
regressors space producing smooth step functions. For every time step $t$ of
the Monte Carlo simulation, consider a neighborhood of $x$ and assume a local
functional relationship $m(.)$ between the $Y,X$ variables. If $m(.)$ is
continuous and smooth, it should be somewhat constant over a small enough
neighborhood (by Taylor expansion). While linear regressions assume a linear
relationship between variables and minimize a global sum of square errors,
kernel regressions assume a linear relationship only on a small neighborhood,
focusing on the modeling of pice-wise areas of the observed variables, and
repeating the procedure for all centroids. The trick is to determinze the size
of the neighborhood to trade off error bias with model variance. The local
minimization problem is now as follows:

Given a random vector $Y_{t}\in\mathcal{H}$ and a deterministic vector
$X_{t}\subset\mathcal{H}$, and given the functions $K_{h}(\cdot)$ and
$m(\cdot)$, find $\hat{\beta}$ such that:

$\underset{\hat{\beta}}{inf}\;\mathbb{E}[K_{h}(X_{t},Y_{t+\delta}).(Y_{t+\delta}-m(X_{t},\hat{\beta}))^{2}]$

In the local minimization problem, the model for $m(x)$ is a linear expansion
of basis functions as in equation 6.

$m(X_{t},\hat{\beta})=\sum_{i=0}^{P}\hat{\beta}_{i}.\phi_{i}(X_{t})$

Let $p$ be the order of the $\phi(X_{t})$ polynomial, some examples of $m(.)$
are:

  * •

if p = 0, then $m(X_{t},\hat{\beta})=\beta_{0}$, i.e. the constant function
(i.e. intercept only). This results in the NW estimate in equation 7.

  * •

if $p=1$ then $m(X_{t},\hat{\beta})=\beta_{0}+\beta_{1}(x_{i}-x)$ with both
$x_{i}$ and the centroid $x$ $\in X_{t}$ gives a local linear regression.

The $\hat{\beta}$ are solved by weighted least squares. As with the linear
case, the vector $W(Y_{t+\delta}-m(X_{t},\hat{\beta}))$ should be orthogonal
to all other vectors of X, but now the relevant projection is orthogonal with
respect to this new inner product:

$\langle W(Y_{t+\delta}-m(X_{t},\hat{\beta})),m(X_{t},\hat{\beta})\rangle$ (8)

where $W$ is a sequence of weights
$W(X)=\operatorname{diag}\left(K_{{h_{{\lambda}}}}(X,X_{i})\right)_{{N\times
N}}$.

The solution is given by:

$\hat{\beta}=\left(X^{T}WX\right)^{-1}\left(X^{T}WY\right)$ (9)

We can obtain the same expression for $m(x)$ as in equation 7:

$m(X_{t})=\beta_{0}=\left[\left(X^{T}WX\right)^{-1}X^{T}W\right]Y=\sum_{i=1}^{N}w_{i}y_{i}$

The task is now to choose the $K_{h}(.)$ function. One common choice in the
literature are the radial basis functions, in which each basis function
depends only on the magnitude of the radial distance (often Euclidean) from a
centroid $x_{0}\in X$. The most cited in the literature is the Gaussian
kernel:

$k(x,x_{0})=\exp\left(\frac{-\parallel x-x_{0}\parallel}{2\sigma^{2}}\right)$

####  5.6.3 Deep Neural Networks

Deep Neural Networks (DNN) have been proposed to estimate initial margin by
[12] and [18]. Deep NN is an umbrella term for different related
methodologies, the focus of this paper is on methods proposed on the initial
margin literature, in particular the feed-forward neural network as
implemented in [18].

A DNN can be thought of as a nonlinear generalization of a linear model, where
the parameters (called weights) have to be ”learned” from the data. The idea
is to create a nonlinear function from a stack of linear functions. In a feed-
forward DNN, each layer projects onto the next layer of neurons (computational
nodes). Each linear regression is called a neuron. Each neuron takes outputs
from the previous layer as the input by employing linear transformation with
weights and bias, and then applies an activation function to the result. In
order to find the weights for each layer, the total sum of errors between the
final output and the real observation for each example as in equation 5 is
minimized by performing an orthogonal projection of the dependent variables
onto a non-linear subspace.

The theoretical basis for the approximation of a function by neural networks
is given by the Universal Approximation Theorem (UAT) [19]. This theorem
states that any continuous function in a compact subset of $\mathcal{R}^{n}$
can be approximated by a feed-forward network with a single hidden layer
containing a finite number of neurons. However, in practice we might need more
than one layer to improve the approximation. Recently, in [20], it has been
shown that any continuous function on the $d_{in}$-dimensional unit cube can
be approximated to arbitrary precision by DNN with ReLU activations on hidden
layers with width of exactly $d_{in}+1$. ReLU actiovation function is ReLU(x)
$=\max(0,x)$. The activation function for the output layer is linear.

Following [18] and [20], let’s assume that in the feed-forward neural network
there are an input layer, L hidden layers and an output layer. An example
architecture diagram for this general neural network is provided in figure 4.

Figure 4: Feed-forward neural network architecture

Then the functional form for the approximation of the conditional expectation
function is given by

$m(X_{t},W,b)=\hat{Y_{t}}=\sum_{i=1}^{N_{L}}W_{L}^{i}A_{L}^{i}=\sum_{i=1}^{N_{L}}W_{L}^{i}\cdot\phi_{L}\left(\sum_{j=1}^{N_{L-1}}W_{L-1}^{j}A_{L-1}^{j}+b_{L}^{i}\right)$

where for each hidden layer l, $l=1..L$, $\phi_{l}$ is an activation function,
$A^{i}_{l}$ is an activation output for the $i^{th}$ neuron, $W_{l}$ are
weights and $b_{l}$ is the bias. Here, without loss of generality and for the
ease of comparison, bias for the output layer, $b_{L+1}$ is assumed to be
incorporated into the $W_{L}$. Note, that weights $W_{0}$ are applied to the
input layer, $X_{t}$, and $A_{0}=X_{t}$. The authors in [18] use the classic
ReLU function as the activation function for all hidden layers, $\phi_{l}$. As
in [20], activation function for the output layer, $\phi_{L+1}$, is linear.

Except for the input layer, each node $i$ in the layer $l$ is a neuron
(function) that performs a simple computation:
$A_{l}^{i}=\phi_{l}(\sum_{j}W_{l-1}^{j}A_{l-1}^{j}+b_{l}^{i})$, where
$A_{l}^{i}$ is the output and $A_{l-1}^{j}$ is the $j^{th}$ input.

####  5.6.4 Summary

This section summarises theoretical formulations of the three approaches
considered in Sections 5.6.1-5.6.3 and provides advantages and disadvantages
from the practitioners’ point of view in the table below.

Method |  $m(X_{t},\hat{\beta})$ |  Pros and Cons
---|---|---
Linear Maps |  $\sum_{i=0}^{N}\hat{\beta_{i}}\cdot\phi_{i}(X_{t})$ $\hat{\beta}\in R^{N}$ $\phi_{i}$ \- linear basis functions N - degree of polynomial |  + Same few calibrated parameters are used across all simulation scenarios and time steps. + Computationally cheap compared to the two other methods. - Large oscillations in the calibrated parameters can be observed due to daily change in IM value at $t_{0}$. - In a distributed grid setup small simulation run is required to pre-compute $\hat{\beta_{i}}$ prior to the main MC simulation.
Kernel Regressions (NW) |  $\sum_{i=0}^{N}\hat{\beta_{i}}\cdot\phi_{i}(X_{t})$ $\hat{\beta_{i}}=\frac{K_{h}(x_{0},x_{i})}{\sum_{j=1}^{N}K_{h}(x_{0},x_{j})}$ $\phi_{i}=y_{i}$ N - number of neighbourhood pairs ($x_{i},y_{i}$) $K_{h}$ \- Kernel function |  + Computationally cheap + Better empirical fit to data than linear regression. + Ensures functional smoothness. - Assumes parametric functional form. - It could be challenging to obtain sensitivities to model parameters and inputs.
Feed-Forward Neural Networks |  $\sum_{i=0}^{N}\hat{\beta_{i}}\cdot\phi_{i}(X_{t})$ $\hat{\beta_{i}}=W^{i}_{L}$, weight applied to the output of the neuron i in the final hidden layer $\phi_{i}(X_{t})=A_{L}^{i}$ \- activation of the neuron i in the final hidden layer N - number of neurons in the final hidden layer (plus 1 if incorporating bias) |  + Once trained, predicted IM can be calculated extremely fast because it only involves small scale matrix multiplication and is done at the portfolio level. + $\hat{\beta_{i}}$ only need to be updated every quarter compared to daily in linear maps method. - Training requires portfolio sensitivities for each MC path and time-step per SIMM model requirements. - Data generation for training is computationally expensive and needs to be done offline.

## References

  * [1] Anfuso, Fabrizio and Aziz, Daniel and Giltinan, Paul and Loukopoulos, Klearchos. A Sound Modelling and Backtesting Framework for Forecasting Initial Margin Requirements. (January 11, 2017). Available at SSRN: https://ssrn.com/abstract=2716279
  * [2] Andersen, L and Pykhtin, M and Sokol, A. _Credit Exposure in the Presence of Initial Margin_. (July 22, 2016). Available at SSRN: https://ssrn.com/abstract=2806156
  * [3] Andersen, Leif and Pykhtin M editors. _Margin in Derivaties Trading_.(2018). Risk Books.
  * [4] Andersen, Leif B.G. and Dickinson, Andrew Samuel. _Funding and Credit Risk with Locally Elliptical Portfolio Processes: An Application to CCPs_. (April 11, 2018). Available at SSRN: https://ssrn.com/abstract=316115
  * [5] Basel Committee on Banking Supervision and International Organization of Securities Commissions (2019). Margin Requirements for Non-Centrally Cleared Derivatives. D475 July 2019.
  * [6] Green, Andrew David and Kenyon, Chris. MVA: Initial Margin Valuation Adjustment by Replication and Regression.(January 12, 2015) Available at SSRN: https://ssrn.com/abstract=2432281.
  * [7] Garcia Trillos, Camilo and Henrard, Marc and Macrina, Andrea. _Estimation of Future Initial Margins in a Multi-Curve Interest Rate Framework_. (February 3, 2016). Available at SSRN: https://ssrn.com/abstract=2682727
  * [8] Caspers, Peter and Giltinan, Paul and Lichters, Roland and Nowaczyk, Nikolai. _Forecasting Initial Margin Requirements - A Model Evaluation_. (February 3, 2017). Available at SSRN: https://ssrn.com/abstract=2911167
  * [9] Carrier, J. _Valuation of early-exercise price of options using simulations and nonparametric regression_. Insurance: Mathematics and Economics 19, 19-30 (1996).
  * [10] Chan, Justin and Zhu, Shengyao and Tourtzevitch, Boris. _Practical Approximation Approaches to Forecasting and Backtesting Initial Margin Requirements_. (November 29, 2017). Available at SSRN: https://ssrn.com/abstract=3079782
  * [11] Dahlgren, Martin. _Forward Valuation of Initial Margin in Exposure and Funding Calculations_.(2018) Margin in Derivatives Trading, ch 9. Risk Books.
  * [12] Hernandez, Andres. _Estimating Future Initial Margin with Machine Learning_.(March, 2017). PWC.
  * [13] ISDA SIMM – Methodology, version 2.1, Effective Date: December 2018.
  * [14] Kuhn, Steffen. _Kernel Regression by Mode Calculation of the Conditional Probability Distribution_ Available at: https://arxiv.org/pdf/0811.3499.pdf
  * [15] Luenberg, David _Optimization by Vector Space Methods_.(1969). Wiley.
  * [16] Longstaff, Francis A. and Schwartz, Eduardo S. _Valuing American Options by Simulation: A Simple Least-Squares Approach_.(October 1998). Available at SSRN: https://ssrn.com/abstract=137399
  * [17] Naradaya EA _On Estimating Regression_. Theory Probab. Appl., 9(1), 141–142.
  * [18] Ma, Xun and Spinner, Sogee and Venditti, Alex and Li, Zhao and Tang, Strong. _Initial Margin Simulation with Deep Learning_.(March 21, 2019). Available at SSRN: https://ssrn.com/abstract=3357626
  * [19] Sodhi, Anurag. _American Put Option pricing using Least squares Monte Carlo method under Bakshi, Cao and Chen Model Framework (1997) and comparison to alternative regression techniques in Monte Carlo_.(August, 2018). Available at https://arxiv.org/abs/1808.02791
  * [20] Hanin, Boris and Selke, Mark. _Approximating Continuous Functions by ReLU Nets of Minimal Width_.(2018). Available at arXiv preprint arXiv:1710.11278
  * [21] Albanese, Claudio and Gimonet, Guilloume and Pietronero, Giacomo. Coherent global market simulations and securitization measures for counterparty credit risk. (2011). Quantitative Finance, 11, 1-20.
  * [22] Albanese, Claudio and Caenazzo, Simone and Crepey, Stephane. Credit, funding, margin, and capital valuation adjustments for bilateral portfolios. (2017). Probability, Uncertainty and Quantitative Risk, 2(7).