# A category theory framework for Bayesian learning

Kotaro Kamiya & John Welliaveetil
SyntheticGestalt Ltd
{k.kamiya<EMAIL_ADDRESS>Authors have equal
contribution.

###### Abstract

Inspired by the foundational works in [7] and [3], we introduce a categorical
framework to formalize Bayesian inference and learning. The two key ideas at
play here are the notions of Bayesian inversions and the functor GL as
constructed in [3, §2.1]. We find that Bayesian learning is the simplest case
of the learning paradigm described in [3]. We then obtain categorical
formulations of batch and sequential Bayes updates while also verifying that
the two coincide in a specific example.

###### Contents

  1. 1 Introduction
  2. 2 Preliminaries
    1. 2.1 Bayesian Inference
    2. 2.2 Bayesian inversions and $\mathrm{PS}$
      1. 2.2.1 The category $\mathrm{PS}$
    3. 2.3 The Para construction
  3. 3 Bayes Learn
    1. 3.1  Inducing the $\mathcal{M}$-actegory structure
    2. 3.2 Bayes Learn
      1. 3.2.1 The functor $\mathrm{Stat}$
      2. 3.2.2 The Grothendieck Lens
      3. 3.2.3  The functor _R_
      4. 3.2.4 The functor _BayesLearn_
    3. 3.3 Bayes Learning algorithm
  4. 4 Bayes updates
    1. 4.1 Sequential updates
    2. 4.2 Batch updates

##  1 Introduction

A standard problem in Machine learning is to understand the relationship
between two variables or random vectors. Suppose $\mathbf{x}$ is a random
vector and $y$ is a random variable dependent on $\mathbf{x}$. The additive
model assumes that there exists a function $f$ such that

$y=f(\mathbf{x})+\epsilon$

where $\epsilon$ is a random variable with mean $0$. Our goal then reduces to
estimating the function $f$. To this end, we consider a parametrized family of
functions $\\{f_{\theta}\\}_{\theta\in P}$ where $P$ is the parameter space
and by means of a learning algorithm and a training data set, we traverse $P$
to find a candidate that we believe will provide a reasonable approximation to
the true function $f$. Several theoretical results underpin the validity of
this approach - for instance the Universal Approximation Theorem.

A formulation of the above framework within Category theory was achieved in
the foundational work of Fong, Spivak and Tuyeras [7]. In this paper, the
authors introduce the category $\mathbf{Learn}$ whose objects are sets and a
morphism from $A$ to $B$ in $\mathbf{Learn}$ consists of the following data. A
parameter set $M$, an implementation $I\colon M\times A\to B$, an update
function $u\colon M\times A\times B\to M$ and a request function $r\colon
M\times A\times B\to A$. In related work, the authors of [3], develop a more
general theory by introducing the notion of a Cartesian Reverse Differential
category (cf. [3, §2.2]). Associated to a CRDC - $\mathcal{C}$, they construct
the analog of the category $\mathbf{Learn}$ as a composite of the
$\mathbf{Para}$ and $\mathbf{Lens}$ constructions (cf. [3, §2.1, 2.3]). In
this setting, the learning algorithm will be a functor of type
$\mathbf{Para}(\mathcal{C})\to\mathbf{Para}(\mathbf{Lens}(\mathcal{C}))$. Our
goal in this paper is to employ both these approaches to develop a categorical
framework to discuss Bayesian learning.

While the approach outlined above to model the relationship between two random
variables is effective, in real world situations, it is often the case that
there is no such function $f$. Given the noisy nature of the data, it is more
reasonable to model the conditional probability $p(y|x)$ i.e. to ask the
question - ’Given $x$, what is the probability that we have $y$ ?’.

As before, we model the conditional probability using a parametrized family of
distributions - $p(y|x;\theta)$. However in contrast to our approach from
before, we assume that $\theta$ is a random variable and strive to obtain a
distribution over $\theta$ that agrees with the given training data. More
precisely, we choose a prior distribution $q(\theta)$ on $\theta$ and update
this prior to obtain the posterior distribution on $\theta$. As opposed to
gradient based approaches to learning, Bayesian machine learning updates the
prior distribution on $\theta$ by exploiting Bayes Theorem. The posterior
distribution is defined upto normalizing constant by the formula

$p(\theta|y,x)\propto p(y|x,\theta)q(\theta|x).$

There is considerable flexibility in this approach in that we can use the
posterior to obtain point estimates of the parameter $\theta$ via the MAP
estimate and perform inference by integrating over the entire distribution. We
continue this discussion in greater detail in §2.1 where we illustrate the
essential features of Bayesian learning in the classical context using a
simple example.

Our goal in this paper is to introduce a framework within category theory that
allows us to formalize this set up. We draw on the theory already developed in
[8], [9] and [2]. While we proceed as in [3] by associating to a parametrized
function a morphism in a generalized lens category which allows for the
backward transmission of information, an important observation is that the
Bayes Learning framework simplifies the situation drastically. This is
essentially due to the fact that with the correct setup, Bayesian inversion is
a well defined dagger functor and the functor
$\mathcal{C}^{\mathrm{op}}\to\mathrm{Cat}$ responsible for defining the
generalized lens breaks down. We interpret this to mean that Bayes Learning is
the simplest form of learning that adheres to the framework discussed in [3].
See also Remark 3.10.

The work of Cho and Jacobs in [2] and Fritz in [8] introduces the notion of
Markov categories as a suitable framework within which one can discuss ideas
from Probability theory such as Bayesian inversion, disintegration,
jointification, conditionalization… Hence, let $\mathcal{C}$ be a Markov
category. We use the categories FinStoch and Stoch (cf. Example 2.3) as
guiding examples for the theory we develop. Roughly speaking, the category
FinStoch has finite sets as objects and Markov kernels as morphisms between
them while the objects of Stoch are measurable spaces and morphisms $f\colon
X\to Y$ imply for every $x\in X$, a probability distribution $p(\cdot|x)$ on
$Y$. Central to this paper is the notion of Bayesian inversion which in the
context of FinStoch with reference to a morphism $p\colon X\to Y$ corresponds
to calculating the conditional $p(X|y)$ for $y\in Y$ using Bayes theorem to
obtain a morphism $p^{\dagger}\colon Y\to X$. Note that the inversion which is
a map $Y\to X$ is not necessarily unique and in the case of FinStoch does not
even always exist. To get around this issue, firstly we restrict our attention
to those Markov categories $\mathcal{C}$ which always admit Bayesian
inversion. Secondly, we make use of the symmetric monoidal category
$\mathrm{ProbStoch}(\mathcal{C})$ which was introduced first by Cho and Jacobs
in [2, §5] as the category of couplings or equivalently the category whose
objects coincide with those of the slice category $I\downarrow\mathcal{C}$ and
morphisms are obtained by quotienting with respect to the relation of _almost
sure equality_. Henceforth, we write $\mathrm{PS}(\mathcal{C})$ in place of
$\mathrm{ProbStoch}(\mathcal{C})$ for ease of notation. As a consequence of
our assumption on $\mathcal{C}$, Bayesian inversion is a well defined dagger
functor on $\mathrm{PS}(\mathcal{C})$. A more detailed explanation can be
found in §2.2.

We can interpret the set-up in §2.1, as a parametrized function in a suitable
Kleisli category. In general, we assume that the category $\mathcal{C}$ is
equipped with the structure of an $\mathcal{M}$-actegory where $\mathcal{M}$
is symmetric monoidal. We would like to make use of the
$\mathbf{Para}_{\mathcal{M}}$ construction from [1] which generalizes [7]. For
reasons explained above, we work with the category $\mathrm{PS}(\mathcal{C})$
and in Lemma 3.3, we show that $\mathrm{PS}(\mathcal{C})$ has the structure of
an $\mathrm{PS}(\mathcal{M})$-actegory provided we impose certain technical
assumptions on $\mathcal{M}$ and its action on $\mathcal{C}$. We simplify
notation henceforth and write $\mathrm{PS}$ in place of $\mathrm{PS}$.

Our definition of the functor BayesLearn in Section §3 is inspired in large
part by [3]. The key to the construction of a gradient based learner in [3] is
a functor

$\mathrm{GL}\colon\mathbf{Para}(\mathcal{E})\to\mathbf{Para}(\mathrm{Lens}(\mathcal{E}))$

where we require that $\mathcal{E}$ is a Cartesian reverse differential
category. To fully describe the learning mechanism, the authors introduce
update and error endofunctors on $\mathbf{Para}(\mathcal{E})$. These three
pieces when employed in unison and for appropriate choices of the update and
displacement maps, can describe a wide range of optimization algorithms ([3,
§3.4]).

When adapting the framework of [3] to describe Bayes learning, we see
immediately that we must work with a more general notion of Lens as in [9]. To
this end, we proceed as in [9, Definition 3.1] by introducing a functor

$\mathrm{Stat}\colon\mathrm{PS}(\mathcal{C})^{\mathrm{op}}\to\mathrm{Cat}$

with which we define the associated Grothendieck lens
$\mathrm{Lens}_{\mathrm{Stat}}$. As Bayesian inversion is a well defined
dagger functor, we deduce the existence of a well defined functor

$R\colon\mathrm{PS}(\mathcal{C})\to\mathrm{Lens}_{\mathrm{Stat}}.$

It follows that we have a functor

$\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(R)\colon\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\mathrm{PS}(\mathcal{C}))\to\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\mathrm{Lens}_{\mathrm{Stat}}).$

We refer to the functor $\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(R)$ as the
BayesLearn functor. It captures the essential features of the Bayes learning
algorithm. Indeed, given a parametrized morphism $X\to Y$ with a state on $X$,
after choosing a prior state on the parameter object $P$, we obtain a morphism
in $\mathrm{PS}(\mathcal{C})$ i.e. a morphism

$f\colon(M,\pi_{M})\odot(X,\pi_{X})\to(Y,\pi_{Y}).$

By construction, the induced morphism

$\mathrm{BayesLearn}(f)\colon((M\odot X,\pi_{M}\odot\pi_{X}),(M\odot
X,\pi_{M}\odot\pi_{X}))\to((Y,\pi_{Y}),(Y,\pi_{Y}))$

is given by a pair of morphisms

$f\colon(M\odot X,\pi_{M}\odot\pi_{X})\to(Y,\pi_{Y})$

in the forward direction and

$f^{{\dagger}}\colon(Y,\pi_{Y})\to(M\odot X,\pi_{M}\odot\pi_{X})$

in the backwards direction by Bayesian inversion. This provides a rough
description of the Bayesian learning procedure which we elaborate on in §3 and
§3.3 where we also give a formulation for the Bayes predictive density.

The classical formulation of Bayesian learning enables one to update a given
prior distribution on a parameter space using a training data set and Bayesian
inversion. In §4, we provide a category theoretic formulation of this
phenomenon. Central to the discussion will be the notion of a training set. To
capture the notion of a single training instance, we restrict our attention to
those Markov categories which are of the form $\mathrm{Kl}(\mathcal{P})$ where
$\mathcal{P}\colon\mathcal{D}\to\mathcal{D}$ is a symmetric monoidal monad on
a symmetric monoidal category $\mathcal{D}$. In this context, we introduce the
notion of an elementary point of an object $X$ in $\mathcal{C}$ (cf.
Definition 4.2). In the case of FinStoch or Stoch, these correspond to states
on $X$ which concentrate at a point $x\in X$. We show that there are two ways
one can update the prior. As before, let us suppose we have a model i.e. a
morphism $f\colon M\odot X\to Y$ in $\mathcal{C}$. In addition we are given a
prior distribution $\pi_{M,0}$ on $M$ and a state $\pi_{X}\colon I\to X$. Via
Bayesian inversion with respect to $\pi_{M,0}\odot\pi_{X}$ and after
conditionalizing, we get a channel $f^{{\dagger}}_{\mathrm{joint},0}\colon
X\otimes Y\to M$. The precise details can be found in §3. Let

$\mathbf{T}:=[(x_{1}\otimes y_{1}),\ldots,(x_{n}\otimes y_{n})]$

be a list of _elementary points_ (cf. Definition 4.2) belonging to $X\otimes
Y$. To obtain the posterior, we can proceed sequentially i.e. we update the
prior $\pi_{M,0}$ by the composition
$I\xrightarrow{\delta_{x_{1}}\otimes\delta_{y_{1}}}X\otimes Y\to M$ to get a
state $\pi_{M,1}$ on $M$. Note that $x_{1}$ corresponds to a morphism
$I_{\mathcal{D}}\to X$ and $\delta_{x_{1}}$ is the image of this map in
$\mathcal{C}$. We then implement the inversion procedure with respect to
$\pi_{M,1}$ to get an updated channel $X\otimes Y\to M$ and continue as
before. Repeating the above step to run through the training data set will end
with a state $\pi_{M,n}$ on $M$ which in the case of FinStoch defines a
distribution on $M$ which we call the posterior.

In a similar fashion, we can also define batch updates associated to the
training set $\mathbf{T}$. To do so, we work with the space $Z:=X\otimes Y$
and set $Z_{n}:=\otimes^{n}Z$. A channel $f\colon M\to Z$ naturally gives us a
channel $M\to Z_{n}$ by the composition

$M\xrightarrow{\mathbf{copy}^{n}_{M}}\otimes^{n}M\xrightarrow{\otimes^{n}f}Z_{n}.$

Here we abuse notation and write $\mathbf{copy}^{n}_{M}$ for the composition

$M\xrightarrow{\mathbf{copy}_{M}}M\otimes
M\xrightarrow{\mathrm{id}\otimes\mathbf{copy}_{M}}M\otimes M\otimes
M\ldots\xrightarrow{\mathrm{id}\otimes\ldots\otimes\mathbf{copy}_{M}}\otimes^{n}M.$

Note that the map $M\to Z$ is obtained naturally from the model $f\colon
M\odot X\to Y$ via the composition

$M\odot I\xrightarrow{\mathrm{id}\odot\pi_{X}}M\otimes
X\xrightarrow{f\otimes\mathrm{id}}Y\otimes X\simeq X\otimes Y.$

The Bayesian inversion with respect to $\pi_{M,0}$ implies a channel $Z_{n}\to
M$. The composition

$I\xrightarrow{(\delta_{x_{1}}\otimes\delta_{y_{1}})\otimes\ldots\otimes(\delta_{x_{n}}\otimes\delta_{y_{n}})}Z_{n}\to
M$

defines a state on $M$ which we refer to as the batch update with respect to
$\mathbf{T}$ and denote $\pi_{M,\mathbf{T}}$.

It is natural in this setting to ask if there is a relationship between the
sequential and batch updates or what hypothesis we must impose on
$\mathcal{C}$ to ensure that they are equal. We leave this question open to
further investigation and show that the two are equal in the case of FinStoch
(cf. Examples 4.5 and 4.4).

String diagrams: In this paper, we orient our string diagrams from top to
bottom and display monoidal product as moving from left to right. All string
diagrams have been made using DisCoPy [4].

##  2 Preliminaries

###  2.1 Bayesian Inference

Our goal in this section is to give a high level overview of the approach
Bayesian learning takes in modelling the relationship between random variables
of interest. We use the lecture [11] as a reference to the material in this
section.

Let us illustrate the approach with an example. Let
$\mathbf{x}=(x_{1},\ldots,x_{n})$ be a random vector and $y$ be a random
variable. We assume that $y$ is related to $\mathbf{x}$ via the equation

$y=f(\mathbf{x})+\epsilon$

where $\epsilon$ is normally distributed with mean $0$ and variance
$\sigma^{2}$ and $f(\mathbf{x}):=\beta^{T}\mathbf{x}$ where
$\beta\in\mathbb{R}^{n}$. For simplicity, we assume that the quantity $\sigma$
is a known constant.

Let us suppose we are given a training set
$\mathbf{T}:=\\{(\mathbf{x}_{1},y_{1}),\ldots,(\mathbf{x}_{N},y_{N})\\}$. One
can divide the Bayesian approach into two parts. As outlined in the
introduction we treat $\beta$ as a random variable and choose a prior
distribution which encodes our pre-conceived beliefs about $\beta$. Then using
the set $\mathbf{T}$, we update the distribution on $\beta$ to get its
posterior distribution by applying Bayes’ rule. The posterior distribution can
then be used to obtain point estimates of $\beta$.

Let us assume an improper prior on $\beta$ i.e. $q(\beta)=1$. Applying Bayes’
rule gives

$p(\beta|\mathbf{T})\propto p(\mathbf{T}|\beta)q(\beta)$

Observe that our assumption on the nature of the underlying data implies that
$p(y|\mathbf{x})$ is normally distributed with mean $\beta^{T}\mathbf{x}$ and
variance $\sigma^{2}$. Assuming that the training data is independent, we get

$p(\mathbf{T}|\beta)\propto\prod_{i}p(y_{i}|\beta,\mathbf{x}_{i})q(\beta).$

We assumed above that the distribution on $\mathbf{x}$ is known and hence in
effect we condition every random variable with respect to $\mathbf{x}$. Hence,

$p(\mathbf{T}|\beta)=\frac{1}{({2\pi})^{\frac{N}{2}}\sigma^{N}}\mathrm{exp}(-\frac{\sum_{i}(y_{i}-\beta^{T}\mathbf{x}_{i})^{2}}{2\sigma^{2}})$

and

$p(\beta|\mathbf{T})=\frac{1}{({2\pi})^{\frac{N}{2}}\sigma^{N}}\mathrm{exp}(-\frac{\sum_{i}(y_{i}-\beta^{T}\mathbf{x}_{i})^{2}}{2\sigma^{2}})$

In this manner, we have updated the prior distribution on $\beta$ in
accordance with the given data. We now sketch how one might perform inference.
Let $\mathbf{x}_{*}$ be a data point that does not necessarily belong to the
training set $\mathbf{T}$. We would like to find
$p(y_{*}|\mathbf{x}_{*},\mathbf{T})$. We outline two ways on how to proceed.

  1. 1.

Let

$\beta_{\mathrm{MAP}}:=\mathrm{argmax}_{\beta}p(\beta|\mathbf{T})$

This is called the _maximum a posteriori_ estimate of the parameter $\beta$.
It represents a single best guess for the parameter given the training data.
We then set

$p(y_{*}|\mathbf{x}_{*},\mathbf{T}):=p(y_{*}|\mathbf{x}_{*},\beta_{\mathrm{MAP}}).$

Observe that in the context of our ongoing example, $\beta_{\mathrm{MAP}}$
coincides with $\beta_{\mathrm{OLS}}$ \- the ordinary least squares estimate.

  2. 2.

While the MAP estimate of $\beta$ represents a suitable guess for the
parameter, when performing inference we can do better by leveraging our
knowledge of the entire posterior distribution. The true Bayesian way is to
integrate over the posterior distribution $\beta$ i.e.

$p(y_{*}|\mathbf{x}_{*},\mathbf{T}):=\int
p(y_{*}|\mathbf{x}_{*},\beta)p(\beta|\mathbf{T})d\beta.$

In practice, owing to the intractability of the integral above due to the fact
that the posterior distribution is not likely analytic a possible solution is
to proceed by employing a suitable approximation strategy. See for instance
[5] or [6].

######  Remark 2.1.

_Observe that the posterior distribution on $\beta$ with regards to the
training set $\mathbf{T}$ can also be obtained by sequential updates. This is
due to the following phenomenon. Suppose $\mathbf{T}$ can be split into two
training sets $\mathbf{T}_{1}$ and $\mathbf{T}_{2}$ where
$\mathbf{T}_{1}:=\\{(\mathbf{x}_{1},y_{1}),\ldots,(\mathbf{x}_{r},y_{r})\\}$
and
$\mathbf{T}_{2}:=\\{(\mathbf{x}_{r+1},y_{r+1}),\ldots,(\mathbf{x}_{N},y_{N})\\}$.
Let $q_{1}$ be the posterior distribution $p(\beta|\mathbf{T}_{1})$ on $\beta$
obtained by updating the prior $q$. Likewise, let $q_{2}$ be the posterior
distribution on $\beta$ obtained by updating the prior $q_{1}$ using the
dataset $\mathbf{T}_{2}$. A simple calculation shows that the distribution
$q_{2}$ coincides with the posterior obtained by updating $q$ using the entire
dataset $\mathbf{T}$. In §4, we formulate a categorical version of the above
observation._

###  2.2 Bayesian inversions and $\mathrm{PS}$

The theory of Markov categories provides a categorical framework within which
we can discuss and formalize notions from Probability theory. We use the
notation and conventions introduced in [8]. Note that this notion of Markov
categories coincides with that of affine CD-categories as introduced by Cho
and Jacobs in [2].

######  Definition 2.2.

_A_ Markov category _is a semicartesian symmetric monoidal category_
$(\mathcal{C},I,\otimes)$ _in which every object X is equipped with the
structure of a commutative internal comonoid. We denote the comultiplication
and counit maps by_

$\mathbf{copy}_{X}\colon X\to X\otimes X$

_and_

$\mathbf{del}_{X}\colon X\to I$

_respectively and require that they satisfy certain natural conditions (cf.[8,
Definition 2.1])._

The definition above suggest that one can think of a morphism $\pi_{X}\colon
I\to X$ in a Markov category $\mathcal{C}$ as a probability distribution on
$X$. We refer to $\pi_{X}$ as defining a _state_ on $X$. Likewise, a morphism
$f\colon X\to Y$ in $\mathcal{C}$ will be called a _channel_.

For the remainder of this section we fix a Markov category $\mathcal{C}$.

######  Example 2.3.

_We introduce two examples which we will continue to make reference to
throughout the course of the paper._

  1. 1.

_We define the category $\mathrm{FinStoch}$ to be the category of finite sets
with channels. A morphism $f\colon X\to Y$ in $\mathrm{FinStoch}$ is given by
a morphism of sets $f\colon X\to\mathrm{Dist}(Y)$ where $\mathrm{Dist}(Y)$ is
the set of probability distributions on $Y$ i.e._

$\mathrm{Dist}(Y)=\Big{\\{}\omega\colon Y\to[0,1]|\sum_{y\in
Y}\omega(y)=1\Big{\\}}.$

_Given a set $X$, we define $\mathbf{copy}_{X}$ to be the map that sends an
element $x\in X$ to the distribution on $X\times X$ which takes the value $1$
at $(x,x)$ and $0$ everywhere else. The terminal object $I$ is the set with a
single element i.e. $I=\\{*\\}$ and as a consequence we can identify
$\mathrm{Dist}(I)$ with $I$. We define $\mathbf{del}_{X}$ to be the map that
sends every element $x\in X$ to $*$. Lastly, given morphisms $f\colon X\to Y$
and $g\colon Y\to Z$, we define the composition $f;g\colon X\to Z$ to be such
that for every $x\in X$,_

$(f;g)(x)(z):=\sum_{y\in Y}g(y)(z)f(x)(y).$

_Observe that $\mathrm{Dist}$ defines an endofunctor on the category of Finite
sets. One can check without difficulty that it is in fact a commutative monad.
It follows that $\mathrm{FinStoch}$ coincides with the Kleisli category
$\mathrm{Kl}(\mathrm{Dist})$._

  2. 2.

_To deal with distributions whose supports are not necessarily finite, we
introduce $\mathrm{Stoch}$ whose objects are measurable spaces i.e. tuples of
the form $(X,\Sigma_{X})$ where $X$ is a set and $\Sigma_{X}$ is a well
defined $\sigma$-algebra on $X$. Recall that the category of measurable spaces
is endowed with a natural symmetric monoidal structure given by_

$(X,\Sigma_{X})\otimes(Y,\Sigma_{Y}):=(X\times Y,\Sigma_{X}\otimes\Sigma_{Y})$

_where $\Sigma_{X}\otimes\Sigma_{Y}$ corresponds to the $\sigma$-algebra
generated by subsets of the form $U\times V$ where $U\in\Sigma_{X}$ and
$V\in\Sigma_{Y}$._

_A morphism $(X,\Sigma_{X})\to(Y,\Sigma_{Y})$ in $\mathrm{Stoch}$ is given by
a map_

$f\colon X\times\Sigma_{Y}\to[0,1]$

_such that for every $S\in\Sigma_{X}$, the map $f(S,\\_)\colon X\to[0,1]$ is
measurable. Furthermore, we ask that for every $x$,
$f(\\_,x)\colon\Sigma_{Y}\to[0,1]$ is a well defined probability measure. One
checks that $\mathrm{Stoch}$ is a symmetric monoidal category whose monoidal
structure is inherited from the category of measurable spaces. The unit object
$I$ is the pair $(\\{*\\},\\{\\{*\\},\emptyset\\})$. _

_As before, we must define copy and delete morphisms as well as specify how to
compose morphisms in Stoch. We do not define $\mathbf{del}$ as this is obvious
from the definition of $I$. Given
$(X,\Sigma_{X})\in\mathrm{Ob}(\mathrm{Stoch})$, we define_

$\mathbf{copy}_{X}\colon(X\times X)\times\Sigma_{X\times X}\to[0,1]$

_as the map that sends a pair $(S,x)$ to $1$ if $(x,x)\in S$ and $0$
otherwise. Lastly, given morphisms $f\colon(X,\Sigma_{X})\to(Y,\Sigma_{Y})$
and $g\colon(Y,\Sigma_{Y})\to(Z,\Sigma_{Z})$, we define $f;g$ to be the map
$X\times\Sigma_{Z}\to[0,1]$ to be given by_

$(S,x)\mapsto\int g(S,y)f(dy,x)$

_where we abuse notation and write $f(x)$ for the measure $\Sigma_{Y}\to[0,1]$
given by $T\mapsto f(T,x)$._

######  Definition 2.4.

_Let $\pi_{X\otimes Y}\colon I\to X\otimes Y$ be a joint state. A_
disintegration _of $\pi_{X\otimes Y}$ will be a pair consisting of a channel
$f\colon X\to Y$ and a state $\pi_{X}\colon I\to X$ such that the following
string diagrams are equal._

$X$$Y$$\pi_{X\otimes Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$ (1)

_If every joint state admits a disintegration then we say that the category
$\mathcal{C}$ admits_ conditional distributions.

_Likewise, given a pair $f\colon X\to Y$ and $\psi\colon I\to X$, we can
easily define a state on $X\times Y$ via the following string diagram._

$X$$Y$$\pi_{X\otimes Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$

_We refer to this as the jointification of $f$ and $\psi$._

We will require a more general version of conditionalization which permits us
to consider joint distributions parametrized by another object.

######  Definition 2.5.

_Let $\mathcal{C}$ be a Markov category. We say that $\mathcal{C}$ has_
conditionals _if for every morphism $s\colon A\to X\otimes Y$, there is
$t\colon X\otimes A\to Y$ such that we have the following equality of string
diagrams._

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$
(2)

######  Remark 2.6.

_In the case of FinStoch, we see that a state $\pi_{X\times Y}\colon I\to
X\times Y$ corresponds to a probability distribution on $X\times Y$.
Furthermore, a disintegration of $\pi_{X\times Y}$ is given by the conditional
distribution associated to the joint distribution as well as the state
$\pi_{X}\colon I\to X$ obtained by marginalizing $y$ in $\pi_{X\times Y}$. We
define the conditional distribution $c\colon X\to Y$ explicitly by setting_

$c(x)(y):=\frac{\pi_{X\times Y}(x,y)}{\pi_{X}(x)}$

_if $\pi_{X}(x)$ is not zero and in the event that $\pi_{X}(x)=0$, we set
$c(x)(\\_)$ to be any distribution on $Y$._

_Another category which admits conditionals is the category BorelStoch which
is a subcategory of the category Stoch whose objects are standard Borel
spaces._

Observe from the explicit calculation above that the disintegration of a joint
distribution is not necessarily unique. This leads us to the following
definition.

######  Definition 2.7.

_Let $\pi_{X}\colon I\to X$ be a state on an object
$X\in\mathrm{Ob}(\mathcal{C})$. Let $f,g\colon X\to Y$ be morphisms in
$\mathcal{C}$. We say that $f$ is almost surely equal to $g$ with respect to
$\pi_{X}$ or $f\sim_{\pi_{X}\mathrm{-a.s}}g$ if the following string diagrams
coincide._

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$$\pi_{X}$$f$=$\pi_{X}$$g$

Observe that if $\pi_{X\times Y}\colon I\to X\times Y$ is a state and
$\pi_{X}\colon I\to X$ is the associated marginal then if $f,g\colon X\to Y$
are channels such that $(f,\pi_{X})$ and $(g,\pi_{X})$ are both
disintegrations with respect to $\pi_{X\times Y}$ then $f$ is $\pi_{X}$-almost
surely equal to $g$.

######  Definition 2.8.

_Let $\pi_{X}\colon I\to X$ be a state on an object
$X\in\mathrm{Ob}(\mathcal{C})$. Let $f\colon X\to Y$ be a channel. The
Bayesian inversion of $f$ with respect to $\pi_{X}$ is a channel
$f^{\dagger}_{\pi_{X}}\colon Y\to X$ such that we have the following equality
of string diagrams. We say that $\mathcal{C}$ admits Bayesian inversions if
for every state $\pi_{X}\colon I\to X$ and $f\colon X\to Y$ we have a Bayesian
inversion $f_{\pi_{X}}^{\dagger}\colon Y\to X$._

Note that we can rephrase the definition above in terms of disintegrations as
follows. Indeed, if $c$ and $\pi_{X}$ are as in the definition then the
Bayesian inversion $c^{\dagger}_{\pi_{X}}$ can be obtained by disintegrating
the joint distribution $\pi_{X\times Y}\colon I\to Y\times X$ obtained by
swapping the integration of the pair $(c,\pi_{X})$.

######  Remark 2.9.

_As for disintegrations, we see that Bayesian inversions are not necessarily
unique. However, if $c_{1},c_{2}$ are Bayesian inversions of a channel
$c\colon X\to Y$ with respect to a state $\pi_{X}\colon I\to X$ then $c_{1}$
is almost surely equal to $c_{2}$._

####  2.2.1 The category $\mathrm{PS}$

A crucial requirement of our set up that allows us to define the BayesLearn
functor similar to the Gradient learn functor from [3] is that Bayesian
inversions must compose strictly. To this end, we must move away from working
with equivalence classes of almost surely equal morphisms with respect to a
given state and instead use the category $\mathrm{PS}$ (cf. [8, Definition
13.8]).

######  Definition 2.10.

_Suppose that $\mathcal{C}$ is causal (cf.[8, Definition 11.31]). Then the
category $\mathrm{ProbStoch}(\mathcal{C})$ is defined as follows. The objects
of $\mathrm{ProbStoch}(\mathcal{C})$ consist of pairs $(X,\pi_{X})$ where
$X\in\mathrm{Ob}(\mathcal{C})$ and $\pi_{X}\colon I\to X$ is a state on $X$. A
morphism in $\mathrm{ProbStoch}(\mathcal{C})$ between objects $(X,\pi_{X})$
and $(Y,\pi_{Y})$ consists of a map $f\colon X\to Y$ in $\mathcal{C}$
satisfying $\pi_{X};f=\pi_{Y}$ modulo $\pi_{X}$-a.s. equality, with
composition inherited from $\mathcal{C}$ i.e._

$\mathrm{ProbStoch(\mathcal{C})}(X,Y):={\\{f\in\mathcal{C}(X,Y)|\pi_{X};f=\pi_{Y}\\}}/{\sim_{\pi_{X}-a.s}}$

As mentioned in the introduction, we write $\mathrm{PS}$ in place of
$\mathrm{ProbStoch}$ for ease of notation.

######  Remark 2.11.

_Note that the categories $\mathrm{Stoch}$ and $\mathrm{FinStoch}$ are both
causal. It is important to observe that if a category admits conditionals then
it is causal. However the converse is not true (cf. [8, 11.34, 11.35]). _

######  Remark 2.12.

_Recall from[8, Proposition 13.9(a)] that if $\mathcal{C}$ is causal then
$\mathrm{PS}(\mathcal{C})$ is symmetric monoidal. In this case, the unit
object is given by the pair $(I,\iota)$ where $\iota\colon I\to I$ is the
identity map in $\mathcal{C}$._

###  2.3 The Para construction

Recall that our goal is to understand conditional distributions between random
variables or morphisms in a Markov category that satisfies certain
constraints. In this setting, we model a conditional distribution $p(y|x)$
using a parametric function $f(x;\theta)$ while our learning algorithm updates
$\theta$ using the given training set. The notion of parametrized function has
a natural formulation in category theory which we call $\mathbf{Para}$ which
was first introduced in [7]. We use the more generalized version of this
construction which can be found in [1]. Observe that the type of the parameter
$\theta$ need not coincide with that of the variable. To ensure that this
observation is preserved in the categorical formulation, we make use of the
notion of actegories.

######  Definition 2.13.

_Let $(\mathcal{M},J,\star)$ be a symmetric monoidal category and let
$\mathcal{C}$ be a category._

  1. 1.

_We say that $\mathcal{C}$ is an_ $\mathcal{M}$-actegory _if we have a strong
monoidal functor $\Phi\colon\mathcal{M}\to\mathrm{End}(\mathcal{C})$ where
$\mathrm{End}(\mathcal{C})$ is the category of endofunctors on $\mathcal{C}$
for which the monoidal product is given by composition._ _Given
$M\in\mathrm{Ob}(\mathcal{M})$ and $X\in\mathcal{C}$, we write $M\odot
X:=\Phi(M)(X)$._

  2. 2.

_We say that $\mathcal{C}$ is a_ symmetric monoidal $\mathcal{M}$-actegory _if
in addition to being an $\mathcal{M}$-actegory $\mathcal{C}$ is endowed with
natural isomorphisms_

$\kappa_{M,X,Y}\colon M\odot(X\otimes Y)\simeq X\otimes(M\odot Y)$

_satisfying coherence laws reminiscent of the laws of a costrong comonad._

######  Remark 2.14.

_Part (2) of Definition 2.13 is from [1, §2.1 Definition 4]. As in this
reference, we point out that if $(\mathcal{C},I,\otimes)$ is a symmetric
monoidal $(\mathcal{M},J,\star)$-actegory then we have natural isomorphisms_

$\alpha_{M,X,Y}\colon M\odot(X\otimes Y)\simeq(M\odot X)\otimes Y$

_and_

$\iota_{M,N,X,Y}\colon(M\star N)\odot(X\otimes Y)\simeq(M\odot
X)\otimes(N\odot Y)$

_which are called the mixed associator and the mixed interchanger
respectively._

######  Definition 2.15.

[1, §Definition 2] _Let $\mathcal{M}$ be a symmetric monoidal category and let
$\mathcal{C}$ be an $\mathcal{M}$-actegory. The bicategory_
$\mathbf{Para}_{\mathcal{M}}(\mathcal{C})$ _is defined as follows._

  * •

$\mathrm{Ob}(\mathbf{Para}_{\mathcal{M}}(\mathcal{C})):=\mathrm{Ob}(\mathcal{C})$.

  * •

_A 1-cell $f\colon X\to Y$ in $\mathbf{Para}_{\mathcal{M}}(\mathcal{C})$
consists of a pair $(P,\phi)$ where $P\in\mathrm{Ob}(\mathcal{M})$ and
$\phi\colon P\odot X\to Y$ is a morphism in $\mathcal{C}$._

  * •

Let $(P,\phi)\in\mathbf{Para}_{\mathcal{M}}(\mathcal{C})(X,Y)$ and
$(Q,\psi)\in\mathbf{Para}_{\mathcal{M}}(\mathcal{C})(Y,Z)$. The composition
$(P,\phi);(Q,\psi)$ is the map in $\mathcal{C}$

$Q\odot(P\odot X)\to Z$

given by

$Q\odot(P\odot X)\xrightarrow{\phi}Q\odot Y\xrightarrow{\psi}Z$

  * •

_Let $(P,\phi),(Q,\psi)\in\mathbf{Para}_{\mathcal{M}}(\mathcal{C})(X,Y)$. A
2-cell $\alpha\colon(P,\phi)\to(Q,\psi)$ is given by a morphism
$\alpha^{\prime}\colon Q\to P$ such that the following diagram commutes._

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$$\pi_{X}$$f$=$\pi_{X}$$g$${Q\odot
X}$${P\odot
X}$${Y}$$\scriptstyle{\alpha^{\prime}\odot\mathrm{id}_{X}}$$\scriptstyle{\psi}$$\scriptstyle{\phi}$

  * •

_The identity and composition in the category
$\mathbf{Para}_{\mathcal{M}}(\mathcal{C})(X,Y)$ are inherited from the
identity and composition in $\mathcal{M}$._

By [1, Proposition 3, §2], $\mathbf{Para}_{\mathcal{M}}(\\_)$ defines a
pseudo-monad on the category $\mathcal{M}-\mathbf{Mod}$ of
$\mathcal{M}$-actegories. In particular, if we have a functor

$F\colon\mathcal{C}\to\mathcal{D}$

we get an associated functor

$\mathbf{Para}_{\mathcal{M}}(F)\colon\mathbf{Para}_{\mathcal{M}}(\mathcal{C})\to\mathbf{Para}_{\mathcal{M}}(\mathcal{D})$

We make use of this fact when we define the BayesLearn functor in §3.

##  3 Bayes Learn

In this section we outline the construction of the functor BayesLearn which
aims to capture the essential features of Bayesian learning. However in order
for us to discuss these results, we require certain preliminary ideas which
allow us to better understand the actegory structure on
$\mathrm{PS}(\mathcal{C})$ where $\mathcal{C}$ is a Markov category.

###  3.1  Inducing the $\mathcal{M}$-actegory structure

While we work with the flexibility provided by $\mathbf{Para}_{\mathcal{M}}$,
we must be careful to ensure that the categories to which we apply
$\mathbf{Para}_{\mathcal{M}}(\\_)$ are endowed with the structure of an
$\mathcal{M}$-actegory. To this end, we must make certain modifications to the
category $\mathcal{M}$ itself as we would firstly like to ensure that
parameter spaces have a well defined prior associated to them.

######  Remark 3.1.

_It follows directly from the definition of an actegory that if $r\colon M\to
N$ in $\mathcal{M}$ and $f\colon X\to Y$ in $\mathcal{C}$ then the following
diagram commutes._

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$$\pi_{X}$$f$=$\pi_{X}$$g$${M\odot
X}$${M\odot Y}$${N\odot X}$${N\odot Y}$$\scriptstyle{M\odot
f}$$\scriptstyle{r\odot X}$$\scriptstyle{r\odot Y}$$\scriptstyle{N\odot f}$

_where the vertical morphisms are due to the natural transformation between
the functors $M\odot\\_$ and $N\odot\\_$ induced by the morphism $M\to N$. We
write $f\odot g$ to denote the composition $M\odot X\to M\odot Y\to N\odot Y$
or equivalently $M\odot X\to N\odot X\to N\odot Y$._

The following technical condition is required in the proof of Lemma 3.3.

######  Definition 3.2.

_Let ( $\mathcal{M},J,\star)$ and $(\mathcal{C},I,\otimes)$ be Markov
categories and let $\mathcal{C}$ be a symmetric monoidal
$\mathcal{M}$-actegory (cf. Definition 2.13). We say that $\mathcal{C}$ is in_
agreement _with $\mathcal{M}$ if_ _for every $P\in\mathcal{M}$ and
$X\in\mathcal{C}$, the following diagram is commutative and natural in both
$P$ and $X$._

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$$\pi_{X}$$f$=$\pi_{X}$$g$${P\odot
X}$${(P\star P)\odot(X\otimes X)}$${P\odot X}$${(P\odot X)\otimes(P\odot
X)}$$\scriptstyle{\mathrm{copy}_{P}\odot\mathrm{copy}_{X}}$$\scriptstyle{\iota_{P,P,X,X}}$$\scriptstyle{\mathrm{copy}_{P\odot
X}}$

_where $\iota_{P,P,X,X}$ is the mixed interchanger as introduced in 2.14._

Recall from 2.1, that a categorical formulation of Bayesian learning must
capture the notion of state on the parameter as well as a means to update it
via Bayesian inversion. It is hence natural under these circumstances to
utilize $\mathrm{PS}(\mathcal{M})$ as the category of parameters. A brief
discussion of the PS construction was made in 2.2.1.

######  Lemma 3.3.

Let $(\mathcal{M},J,\star)$ be a causal Markov category and
$(\mathcal{C},I,\otimes)$ be a Markov category that is in agreement with
$\mathcal{M}$. The category $\mathrm{PS}(\mathcal{C})$ is a
$\mathrm{PS}(\mathcal{M})$-actegory.

###### Proof.

Let $(P,\pi_{P})\in\mathrm{Ob}(\mathrm{PS}(\mathcal{C}))$. We define an
endofunctor

$(P,\pi_{P})\odot\\_\colon\mathrm{PS}(\mathcal{C})\to\mathrm{PS}(\mathcal{C})$

as follows. Firstly, let $(X,\pi_{X})$ be an object in
$\mathrm{PS}(\mathcal{C})$. We set $(P,\pi_{P})\odot(X,\pi_{X})$ to be the
pair

$I\xrightarrow{\sim}J\odot I\xrightarrow{\pi_{P}\odot\pi_{X}}P\odot X$

where the first isomorphism is due to the $\mathcal{M}$-actegory structure on
$\mathcal{C}$. Observe that the composition above coincides with

$\displaystyle I\xrightarrow{\sim}J\odot
I\xrightarrow{\pi_{P}\odot\mathrm{id}}P\odot
I\xrightarrow{\mathrm{id}\odot\pi_{X}}P\odot X.$ (3)

We check that $(P,\pi_{P})\odot\\_$ defines a functor. Suppose
$f\colon(X,\pi_{X})\to(Y,\pi_{Y})$ in $\mathrm{PS}(\mathcal{C})$. The morphism
$\pi_{Y}\colon I\to Y$ coincides with the composition
$I\xrightarrow{\pi_{X}}X\xrightarrow{\tilde{f}}Y$ where $\tilde{f}$ is a
representative of the equivalence class of $f$ in $\mathcal{C}$. Applying
$P\odot\\_$ gives

$P\odot I\xrightarrow{\mathrm{id}\odot\pi_{X}}P\odot
X\xrightarrow{\mathrm{id}\otimes\tilde{f}}P\odot Y.$

Hence we get a sequence of morphisms

$\displaystyle I\xrightarrow{\sim}J\odot
I\xrightarrow{\pi_{P}\odot\mathrm{id}}P\odot
I\xrightarrow{\mathrm{id}\odot\pi_{X}}P\odot
X\xrightarrow{\mathrm{id}\odot\tilde{f}}P\odot Y.$ (4)

Using the observation in Equation (3) and the fact that
$\pi_{X};\tilde{f}=\pi_{Y}$, we deduce that Equation (4) coincides with
$(P,\pi_{P})\odot(Y,\pi_{Y})$. We have thus shown that we have a map

$(P,\pi_{P})\odot\tilde{f}\colon(P,\pi_{P})\odot(X,\pi_{X})\to(P,\pi_{P})\odot(Y,\pi_{Y})$

in $\mathrm{PS}(\mathcal{C})$. Note that we must show that the map
$(P,\pi_{P})\odot(X,\pi_{X})\to(P,\pi_{P})\odot(Y,\pi_{Y})$ we have defined
above is independent of our choice of $\tilde{f}$. Hence we verify that if
$h,g\colon X\to Y$ are morphisms in $\mathcal{C}$ such that
$h\sim_{\pi_{X}\mathrm{-a.s}}g$ then $(P,\pi_{P})\odot
h\sim_{\pi_{P}\odot\pi_{X}\mathrm{-a.s}}(P,\pi_{P})\odot g$. Since
$h\sim_{\pi_{X}\mathrm{-a.s}}g$, we get that the compositions

$A:=I\to X\to X\otimes X\xrightarrow{h\otimes\mathrm{id}}Y\otimes X$

and

$B:=I\to X\to X\otimes X\xrightarrow{g\otimes\mathrm{id}}Y\otimes X$

coincide. Consider the sequence

$C:=J\xrightarrow{\pi_{P}}P\xrightarrow{\mathrm{copy}_{P}}P\otimes
P\xrightarrow{\mathrm{id}}P\otimes P$

in $\mathcal{M}$. Since $A=B$, we see that $C\odot A=C\odot B$. However, this
implies that

$I\to P\odot X\xrightarrow{\mathrm{copy}_{P}\odot\mathrm{copy}_{X}}(P\star
P)\odot(X\otimes X)\xrightarrow{\mathrm{id}\odot(h\otimes\mathrm{id})}(P\star
P)\odot(Y\otimes X)$

coincides with

$I\to P\odot X\xrightarrow{\mathrm{copy}_{P}\odot\mathrm{copy}_{X}}(P\star
P)\odot(X\otimes X)\xrightarrow{\mathrm{id}\odot(g\otimes\mathrm{id})}(P\star
P)\odot(Y\otimes X).$

It follows that

$I\to P\odot X\xrightarrow{\mathrm{copy}_{P\odot X}}(P\odot X)\otimes(P\odot
X)\xrightarrow{(P\odot h)\otimes\mathrm{id}}(P\odot Y)\otimes(P\odot X)$

coincides with

$I\to P\odot X\xrightarrow{\mathrm{copy}_{P\odot X}}(P\odot X)\otimes(P\odot
X)\xrightarrow{(P\odot g)\otimes\mathrm{id}}(P\odot Y)\otimes(P\odot X).$

This is a consequence of the naturality of the agreement (cf. Definition 3.2)
and mixed interchanger morphisms (cf. Remark 2.14). This verifies that
$(P,\pi_{P})\odot f\sim_{\pi_{P}\odot\pi_{X}\mathrm{-a.s}}(P,\pi_{P})\odot g$.
One checks without difficulty that $(P,\pi_{P})\odot\\_$ respects composition
of morphisms, identities and that

$(P,\pi_{P})\odot\\_\colon\mathrm{PS}(\mathcal{C})\to\mathrm{PS}(\mathcal{C}).$

is indeed a well defined functor. Furthermore, using the notation from Remark
2.12, $(J,\iota)\odot\\_$ coincides with the identity on
$\mathrm{PS}(\mathcal{C})$ and we have a natural isomorphism
$\mathrm{id}_{\mathrm{PS}(\mathcal{C})}\xrightarrow{\sim}(J,\iota)\odot\\_$
which is induced by the isomorphism
$\mathrm{id}_{\mathcal{C}}\xrightarrow{\sim}J\odot\\_.$

Lastly, let $(P,\pi_{P}),(Q,\pi_{Q})\in\mathrm{PS}(\mathcal{M})$ and
$(X,\pi_{X})\in\mathrm{PS}(\mathcal{C})$. Since we have a strong monoidal
functor $P\odot(Q\odot\\_)\xrightarrow{\sim}(P\star Q)\odot\\_$, we deduce
that the following diagram is commutative.

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$$\pi_{X}$$f$=$\pi_{X}$$g$${I}$${J\odot
I}$${J\odot(J\odot I)}$${P\odot(Q\odot X)}$${I}$${J\odot I}$${(J\star J)\odot
I}$${(P\star Q)\odot X.}$$\scriptstyle{\sim}$

$\scriptstyle\sim$

$\scriptstyle{\pi_{P}\odot(\pi_{Q}\odot\pi_{X})}$

$\scriptstyle\sim$

$\scriptstyle{\sim}$$\scriptstyle{(\pi_{P}\star\pi_{Q})\odot\pi_{X}}$

We deduce from this and similar such arguments that we have an isomorphism

$(P,\pi_{P})\odot((Q,\pi_{Q})\odot\\_)\xrightarrow{\sim}((P,\pi_{P})\star(Q,\pi_{Q}))\odot\\_$

making the functor

$\mathrm{PS}(\mathcal{M})\to\mathrm{End}(\mathrm{PS}(\mathcal{C}))$

given by

$(P,\pi_{P})\mapsto(P,\pi_{P})\odot\\_$

a strong monoidal functor. This concludes the proof. ∎

######  Lemma 3.4.

Let $(\mathcal{A},I,\otimes)$ and $(\mathcal{M},J,\star)$ be symmetric
monoidal categories such that $\mathcal{A}$ is an $\mathcal{M}$-actegory. Let

$S\colon\mathcal{A}^{\mathrm{op}}\to\mathcal{M}-\mathrm{Mod}$

be a functor which takes values in the category of $\mathcal{M}$-actegories.
We suppose that $S$ satisfies the following property.

  1. 1.

There exists an $\mathcal{M}$-actegory $\mathcal{B}$ such that for every
$X\in\mathrm{Ob}(\mathcal{A})$, $S(X)=\mathcal{B}$.

  2. 2.

For every $M\in\mathrm{Ob}(\mathcal{M})$, $X,Y\in\mathrm{Ob}(\mathcal{A})$,
$f\in\mathcal{A}(X,Y)$, $A\in\mathrm{Ob}(S(X))$ and $B\in\mathrm{Ob}(S(Y))$,
we have that

$M\odot S(f)(B)=S(M\odot f)(M\odot B)$

The Grothendieck lens $\mathrm{Lens}_{S}$ is then an $\mathcal{M}$-actegory.

Note that in the statement of the above lemma, condition (2) only makes sense
if we have condition (1).

###### Proof.

Recall that the objects of $\mathrm{Lens}_{S}$ are tuples of the form $(X,A)$
where $X$ is an object in $\mathcal{A}$ and $A\in\mathrm{Ob}(S(X))$. Given
$M\in\mathrm{Ob}(\mathcal{M})$, we define

$M\odot(X,A):=(M\odot X,M\odot A).$

We show that $M\odot\\_$ defines an endofunctor on $\mathrm{Lens}_{S}$. Let
$\mathbf{f}\colon(X,A)\to(Y,B)$ be given by a pair of morphisms $f\colon X\to
Y$ and a map $f^{*}\colon S(f)(B)\to A$. We define

$M\odot\mathbf{f}\colon M\odot(X,A)\to M\odot(Y,B)$

in $\mathrm{Lens}_{S}$ as follows. Since $\mathcal{M}$ is a
$\mathcal{M}$-actegory, we have a morphism

$M\odot{f}\colon M\odot X\to M\odot Y$

and a morphism

$M\odot{f^{*}}\colon M\odot S(f)(B)\to M\odot A.$

We apply the assumption on $S$ to get

$M\odot{f^{*}}\colon S(M\odot f)(M\odot B)\to M\odot A.$

The pair $(M\odot{f},M\odot{f^{*}})$ defines a morphism $M\odot(X,A)\to
M\odot(Y,B)$ in $\mathrm{Lens}_{S}$. Clearly, $M\odot\\_$ preserves
identities. One checks without difficulty that $M\odot\\_$ respects
compositions as well. We have thus shown that $M\odot\\_$ defines an
endofunctor on $\mathrm{Lens}_{S}$.

It remains to show that the morphism
$\mathcal{M}\to\mathrm{End}(\mathrm{Lens}_{S})$ is a strong monoidal functor.
Firstly, using that $\mathcal{A}$ is a $\mathcal{M}$-actegory and for every
$X\in\mathrm{Ob}(\mathcal{A})$, $S(X)$ is a $\mathcal{M}$-actegory, we deduce
that if $J$ is the unit object of $\mathcal{M}$ then $J\odot\\_$ is naturally
isomorphic to the identity endofunctor. In a similar fashion, given
$N,M\in\mathrm{Ob}(\mathcal{M})$ and $(X,A)$ in $\mathrm{Lens}_{S}$ we verify
that we have a natural isomorphism of endofunctors

$N\odot(M\odot(X,A))\xrightarrow{\sim}(N\star M)\odot(X,A).$

This concludes the proof. ∎

###  3.2 Bayes Learn

Let $\mathcal{C}$ be a Markov category which admits conditionals. Note that
this is equivalent to saying that $\mathcal{C}$ admits Bayesian inversions.
Since Bayesian inversions in general are defined up to an equivalence
relation, we restrict our attention to the category $\mathrm{PS}(\mathcal{C})$
(cf.§2.2.1). In fact, in this case, Bayesian inversion defined a symmetric
monoidal dagger functor on $\mathrm{PS}(\mathcal{C})$ in the sense of dagger
categories. We refer the reader to [8, Remark 13.10].

Recall from [3], the basis of the gradient learning functor $\mathrm{GL}$
comes from a functor

$R\colon\mathcal{C}\to\mathrm{Lens}(\mathcal{C})$

where in this case $\mathcal{C}$ is a Cartesian reverse differential category.
In our situation, we can mirror this construction via the mechanism of
Bayesian inversion and the notion of generalized lenses. We proceed below in
greater detail.

As above, our goal is to define a functor

$R\colon\mathrm{PS}(\mathcal{C})\to\mathrm{Lens}_{F}$

where $\mathrm{Lens}_{F}$ is the $F$-lens associated to a functor

$\mathrm{PS}(\mathcal{C})^{\mathrm{op}}\to\mathrm{Cat}$

(cf. [10]).

####  3.2.1 The functor $\mathrm{Stat}$

We define the functor

$\mathrm{Stat}\colon\mathrm{PS}(\mathcal{C})^{\mathrm{op}}\to\mathrm{Cat}$

as follows. Given $X\in\mathrm{Ob}(\mathrm{PS}(\mathcal{C}))$, let

$\mathrm{Stat}(X):=\mathrm{PS}(\mathcal{C}).$

Given a map $f\colon X\to Y$ in $\mathrm{PS}(\mathcal{C})$, the natural
transformation $F(Y)\to F(X)$ is the identity functor.

######  Remark 3.5.

_Observe that our definition of $\mathrm{Stat}$ coincides with the definition
of $\mathrm{Stat}$ from [9]. Indeed, since $\mathcal{C}$ is Markov, the unit
$I$ is a terminal object and hence there is a unique state $\iota\colon I\to
I$. We simplify notation as above and write $I$ in place of $(I,\iota)$.
$\mathrm{PS}(\mathcal{C})(I,(X,\pi_{X}))$ hence consists of a single element
corresponding to the state $\pi_{X}$. If $X\in\mathrm{PS}(\mathcal{C})$ and
$(A,\pi_{A}),(B,\pi_{B})\in\mathrm{Ob}(\mathrm{Stat}(X))$ then a morphism
$f\colon(A,\pi_{A})\to(B,\pi_{B})$ in $\mathrm{Stat}(X)$ coincides with a
morphism of sets_

$\mathrm{PS}(\mathcal{C})(I,(X,\pi_{X}))\to\mathrm{PS}(\mathcal{C})((A,\pi_{A}),(B,\pi_{B}))$

_since $\mathrm{PS}(\mathcal{C})(I,(X,\pi_{X}))$ is the singleton set. This is
precisely how $\mathrm{Stat}$ is defined in [9]._

####  3.2.2 The Grothendieck Lens

Let $\mathrm{Lens}_{\mathrm{Stat}}$ be the lens associated to the functor
$\mathrm{Stat}$ as introduced in [10, §3.1]. More precisely, we have that

  * •

The objects of the category $\mathrm{Lens}_{\mathrm{Stat}}$ are pairs
$((X,\pi_{X}),(A,\pi_{A}))$ where $(X,\pi_{X})\in\mathrm{PS}(\mathcal{C})$ and
$(A,\pi_{A})\in\mathrm{Stat}(X)$.

  * •

A morphism $\phi\colon((X,\pi_{X}),(A,\pi_{A}))\to((Y,\pi_{Y}),(B,\pi_{B}))$
is given by a morphism $(X,\pi_{X})\to(Y,\pi_{Y})$ in
$\mathrm{PS}(\mathcal{C})$ and a morphism $(B,\pi_{B})\to(A,\pi_{A})$ in
$\mathrm{Stat}(X)=\mathrm{PS}(\mathcal{C})$.

One checks that this is a well defined category and is an instance of the
_Grothendieck construction_ as in [10, §3.1].

######  Remark 3.6.

_The category $\mathrm{Lens}_{\mathrm{Stat}}$ simplifies considerably in our
situation._

$\mathrm{Lens}_{\mathrm{Stat}}\simeq\mathrm{PS}(\mathcal{C})\times\mathrm{PS}(\mathcal{C})^{\mathrm{op}}$

####  3.2.3  The functor _R_

We define the functor

$R\colon\mathrm{PS}(\mathcal{C})\to\mathrm{Lens}_{\mathrm{Stat}}$

as follows.

  * •

Given $(X,\pi_{X})\in\mathrm{PS}(\mathcal{C})$, we set
$R((X,\pi_{X})):=((X,\pi_{X}),(X,\pi_{X}))$.

  * •

If $f\colon(X,\pi_{X})\to(Y,\pi_{Y})$ is a morphism in
$\mathrm{PS}(\mathcal{C})$ then the map

$R(f)\colon((X,\pi_{X}),(X,\pi_{X}))\to((Y,\pi_{Y}),(Y,\pi_{Y}))$

in $\mathrm{Lens}_{\mathrm{Stat}}$ is defined to be the pair
$(f,f_{\pi_{X}}^{{\dagger}})$ where $f^{{\dagger}}_{\pi_{X}}$ is the Bayesian
inversion of $f$ with respect to the state $\pi_{X}$ on $X$. Note that this
simplification is a consequence of our discussion above.

######  Proposition 3.7.

The functor $R$ is well defined.

###### Proof.

We must essentially verify that $R$ behaves well with regards to composition.
This is a consequence of the fact that in the category
$\mathrm{PS}(\mathcal{C})$, Bayesian inversions are unique and their
composition is indeed well defined. ∎

####  3.2.4 The functor _BayesLearn_

Let $\mathcal{M}$ and $\mathcal{C}$ be Markov categories with $\mathcal{M}$
causal. To define BayesLearn, we now specialize to the case where
$\mathcal{C}$ is a symmetric monoidal $\mathcal{M}$-actegory which in addition
is in agreement with $\mathcal{M}$.

Since $\mathcal{M}$ is causal, $\mathrm{PS}(\mathcal{M})$ is a well defined
symmetric monoidal category. By Lemmas 3.3 and 3.4, the categories
$\mathrm{PS}(\mathcal{C})$ and $\mathrm{Lens}_{\mathrm{Stat}}$ are
$\mathrm{PS}(\mathcal{M})$-actegories. Recall from Section 2.3 that
$\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\\_)$ is a well defined functor
which when applied to $R$ gives a functor

$\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(R)\colon\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\mathrm{PS}(\mathcal{C}))\to\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\mathrm{Lens}_{\mathrm{Stat}})$

Recall that if $(\mathcal{P},J,\star)$ is a symmetric monoidal category then a
$\mathcal{P}$-actegory $\mathcal{A}$ admits a canonical functor
$j_{\mathcal{P},\mathcal{A}}\colon\mathcal{A}\to\mathbf{Para}_{\mathcal{P}}(\mathcal{A})$
given by $A\mapsto J\odot A$. Note that $j_{\mathcal{P},\mathcal{A}}$ is the
unit for the pseudo-monad defined by $\mathbf{Para}_{\mathcal{P}}(\cdot)$. We
thus have a diagram

$X$$Y$$\pi_{X\otimes
Y}$=$X$$X$$X$$Y$$\pi_{X}$$f$$X$$X$$X$$Y$$\psi$$f$$A$$X$$Y$$s$=$A$$A$$A$$X$$Y$$X$$X$$Y$$s$$t$$\pi_{X}$$f$=$\pi_{X}$$g$${\mathcal{C}}$${\mathrm{PS}(\mathcal{C})}$${\mathrm{Lens}_{\mathrm{Stat}}}$${\mathbf{Para}(\mathrm{PS}(\mathcal{C}))}$${\mathbf{Para}(\mathrm{Lens}_{\mathrm{Stat}})}$$\scriptstyle{R}$$\scriptstyle{j_{\mathrm{PS}(\mathcal{M}),\mathcal{C}}}$$\scriptstyle{j_{\mathrm{PS}(\mathcal{M}),\mathrm{Lens}_{\mathrm{Stat}}}}$$\scriptstyle{\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(R)}$

######  Definition 3.8.

_We define_

$\mathrm{BayesLearn}:=\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(R).$

######  Remark 3.9.

_Observe that the BayesLearn functor in § 3.2.4 does not have an update or
displacement endofunctor as the Gradient learning functor from [3, §3.5]. This
is due to the relatively simplified nature of Bayesian learning where
parameter updates correspond to obtaining the posterior distribution using the
prior and the likelihood and not as a result of optimizing with respect to a
loss function. Equivalently, in the categorical setting, Bayesian inversion
provides an update rule without recourse to a displacement or error
endofunctor._

######  Remark 3.10.

_Note that when we contrast the Bayesian learning framework described above
with the gradient learning framework described in[3], we see that Bayesian
learning is considerably simpler. By using the Grothendieck lens in place of
the standard Lens construction and $\mathrm{ProbStoch}(\mathcal{C})$ in place
of $\mathcal{C}$, the existence of Bayesian inversion as a dagger functor
implies the breakdown of the functor Stat resulting in the simplified form of
$\mathrm{Lens}_{\mathrm{Stat}}$ as described in Remark 3.6. In addition, the
absence of error and update endofunctors further distinguishes the Bayes
Learning framework. In this sense, we believe Bayesian learning to be the
simplest case of the categorical learning as described in [3]. _

######  Remark 3.11.

_Lemma 3.4 provides $\mathrm{PS}(\mathcal{C})$ with the structure of a
$\mathrm{PS}(\mathcal{M})$-actegory. However, by defining Stat for the
category $\mathcal{M}$ which we denote $\mathrm{Stat}_{\mathcal{M}}$, we also
get that_

$\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{M}}}\simeq\mathrm{PS}(\mathcal{M})\times\mathrm{PS}(\mathcal{M})^{\mathrm{op}}.$

_Hence $\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{\mathcal{C}}}}$ in fact has the
structure of $\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{M}}}$-actegory. Since we
are interested only in the BayesLearn functor and in particular its image, we
do not concern ourselves with this more general action of
$\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{M}}}$._

_Note also that we can endow $\mathrm{PS}(\mathcal{C})$ with the structure of
$\mathrm{Lens}_{\mathrm{Stat}_{\mathrm{PS}(\mathcal{M})}}$-actegory via the
projection_

$\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{M}}}\to\mathrm{PS}(\mathcal{M}).$

_This implies that we can also view
$\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{C}}}$ as a
$\mathrm{Lens}_{\mathrm{Stat}_{\mathcal{M}}}$-actegory in line with the theory
from [1]._

###  3.3 Bayes Learning algorithm

We now detail the Bayes Learning algorithm in our current setup. We preserve
our assumptions on $\mathcal{C}$ and $\mathcal{M}$ from the previous section.
We are given training data which consists of objects $X_{T}$ and $Y_{T}$ in
$\mathcal{C}$ and a joint distribution $\omega_{T}\colon I\to X_{T}\otimes
Y_{T}$. We would like to perform inference for objects $X_{*}$ and $Y_{*}$ in
$\mathcal{C}$. We are also provided with states $\pi_{X_{*}}\colon I\to X_{*}$
and $\pi_{X_{T}}\colon I\to X_{T}$ where $\pi_{X_{T}}$ is obtained by
marginalizing $\omega_{T}$. Note that in practice $X_{T}=X_{*}$ and
$Y_{T}=Y_{*}$ We proceed as follows.

  1. 1.

We model the given data by choosing a function $f\colon X_{T}\to Y_{T}$ in
$\mathbf{Para}_{\mathcal{M}}(\mathcal{C})$. This corresponds to a morphism
$f\colon M\odot X_{T}\to Y_{T}$. We assume that model satisfies a technical
assumption which we precise in (2). We assume we have a similar model
applicable for the inference data i.e. $f_{*}\colon M\odot X_{*}\to Y_{*}$.

  2. 2.

The morphism $f$ descends to give a morphism in the category
$\mathrm{PS}(\mathcal{C})$ as follows. Firstly, we endow $M$ with a prior
distribution i.e. a state $\pi_{M}\colon J\to M$. The morphism
$\pi_{M}\odot\pi_{X_{T}}\colon I\simeq J\odot I\to M\odot X_{T}$ defines a
state on $M\odot X_{T}$. By composing with $f$ we obtain a state on $Y_{T}$
i.e. a morphism $\pi_{Y_{T}}:=(\pi_{M}\odot\pi_{X_{T}});f\colon I\to Y_{T}$.
We assume that the model $f$ and the prior $\pi_{M}$ were chosen so as to
guarantee that $\pi_{Y_{T}}$ coincides with the marginal distribution on
$Y_{T}$ from $\omega_{T}$. The equivalence class of $f$ defines a morphism
$(M,\pi_{M})\odot(X_{T},\pi_{X_{T}})\to(Y_{T},\pi_{Y_{T}})$. Equivalently, we
have a map in
$\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\mathrm{PS}(\mathcal{C}))$.

  3. 3.

By construction,

$\mathrm{BayesLearn}(f):=(f,f^{\dagger})$

is a morphism in
$\mathbf{Para}_{\mathrm{PS}(\mathcal{M})}(\mathrm{Lens}_{\mathrm{Stat}})$
between objects $((X_{T},\pi_{X_{T}}),(X_{T},\pi_{X_{T}}))$ and
$((Y_{T},\pi_{Y_{T}}),(Y_{T},\pi_{Y_{T}}))$. Let

$f^{\dagger}\colon(Y_{T},\pi_{Y_{T}})\to(M,\pi_{M})\odot(X_{T},\pi_{X_{T}})$

denote the corresponding inversion.

  4. 4.

Recall that in §2.1, we outlined how to leverage the posterior distribution to
make predictions. This can be formalized in a categorical setting as follows.
Consider the composition

$\displaystyle(Y_{T},\pi_{Y_{T}})\otimes(X_{*},\pi_{X_{*}})$
$\displaystyle\xrightarrow{f^{\dagger}\otimes\mathrm{id}}((M,\pi_{M})\odot(X_{T},\pi_{X_{T}}))\otimes(X_{*},\pi_{X_{*}})$
$\displaystyle\simeq(X_{T},\pi_{X_{T}})\otimes((M,\pi_{M})\odot(X_{*},\pi_{X_{*}}))$

The isomorphism above is obtained by composing the isomorphisms

$\displaystyle((M,\pi_{M})$
$\displaystyle\odot(X_{T},\pi_{X_{T}}))\otimes(X_{*},\pi_{X_{*}})$
$\displaystyle\overset{(i)}{\simeq}(((M,\pi_{M})\odot
I)\otimes(X_{T},\pi_{X_{T}}))\otimes(X_{*},\pi_{X_{*}})$
$\displaystyle\overset{(ii)}{\simeq}((X_{T},\pi_{X_{T}})\otimes((M,\pi_{M})\odot
I))\otimes(X_{*},\pi_{X_{*}})$
$\displaystyle\overset{(iii)}{\simeq}(X_{T},\pi_{X_{T}})\otimes(((M,\pi_{M})\odot
I)\otimes(X_{*},\pi_{X_{*}}))$
$\displaystyle\overset{(iv)}{\simeq}(X_{T},\pi_{X_{T}})\otimes((M,\pi_{M})\odot(X_{*},\pi_{X_{*}}))$

where (i) is due to the mixed associator, (ii) is because of the swap
isomorphism, (iii) is a consequence of the associative property of the
monoidal product and (iv) is obtained by applying the mixed associator again.

We conditionalize the morphsim to get a morphism

$(X_{T},\pi_{X_{T}})\otimes(Y_{T},\pi_{Y_{T}})\otimes(X_{*},\pi_{X_{*}})\to(M,\pi_{M})\odot(X_{*},\pi_{X_{*}})$

By composing on the right by $f_{*}$, we get

$(X_{T},\pi_{X_{T}})\otimes(Y_{T},\pi_{Y_{T}})\otimes(X_{*},\pi_{X_{*}})\to(Y_{*},\pi_{Y_{*}}).$

This is the Bayes predictive distribution. By pre-composing with the state
$\pi_{X_{T}}\otimes\pi_{Y_{T}}$, we get a map
$(X_{*},\pi_{X_{*}})\to(Y_{*},\pi_{Y_{*}})$. We call this the full predictive
distribution obtained by averaging out the predictive distributions as they
vary over different instances of the training data.

######  Example 3.12.

_Let us work within the category BorelStoch which is the subcategory of Stoch
from Example 2.3. We make this restriction because BorelStoch admits
conditionals. In BorelStoch, we can view a morphism $f\colon A\to B$ as
defining a conditional distribution $p(b|a)$._

_Let $X$ and $Y$ be Borel spaces. Our training data consists of a list
$\mathbf{T}:=[(x_{1},y_{1}),\ldots,(x_{n},y_{n})]$ of points in
$\mathrm{List}(X\times Y)$. To align with our notation from §3.3, we write
$X_{T}$ and $Y_{T}$ to be copies of $X$ and $Y$ and endow $X_{T}\times Y_{T}$
with the empirical distribution obtained from $\mathbf{T}$. Our goal is to
obtain an estimate of the conditional probability $p(y_{*}|x_{*})$ for general
points $x_{*}\in X_{*}$ and $y_{*}\in Y_{*}$. As for the training set, let
$X_{*}$ and $Y_{*}$ be copies of $X$ and $Y$ respectively which we use for
inference. _

_As outlined in § 3.3 and §2.1, we model the given data via a parametrized
function of the form $f\colon P\times X_{T}\to Y_{T}$ where
$P\in\mathrm{BorelStoch}$. We endow $P$ with a prior distribution i.e. a state
$\pi_{P}\colon I\to P$ where $I=\\{*\\}$ is the unit object. We must update
the prior $p$ to get the posterior distribution. This is accomplished via the
Bayesian inversion $f^{\dagger}\colon Y_{T}\to P\times X_{T}$. Note that
$f^{\dagger}$ is not unique. By conditionalizing, we get a map $X_{T}\times
Y_{T}\to P$ in BorelStoch which defines the posterior._

_In this case, the Bayes predictive density as described in § 3.3 is given by_

$p(y_{*}|x_{*},T)=\int_{P}p(y_{*}|P,x_{*},T)p(P|x_{*},T).$

_This is a consequence of how compositions are defined in Stoch i.e. via the
Chapman-Kolmogorov equation cf. § 2.3._

##  4 Bayes updates

While Section 3.3 describes the Bayes Learning algorithm, we observe that it
does not provide a mechanism by which we can _update_ the prior on the
parameter space. Recall, via Bayesian inversion we obtain a channel
$(X_{T},\pi_{X_{T}})\otimes(Y_{T},\pi_{Y_{T}})\to(M,\pi_{M})$. However, in
practice, when working in a suitable sub-category of Stoch or in FinSet, we
are given a training set

$\mathbf{T}:=\\{(x_{1},y_{1}),\ldots,(x_{n},y_{n})\\}$

which we use to obtain the posterior distribution on $M$. Our goal in this
section is to translate this into the categorical framework we have developed
so far.

In the case of Stoch, we can represent a data point $(x,y)\in X_{T}\times
Y_{T}$ as the product of a pair of morphisms $I\to X_{T}$ and $I\to Y_{T}$
mapping $*$ to the probability measures that concentrate at the points $x$ and
$y$ respectively. We use $\delta_{x}$,$\delta_{y}$ respectively to denote
these maps. Given a channel $c\colon X\times Y\to M$ in Stoch, we define a
state on $M$ via the composition

$I\to I\times I\xrightarrow{\delta_{x_{1}}\times\delta_{y_{1}}}X\times
Y\xrightarrow{c}M.$

This effectively defines the posterior on $M$ given the training set
$\mathbf{T}_{1}$ where $\mathbf{T}_{1}:=\\{(x_{1},y_{1})\\}$. The natural
question to ask in this setting is how to sequentially update the posterior to
achieve the required update over the entire training dataset and if one can
also achieve such an update all at once. We outline a possible solution in
what follows.

For the remainder of this section we work with a Markov category
$(\mathcal{C},I,\otimes)$ such that $\mathcal{C}=\mathrm{Kl}(\mathcal{P})$
where $\mathcal{P}\colon\mathcal{D}\to\mathcal{D}$ is a symmetric monoidal
monad on the symmetric monoidal category $(\mathcal{D},I_{\mathcal{D}},\ast)$.
Furthermore, we suppose that $\mathcal{C}$ admits conditionals. As before, let
$(\mathcal{M},J,\star)$ be a symmetric monoidal category such that
$\mathcal{C}$ is a symmetric monoidal $\mathcal{M}$-actegory.

We are given a model i.e. a morphism $f\colon M\odot X\to Y$ in $\mathcal{C}$
where we think of $M\in\mathrm{Ob}(\mathcal{M})$ as the parameter space and
$f$ models a true morphism $X\to Y$. We suppose as before that we are provided
with a state $\pi_{X}\colon I\to X$ and a prior $\pi_{M,0}\colon J\to M$.
Lastly, we abuse notation and write $M$ in place of $M\odot I$ when necessary.
Note a prior $\pi_{M}\colon J\to M$ implies a prior
$\pi_{M}\odot\mathrm{id}\colon J\odot I\to M\odot I$ which we also refer to as
$\pi_{M}$.

###  4.1 Sequential updates

Recall from Definition 2.4 that the state $\pi_{X}$ and the channel $f$ give
us a morphism

$f_{\mathrm{joint}}\colon M\odot I\to X\otimes Y$ (5)

in $\mathcal{C}$. Since $\mathcal{C}$ admits conditionals, it also admits
Bayesian inversions. Hence, we get a morphism

$f_{\mathrm{joint}}^{\dagger}\colon Z\to M\odot I$

with respect to the prior state $\pi_{M,0}$ on $M$ where $Z:=X\otimes Y$.

Note that $f_{\mathrm{joint}}^{\dagger}$ is not unique. In the previous
section, we got around this issue by working in the category
$\mathrm{PS}(\mathcal{C})$. However, if we want to update the state on
$\mathcal{M}$ sequentially then this corresponds to sequentially updating
objects in $\mathrm{PS}(\mathcal{M})$ which will then require us to update the
model $f$ or more precisely its image in $\mathrm{PS}(\mathcal{C})$. Instead,
we introduce Definition 4.1 to ensure that the updated priors remain well
defined.

Recall that $\mathcal{C}=\mathrm{Kl}(\mathcal{P})$ where $\mathcal{P}$ is a
monad on the symmetric monoidal category $(\mathcal{D},I_{\mathcal{D}},\ast)$.
Since $\mathcal{P}$ is a monad, we have a family of maps $\eta_{X}\colon
X\to\mathcal{P}(X)$ for every $X\in\mathrm{Ob}(\mathcal{D})$. Given a map
$a\colon X\to Y$ in $\mathcal{D}$, let $\eta(a)$ denote its image in
$\mathcal{C}$ i.e. the composition
$X\xrightarrow{a}Y\xrightarrow{\eta_{Y}}\mathcal{P}(Y)$.

######  Definition 4.1.

_Let $f\colon X\to Y$ be a morphism in $\mathcal{C}$ and $\pi_{X}\colon I\to
X$ be a state on $X$. Let $y\colon I_{\mathcal{D}}\to Y$ be a morphism in
$\mathcal{D}$. We say that the_ Bayesian inverse of $f$ is uniquely defined at
$y$ _if for any morphisms $g,h\colon Y\to X$ in $\mathcal{C}$ such that
$g\sim_{\pi_{X}-a.s}h$ and $g$ is a Bayesian inversion of $f$ then
$\eta(y);g=\eta(y);h$._

We refer to the morphism $y$ that appears in the definition above as an
_elementary point_ of the object $Y$. A precise definition is as follows.

######  Definition 4.2.

_Let $Y$ be an object of $\mathcal{C}$. By an_ elementary point of $Y$ _we
mean a morphism $y\colon I_{\mathcal{D}}\to Y$ in $\mathcal{D}$. We use
$\delta_{y}$ to denote the image of $y$ in $\mathcal{C}$ i.e.
$\delta_{y}:=\eta(y)$._

######  Example 4.3.

_We provide an example of Definition 4.1. Recall from Example 2.3, the
category FinStoch. Note that $\mathrm{FinStoch}=\mathrm{Kl}(\mathrm{Dist})$
where $\mathrm{Dist}\colon\mathrm{FinSet}\to\mathrm{Finset}$ is a symmetric
monoidal monad on the symmetric monoidal category FinSet whose objects are
finite sets and morphisms are functions of sets. Let $X$ be a finite set,
$\pi_{X}\colon I\to X$ be a state on $X$ and $f\colon X\to Y$ be a morphism in
FinStoch. By definition, $\pi_{X}$ corresponds to a probability distribution
$p_{X}$ on $X$ while $f$ defines a conditional distribution. Let $y_{0}\colon
I\to Y$ be a morphism in FinSet. It follows that $y_{0}$ is uniquely
determined by a point in $Y$ which we abuse notation for and call $y_{0}$ as
well. The Bayesian inversion $g\colon Y\to X$ is defined by_

$g(y)(x)=\frac{f(x)(y)p_{X}(x)}{\sum_{x^{\prime}\in
X}f(x^{\prime})(y)p_{X}(x^{\prime})}$

_if $\sum_{x^{\prime}\in X}f(x^{\prime})(y)p_{X}(x^{\prime})\neq 0$ and if $y$
is such that $\sum_{x^{\prime}\in X}f(x^{\prime})(y)p_{X}(x^{\prime})=0$ then
$g(y)$ can be any probability distribution on $X$. Thus we see in this
situation that_ the Bayesian inversion $g$ is uniquely defined at $y_{0}$ _if
and only if_

$\sum_{x^{\prime}\in X}f(x^{\prime})(y_{0})p_{X}(x^{\prime})\neq 0.$

_In the case of the category Stoch, things are more complicated since it is
not always true that a Bayesian inversion exists. However, in certain cases
where we are working with subspaces of $\mathbb{R}$ and both state and channel
are defined using density functions then a similar calculation as above can be
performed (cf.[2, Example 3.9])._

Let $\mathbf{T}:=[x_{1}\otimes y_{1},\ldots,x_{n}\otimes y_{n}]$ be a list of
$n$ elementary points of $X\otimes Y$ where for every $i$,
$z_{i}:=x_{i}\otimes y_{i}$ satisfies a property to be specified below. We
begin with a prior $\pi_{M,0}$ on the parameter object $M$. Let us suppose
that we have obtained the $i$-th sequential update i.e. a state $\pi_{M,i}$ on
$M$. We define $\pi_{M,i+1}$ as follows. By taking the Bayesian inversion of
$f_{\mathrm{joint}}$ with respect to $\pi_{M,i}$, we get

$f^{{\dagger}}_{\mathrm{joint},i}\colon X\otimes Y\to M.$

The state $\pi_{M,i+1}$ on $X\otimes Y$ is given by the composition

$I\xrightarrow{\delta_{x_{i+1}}\otimes\delta_{y_{i+1}}}X\otimes
Y\xrightarrow{f^{{\dagger}}_{\mathrm{joint},i}}M$

and we suppose that the point $z_{i+1}$ is such that
$f^{{\dagger}}_{\mathrm{joint},i}$ is unique at $z_{i+1}$.

######  Example 4.4.

_Let us demonstrate the sequential update procedure in the category FinStoch
acting on itself. As above, we are given model $f\colon M\times X\to Y$ where
$M,X$ and $Y$ are finite sets and $f$ is a morphism in FinStoch. This induces
a function $f_{\mathrm{joint}}\colon M\to Z$ where $Z:=X\times Y$. Let us
assume we are give a training set_

$\mathbf{T}:=[(x_{1},y_{1}),(x_{2},y_{2})]$

_and a prior state $\pi_{M,0}$ on $M$. In this context, this means a
probability distribution on $M$. Let $z_{i}:=(x_{i},y_{i})$._

  1. 1.

_For $i=1$, we have that for $m\in M$ and $z\in Z$,_

$f^{{\dagger}}_{\mathrm{joint},0}(z)(m)\pi_{Z,0}(z)={f_{\mathrm{joint}}(m)(z)\pi_{M,0}(m)}$

_where for $z\in Z$,_

$\pi_{Z,0}(z)=\sum_{m\in
M}{f_{\mathrm{joint}}(m^{\prime})(z)\pi_{M,0}(m^{\prime})}.$

_Recall our assumption that $\pi_{Z,0}(z_{1})\neq 0$. We update the prior by
setting $\pi_{M,1}(m):=f^{{\dagger}}_{\mathrm{joint},0}(z_{1})(m)$._

  2. 2.

_Likewise, for $i=2$,_

$f^{{\dagger}}_{\mathrm{joint},1}(z)(m)\pi_{Z,1}(z)={f_{\mathrm{joint}}(m)(z)\pi_{M,1}(m)}$

_where for $z\in Z$,_

$\pi_{Z,1}(z)=\sum_{m\in
M}{f_{\mathrm{joint}}(m^{\prime})(z)\pi_{M,1}(m^{\prime})}.$

_Expanding using (1),_

$f^{{\dagger}}_{\mathrm{joint},1}(z_{2})(m)\propto{f_{\mathrm{joint}}(m)(z_{2})f_{\mathrm{joint}}(m)(z_{1})\pi_{M,0}(m)}$

_We update the prior by setting
$\pi_{2}(m):=f^{{\dagger}}_{\mathrm{joint},1}(z_{2})(m)$._

###  4.2 Batch updates

To obtain an update of the prior all at once, we work with an object built
from $X\otimes Y$ but whose _points_ correspond to datasets of a specified
cardinality.

Let $n\in\mathbb{N}$ and we set

$Z_{n}:=\otimes^{n}(X\otimes Y).$

The model $f$ induces a morphism

$f_{\mathrm{joint}}^{n}\colon M\to Z_{n}$

defined as the composition

$M\xrightarrow{\otimes^{n}\mathbf{copy}_{M}}\otimes^{n}M\xrightarrow{\otimes^{n}f_{\mathrm{joint}}}Z_{n}.$

As before, let $\mathbf{T}:=[x_{1}\otimes y_{1},\ldots,x_{n}\otimes y_{n}]$ be
a list of $n$ elementary points of $X\otimes Y$. The list $\mathbf{T}$ defines
an elementary point of $Z_{n}$. Indeed, we set $z_{\mathbf{T}}$ to be the
composition

$I_{\mathcal{D}}\xrightarrow{\otimes^{n}\mathbf{copy}_{I_{\mathcal{D}}}}\otimes^{n}I_{\mathcal{D}}\xrightarrow{z_{1}\otimes\ldots\otimes
z_{n}}Z_{n}$

where as before $z_{i}=x_{i}\otimes y_{i}$. We suppose $\mathbf{T}$ is such
that the Bayesian inversion of $f_{\mathrm{joint}}^{n}$ is uniquely defined at
$z_{\mathbf{T}}$. The batch update of the prior $\pi_{M,0}$ with respect to
$\mathbf{T}$ is given by the composition

$I\xrightarrow{\delta_{z_{\mathbf{T}}}}Z\xrightarrow{(f_{\mathrm{joint}}^{n})^{{\dagger}}}M$

where $(f_{\mathrm{joint}}^{n})^{{\dagger}}$ is a Bayesian inversion of
$f_{\mathrm{joint}}^{n}$ with respect to the prior $\pi_{M,0}$. Let
$\pi_{M,\mathbf{T}}\colon I\to M$ denote this updated prior.

We can ask the following question. Under what conditions, can we ensure that
$\pi_{M,n}=\pi_{M,\mathbf{T}}$ where $\pi_{M,n}$ is as defined at the end of
§4.1 for $\mathbf{T}$.

######  Example 4.5.

_Let us continue our discussion as in Example 4.4 using the category FinStoch.
As in 4.4, we are given a model $f\colon M\times X\to Y$ where $M,X$ and $Y$
are finite sets and $f$ is a morphism in FinStoch, a training set_

$\mathbf{T}:=\\{(x_{1},y_{1}),(x_{2},y_{2})\\}$

_and a prior state $\pi_{M,0}$ on $M$. Let $z_{i}:=(x_{i},y_{i})$ and
$z_{\mathbf{T}}:=(z_{1},z_{2})$. We import notation from 4.4._

_It follows that_

$Z_{2}=(X\times Y)^{2}$

_and_

$f^{2}_{\mathrm{joint}}\colon M\to Z_{2}.$

_Hence, we get that_

$(f^{2}_{\mathrm{joint}})^{{\dagger}}\colon Z_{2}\to M.$

_For $m\in M$ and $w\in Z_{2}$,_

$(f^{2}_{\mathrm{joint}})^{{\dagger}}(w)(m)\propto
f^{2}_{\mathrm{joint}}(m)(w)\pi_{M,0}(m).$

_We set_

$\pi_{M,\mathbf{T}}(m):=(f^{2}_{\mathrm{joint}})^{{\dagger}}((z_{1},z_{2}))(m)$

_for $m\in M$ and hence_

$\pi_{M,\mathbf{T}}(m)\propto
f_{\mathrm{joint}}(m)(z_{2})f_{\mathrm{joint}}(m)(z_{1})\pi_{M,0}(m)$

_since by definition,
$f^{2}_{\mathrm{joint}}(m)((a,b))=f_{\mathrm{joint}}(m)(a)f_{\mathrm{joint}}(m)(b).$_

######  Remark 4.6.

_Observe from examples 4.4 and 4.5 that the sequential Bayes update and batch
update coincide. This begs the following question. What conditions can we
impose to relate the sequential and batch updates in the general setting of
the category $\mathcal{C}$ used throughout this section._

## References

  * [1] Matteo Capucci, Bruno Gavranović, Jules Hedges, and Eigil Fjeldgren Rischel.  Towards foundations of categorical cybernetics, 2021.
  * [2] Kenta Cho and Bart Jacobs.  Disintegration and bayesian inversion via string diagrams.  Mathematical Structures in Computer Science, 29(7):938?971, 2019\.
  * [3] G. S. H. Cruttwell, Bruno Gavranovic, Neil Ghani, Paul Wilson, and Version 1 Fabio Zanasi.  Categorical foundations of gradient-based learning, 2021.
  * [4] Giovanni de Felice, Alexis Toumi, and Bob Coecke.  Discopy: Monoidal categories in python.  Electronic Proceedings in Theoretical Computer Science, 333:183’Äì197, Feb 2021.
  * [5] Michael Evans and Tim B. Swartz.  Methods for approximating integrals in statistics with special emphasis on bayesian integration problems.  Statistical Science, 10:254–272, 1995.
  * [6] Anita C. Faul and Michael E. Tipping.  A variational approach to robust regression.  In Georg Dorffner, Horst Bischof, and Kurt Hornik, editors, Artificial Neural Networks — ICANN 2001, pages 95–102, Berlin, Heidelberg, 2001. Springer Berlin Heidelberg.
  * [7] Brendan Fong, David I. Spivak, and Rémy Tuyéras.  Backprop as functor: A compositional perspective on supervised learning, 2019.
  * [8] Tobias Fritz.  A synthetic approach to markov kernels, conditional independence and theorems on sufficient statistics.  Advances in Mathematics, 370:107239, 2020.
  * [9] Toby St. Clere Smithe.  Bayesian updates compose optically, 2020.
  * [10] David I. Spivak.  Generalized lens categories via functors $\mathcal{C}^{\rm op}\to\mathsf{Cat}$, 2020.
  * [11] ME Tipping.  Bayesian inference: An introduction to principles and practice in machine learning.  pages 41–62, 01 2004.