# Mix-GENEO: A flexible filtration for multiparameter persistent homology
detects digital images

Jiangxing He  Jiangxing He, School of Mathematics, Jilin University, 130012,
Changchun, P. R. China<EMAIL_ADDRESS>,  Bingzhe Hou  Bingzhe Hou, School
of Mathematics, Jilin University, 130012, Changchun, P. R. China
<EMAIL_ADDRESS>,  Tieru Wu  Tieru Wu, School of Mathematics, Jilin
University, 130012, Changchun, P. R. China  School of Artificial Intelligence,
Jilin University, 130012, Changchun, P. R. China<EMAIL_ADDRESS>and  Yue Xin
Yue Xin, School of Mathematics, Jilin University, 130012, Changchun, P. R.
China<EMAIL_ADDRESS>
###### Abstract.

Two important problems in the field of Topological Data Analysis are defining
practical multifiltrations on objects and showing ability of TDA to detect the
geometry. Motivated by the problems, we constuct three multifiltrations named
multi-GENEO, multi-DGENEO and mix-GENEO, and prove the stability of both the
interleaving distance and multiparameter persistence landscape of multi-GENEO
with respect to the pseudometric of the subspace of bounded functions. We also
give the estimations of upper bound for multi-DGENEO and mix-GENEO. Finally,
we provide experiment results on MNIST dataset to demonstrate our
bifiltrations have ability to detect geometric and topological differences of
digital images.

###### Key words and phrases:

Topological Data Analysis, Multifiltration, Interleaving Distance,
Multiparameter Persistence Landscape, Machine Learning.

##  1\. Introduction

Topological and Geometric Data Analysis (TGDA) describes an emerging method to
distinguish topological and geometry features combined with data analysis
tools. While the history of TDA (Topological Data Analysis) could date back to
the 1990’s, the field has been developed rapidly in recent years, which leads
to a rich of theoretical foundations [1, 2, 3, 4], high efficient algorithms
[5] and software [6, 7], and a board range of applications including medicine,
ecology, materials science, deep learning and graphics [8, 9, 10, 11, 12, 13,
14].

A ubiquitous tool in TGDA is Persistence Homology(PH). The theory of PH
studies the homological group of a family of topological spaces and its
representation which is called persistence module, see [15, 16] for details.
In probability theory, several authors have proposed estimators of fractal
dimension defined in terms of minimum spanning trees and higher dimensional
persistent homology [17, 18, 19, 20].

However, a single filtered space can not often adequately capture the
structure of interest in the data. This leads one to consider multiparameter
persistence. Multiparameter Persistence Homology(MPH) was first considered by
Calsson and Zomorodian in [21], in which they studied a multifiltration: a
family of spaces parametrized along multiple geometric dimensions. The
algebraic invariants of these multifiltrations are called multiparameter
persistence modules. Unlike the single persistence, there is not an analogous
complete discrete invariant for multiparamter module.

[22] and [23] introduced the stable vectorization of the persistence diagram
called persistence landscape and the stable vectorization of the
multiparameter persistence module called multiparameter persistence landscape,
respectively. Besides, another stable vectorization of the persistence diagram
is the persistence image [24] which has been shown to produce favourable
classification results when combined with machine learning methods.
Multiparameter persistence image was introduced in [25] which is suitable for
machine learning and statistical frameworks.

Moreover, by using the geometric features of data extracted by PH and MPH as
inputs for statistic techniques, one can provide new insights into the data.
Persistence diagram could mark the parameter values for births and deaths of
homological features. In a popular point of view, it is said that the long
intervals represent the topological signal and the short intervals represent
the noise. However, authors in [26] proved that persistent homology detects
the curvature of disks which showing that the short intervals also encode
geometric information. In [27] persistent homology is a mathematically
motivated out-of-the-box tool that one can use to summarize not only the
global topology but also the local geometry of a wide variety of datasets.

Within the framework of 1-parameter persistent homology, there have been many
proposals to build filtrations, including the removal of low density outliers
[28], filtering by density functions and kernel density functions [29, 30,
31], measuring constructions by distances [32, 33, 34, 35], and subsampling
[36]. However, there are several disadvantages for 1-parameter persistence.
For instance, 1-parameter persistence is only determined by one single
parameter, it is unable to distinguish small spatial features and large ones.

Several methods to construct multipatameter filtrations for points have been
proposed such as the superlevel-rips bifiltration [21], the multicover
bifiltration [37] and the rhomboid bifiltration [38]. These approaches can be
found in [39] for details. In [25], the authors constructed a 2-parameter
sublevel filtration from a pair by two images from a piece of human issue of a
patient suffering from beast cancer.

###  1.1. Motivation

Many applications of 1-parameter persistent homology concern image analysis,
where sublevel filtrations are often used. There is not yet a consensus on
what the most natural or useful multifiltrations are for image analysis, but
one promising idea is that a second persistence parameter can be used to
thicken sublevel or superlevel sets, thereby introducing some sensitivity to
the width of features that the ordinary sublevel and superlevel filtrations
lack. One construction [40] along these lines is a framework to use
morphological operations naturally form a multiparameter filtration to
denoise.

We would like to build a multifiltration of digital images to compute
multiparameter homology, and then detect significant topological and geometric
features from the multiparameter persistent landscape.

The multiparameter landscape functions are sensitive to homological features
of large, medium and small persistence. The landscapes also have the advantage
of being interpretable since they are closely related to the rank invariant.

Frequently in topological data analysis, we need to consider several
$\mathbb{R}$-valued functions

$\gamma_{i}:X\rightarrow\mathbb{R},\ i=1,...,n.$

It is equivalent to consider a function $\gamma:X\rightarrow{\mathbb{R}^{n}}$
on a topological space $X$ which gives rise to an $n$-$parameter\ sublevelset\
filtration\ \mathcal{S}(\gamma)$, defined by

$\mathcal{S}(\gamma)_{\bm{a}}=\\{\ y\in{X}\ \mid\ \gamma(y)\leq\bm{a},\
\bm{a}\in\mathbb{R}^{n}\ \\}.$

We want to explore the impact of different levels of filtration on
multiparameter persistence module. In [41], authors defined group equivariant
non-expansive operators(GENEOs) whose space is compact and convex with respect
to the proper pseudometrics. Based on the stability, they described a simple
strategy to select and sample operators and show how the operators can be used
to perform machine learning. Also, they provided a flexible way to select
operators. GENEO can be viewed as Gaussian blur, and Laplace operator can be
viewed as sharpening which can be thought of the subtraction of two different
GENEOS called DOG. Combined with the definition of GENEO, we can use GENEO to
construct n-parameter persistent filtrations which are named multi-GENEO,
multi-DGENEO and mix-GENEO in the present paper.

To construct n-parameter filtration from a data set, we represent data as
function. The following notations are from [41]. Suppose that $X$ be a non-
empty set and $\varPhi$ be a topological subspace of all bounded functions
from $X$ to $\mathbb{R}$. Obviously $\varPhi$ is naturally endowed with the
topology induced by the distance

$D_{\varPhi}:={\|}\varphi_{1}-\varphi_{2}{\|}_{\infty}\ .$

Denote by $Homeo(X)$ the set of all homeomorphisms from X to X. For
$g\in{Homeo(X)}$, if for every $\varphi\in{\varPhi}$,
$\varphi\circ{g}\in{\varPhi}$ and $\varphi\circ{g^{-1}}\in{\varPhi}$ , we say
$g$ is a $\varPhi-$preserving homeomorphism. Denote by $Homeo_{\varPhi}(X)$
the set of all $\varPhi-$preserving homeomorphisms on X. Let $G$ be the
subgroup of $Homeo_{\varPhi}(X)$, the pair $(\varPhi,G)$ is called a
$perception$ $pair$. Let $(\varPhi,G)$, $(\varPsi,H)$ be two perception pairs
and $T:G\rightarrow{H}$ be a fixed homomorphism. If each linear operator
$F:\varPhi\rightarrow\varPsi$ satisfies
$F(\varphi\circ{g})=F(\varphi)\circ{T(g)}$ for every $\varphi\in\varPhi$,
$g\in G$ is said to be a $group$ $equivariant$ $operator$ from $(\varPhi,G)$
to $(\varPsi,H)$. Moreover, the definition of GENEO is as follows,

######  Definition 1.1.

[41] Assume that $(\varPhi,G)$, $(\varPsi,H)$ are two perception pairs and a
homomorphism $T:G\rightarrow{H}$ has been fixed. If $F$ is a group equivariant
operator from $(\varPhi,G)$ to $(\varPsi,H)$ with respect to $T$ and $F$ is
non-expansive(i.e.,
$D_{\varPsi}(F(\varphi_{1},F(\varphi_{2}))\leq{D_{\varPhi}(\varphi_{1},\varphi_{2})}$
for every $\varphi_{1},\varphi_{2}\in{\varPhi}$), then $F$ is called a $Group$
$Equivariant$ $Non-Expansive$ $Operator$ $(GENEO)$ associated with
$T:G\rightarrow{H}$.

In this paper, we could define multi-GENEO as follows.

######  Definition 1.2.

A multi-GENEO filtration $\\{\gamma_{i}\\}_{i=1}^{n}$ is a multiparameter
filtration defined by $\gamma_{i}=F^{i}(\varphi)$, where $\varphi\in\varPhi$,
and each $F^{i}$ is a GENEO, $i=1,...,n$. A multi-DGENEO
$\\{\gamma_{i}\\}_{i=1}^{n}$ is a multiparameter filtration defined by
$\gamma_{i}=L^{i}(\varphi)=F^{1,i}(\varphi)-F^{2,i}(\varphi)$ where
$\varphi\in\varPhi$, $F^{1,i}$ and $F^{1,i}$ are two elements in GENEO,
$i=1,...,n$. Moreover, each $\gamma_{i}$ is chosen to be $F^{i}(\varphi)$ or
$L^{i}(\varphi)$, we call $\\{\gamma_{i}\\}_{i=1}^{n}$ is a mix-GENEO.

###  1.2. Contributions

In this paper we provide a flexible framework to build multiparameter
filtrations on digital images.

$\bullet$ We introduce three methods to build multiparameter filtrations
called multi-GENEO, multi-DGENEO and mix-GENEO.

$\bullet$ We show the stability of both interleaving distance and
multiparameter persistence landscape of multi-GENEO, and also provide bound
estimates for both multi-DGENEO and mix-GENEO with respect to pseudometric for
the subset of bounded functions.

$\bullet$ We conduct experiments on MNIST dataset and demonstrate our
bifiltrations making sense to identify features of persistence modules by
traditional machine learning methods, which shows the ability of the
multiparameter persistence homology to detect geometric and topological
differences of digital images.

To foster further developments at the intersection of multiparameter
persistent homology and machine learning theory, we release our source code
under:https://github.com/HeJiaxing-hjx/Mix-GENEO/.

##  2\. Background

In this section, we will introduce some definitions and properties used in
this paper.

Let $\mathbb{Z}$ be the set of integers, $\mathbb{N}$ be the set of non-
negative integers and $\mathbb{R}$ be the set of real numbers. Suppose that
$\mathbb{K}$ is one of $\mathbb{Z}$, $\mathbb{N}$ and $\mathbb{R}$. For
vectors $\bm{a}$, $\bm{b}$ in $\mathbb{K}^{n}$, there is a natural partial
order on $\mathbb{K}^{n}$ by taking $(a_{1},...,a_{n})\leq(b_{1},...,b_{n})$
if and only if $a_{i}\leq b_{i}$ for all $1\leq i\leq n$. Denote by $X$ a
collection $\\{X_{\bm{a}}\\}_{\bm{a}\in\mathbb{R}^{n}}$ and denote by $\pi$
the collection of continuous maps $\pi_{\bm{a},\bm{b}}:X_{\bm{a}}\rightarrow
X_{\bm{b}}$ such that the diagram

${X_{\bm{a}}}$${X_{\bm{b}}}$${X_{\bm{c}}}$$\scriptstyle{\pi_{\bm{a},\bm{c}}}$$\scriptstyle{\pi_{\bm{a},\bm{b}}}$$\scriptstyle{\pi_{\bm{b},\bm{c}}}$

commutes. Denote by $\bm{Top}^{\mathbb{K}^{n}}$ the category whose objects are
$(X,\pi)$ and whose morphisms are maps $f:(X,\pi)\rightarrow(Y,\tilde{\pi})$
which is a collection of all continuous maps $\\{f_{\bm{a}}\\}$ for all
$\bm{a}\in\mathbb{K}^{n}$ such that $f_{\bm{a}}:X_{\bm{a}}\rightarrow
Y_{\bm{a}}$ and the diagram

${X_{\bm{a}}}$${X_{\bm{b}}}$${Y_{\bm{a}}}$${Y_{\bm{b}}}$$\scriptstyle{f_{\bm{a}}}$$\scriptstyle{\pi_{\bm{a},\bm{b}}}$$\scriptstyle{f_{\bm{b}}}$$\scriptstyle{\tilde{\pi}_{\bm{a},\bm{b}}}$

commutes.

######  Example 2.1.

Denote the sublevelset filtration $X_{\bm{t}}$ by

$\mathcal{S}(\gamma)_{\bm{t}}=\\{\ y\in X\ |\ \gamma(y)\leq\bm{t}\ \\}$

with natural inclusion $\pi_{\bm{t},\bm{s}}$,
$\bm{t}\leq\bm{s}\in\mathbb{R}^{n}$. Then $(X,\pi)$ is one object of
$\bm{Top}^{\mathbb{K}^{n}}$. Similarly, let
$Y_{\bm{t}}=\mathcal{S}(\gamma\circ f^{-1})_{\bm{t}}$ with natural inclusion
$\tilde{\pi}_{\bm{t},\bm{s}}$, $\bm{t}\leq\bm{s}\in\mathbb{R}^{n}$. Then
$(Y,\tilde{\pi})$ is also an object in $\bm{Top}^{\mathbb{K}^{n}}$. For a
homeomorphism $\iota:X\rightarrow Y$, it can induce a morphism
$f:(X,\pi)\rightarrow(Y,\tilde{\pi})$, where $f_{\bm{t}}:X_{\bm{t}}\rightarrow
Y_{\bm{t}}$ induced by $f_{t}(x)=\iota(x)$.

Let $M=\oplus_{\bm{a}\in\mathbb{K}^{n}}M_{\bm{a}}$, where $M_{\bm{a}}$ is a
module. For any $\bm{a}\leq\bm{b}$, there is a homomorphism
$\tau_{\bm{a},\bm{b}}:M_{\bm{a}}\rightarrow M_{\bm{b}}$ such that the
following diagram commutes,

${M_{\bm{a}}}$${M_{\bm{b}}}$${M_{\bm{c}}}$$\scriptstyle{\tau_{\bm{a},\bm{c}}}$$\scriptstyle{\tau_{\bm{a},\bm{c}}}$$\scriptstyle{\tau_{\bm{b},\bm{c}}}$

when $\bm{a}\leq\bm{b}\leq\bm{c}$. Denote by $\tau$ a collection of
$\\{\tau_{\bm{a},\bm{b}}\\}$ for all $\bm{a}\leq\bm{b}$. Denote by
$M^{\mathbb{K}^{n}}$ the category whose objects are $(M,\tau)$ and whose
morphisms are maps $h:(M,\tau)\rightarrow(N,\tilde{\tau})$ which is a
collection of all continuous maps $\\{h_{\bm{a}}\\}$ for all
$\bm{a}\in\mathbb{K}^{n}$ such that $h_{\bm{a}}:M_{\bm{a}}\rightarrow
N_{\bm{a}}$ and the diagram

${M_{\bm{a}}}$${M_{\bm{b}}}$${N_{\bm{a}}}$${N_{\bm{b}}}$$\scriptstyle{h_{\bm{a}}}$$\scriptstyle{\tau_{\bm{a},\bm{b}}}$$\scriptstyle{h_{\bm{b}}}$$\scriptstyle{\tilde{\tau}_{\bm{a},\bm{b}}}$

commutes.

Notice that homology can be viewed as a functor from
$\bm{Top}^{\mathbb{K}^{n}}$ to $M^{\mathbb{K}^{n}}$. Define the functor
$H:\bm{Top}^{\mathbb{K}^{n}}\rightarrow M^{\mathbb{K}^{n}}$ assigns to each
object $(X,\pi)$ in $\bm{Top}^{\mathbb{K}^{n}}$ the object $(H(X),\pi_{*})$ in
$M^{\mathbb{K}^{n}}$ and to each morphism
$f\in\textrm{Mor}((X,\pi),(Y,\tilde{\pi}))$ in $\bm{Top}^{\mathbb{K}^{n}}$ the
morphism $f_{*}\in\textrm{Mor}((H(X),\pi_{*}),(H(Y),\tilde{\pi}_{*}))$ in
$M^{\mathbb{K}^{n}}$. Notice that $\mathbbm{1}_{*}=\mathbbm{1}$, $(g\circ
f)_{*}=g_{*}\circ f_{*}$ and $(\tilde{\pi}_{\bm{a},\bm{b}}\circ
f)_{*}=\tilde{\pi}_{(\bm{a},\bm{b})*}\circ f_{*}$. To see more details about
homology theory, we refer to [42].

Next, we would like to introduce three pseudometrics $d_{\infty}$, $d_{I}$ and
$d_{\lambda}^{(p)}$. Recall that an extended pseudometric on $X$ is a function
$d:X\times X\rightarrow[0,\infty]$ with the following three properties:
(1)$d(x,x)=0$, for all $x\in X$.
(2)$d(x,y)=d(y,x)$, for all $x,y\in X$.
(3)$d(x,z)\leq d(x,y)+d(y,z)$, for all $x,y,z\in X$ with
$d(x,y),d(y,z)<\infty$.

An extended metric is an extended pseudometric $d$ with the additional
property that $d(x,y)\neq{0}$ whenever $x\neq{y}$. In this paper, we refer to
extended (pseudo)metrics simply as (psedo)metrics.

The filtrations of multi-GENEO we constructed are sublevelset filtrations. Let
$\gamma^{X}:X\rightarrow{\mathbb{R}^{n}}$ be a sublevelset filtration function
and $\textrm{hom}(\gamma^{X},\gamma^{Y})$ be the set of all continuous
functions $f:X\rightarrow{Y}$ such that
$\gamma^{X}(p)\geq{\gamma^{Y}\circ{f(p)}}$ for every $p\in{X}$. We can define
an n-parameter sublevelset filtration $S(\gamma)$ of any function
$\gamma^{X}$.

For a function $\gamma:X\rightarrow{\mathbb{R}^{n}}$, let

${\|}\gamma{\|}=\left\\{\begin{array}[]{rcl}\sup_{p\in{X}}{\|}\gamma(p){\|}_{\infty}&&{if\
X\ \neq\ \emptyset}\\\ 0&&{if\ X\ =\ \emptyset}\end{array}\right.$

Given $\gamma^{X}:X\rightarrow{\mathbb{R}^{n}}$ and
$\gamma^{Y}:Y\rightarrow{\mathbb{R}^{n}}$. Let

$d_{\infty}(\gamma^{X},\gamma^{Y})=\inf_{h\in{\mathcal{H}}}{\|}\gamma^{X}-\gamma^{Y}\circ{h}{\|}_{\infty}$

where $\mathcal{H}$ is the set of homeomorphisms from $X$ to $Y$.

For $i\geq{0}$, we say that a pseudometric $d$ is $i$-stable for any
topological spaces $X$, $Y$ and any functions
$\gamma^{X}:X\rightarrow{\mathbb{R}^{n}}$,
$\gamma^{Y}:Y\rightarrow{\mathbb{R}^{n}}$, we have

$d(H_{i}(\gamma^{X}),H_{i}(\gamma^{Y}))\leq{d_{\infty}(\gamma^{X},\gamma^{Y})}.$

Moreover, we say a pseudometric is stable if it is $i$-stable for all $i\geq
0$.

For $\epsilon\in\mathbb{R}$, let $\vec{\bm{\epsilon}}\in\mathbb{R}^{n}$ denote
the vector whose components are each $\epsilon$. Write
$(\cdotp)(\vec{\bm{\epsilon}}):M^{\mathbb{R}^{n}}\rightarrow
M^{\mathbb{R}^{n}}$ simply as $(\cdotp)(\epsilon)$. Define
$\tau_{\bm{a},\bm{a}+\vec{\bm{\epsilon}}}$ to be the $\epsilon$-$transition$
$morphism$ $\varphi_{M}^{\epsilon}:M_{\bm{a}}\rightarrow
M_{\bm{a}+\vec{\bm{\epsilon}}}$ for all $\bm{a}\in\mathbb{R}^{n}$. Simply
define $M(\epsilon)=M_{\bm{a}+\vec{\bm{\epsilon}}}$. Two n-modules M and N are
said to be $\epsilon$-$interleaved$ if there exist morphisms $f:M\rightarrow
N(\epsilon)$ and $g:N\rightarrow M(\epsilon)$ such that

$\displaystyle g(\epsilon)\circ f=\varphi_{M}^{2\epsilon},\ f(\epsilon)\circ
g=\varphi_{N}^{2\epsilon}.$

Here, we call $f$ and $g$ $\epsilon$-$interleaving$ $morphisms$.

Define the interleaving distanve $d_{I}:M\times N\rightarrow{[0,\infty)}$ as
follows:

$d_{I}(M,N)=\inf\\{\epsilon\in{[0,\infty)}~{}\mid~{}M\ and\ N\ are\
\epsilon\text{-}interleaved\\}.$

The above $d_{I}$ is the same as the definition in [4], and the author [4]
also gave the stability of $d_{I}$.

######  Theorem 2.2.

[4] $d_{I}$ is stable.

Multiparameter persistence landscape proposed by Oliver Vipond [23] is a
stable representation with respect to the interleaving distance and
persistence weighted Wasserstein distance. The author also provided examples
and statistical tests to demonstrate a range of potential applications which
is convenient to be used.

Let $M\in{M^{\mathbb{R}^{n}}}$. Consider the function
$\lambda:\mathbb{N}\times\mathbb{R}^{n}\rightarrow\mathbb{R}$.

$\lambda(k,\bm{x})=\sup\\{\varepsilon\geq{0}:\beta^{\bm{x-h},\bm{x+h}}\geq{k}\
for\ all\ \bm{h}\geq\bm{0}\ with\ {\|}\bm{h}{\|}_{\infty}\leq\epsilon\\}$

where
$\beta^{\bm{a},\bm{b}}=\textrm{dim}(\textrm{Im}(M_{a}\rightarrow{M_{b}}))$
gives the corresponding Betti number for $\bm{a}\leq\bm{b}$ is the rank
invariant of $M$. The multiparameter persistence landscape of $M$ is the set
of such function $\lambda(k,\bm{x})$ which describes the maximal radius over
which k features persist in every(positive) direction through $\bm{x}$ in the
parameter space.

Let $M$, $N$ be multiparameter persistence modules. The $p$-$landscape$
distance $d_{\lambda}^{(p)}(M,N)$ between $M$ and $N$ is defined to be,

$d_{\lambda}^{(p)}(M,N)={\|}\lambda(M)-\lambda(N){\|}_{p}$

where ${\|}\cdot{\|}$ is $L^{p}\text{-}norm$ for the $\mathbb{R}$-valued
functions on $\mathbb{N}\times\mathbb{R}^{n}$.

######  Theorem 2.3.

[23] Let $M$, $N\in{M^{\mathbb{R}^{n}}}$ be multiparameter persistence
modules, then the $\infty-landscape$ distance of the multiparameter
persistence landscapes is bounded by the interleaving distance $d_{I}$, i.e.

$d_{\lambda}^{(\infty)}(M,N)\leq d_{I}(M,N).$

##  3\. Stability and representation

In this section, we will show the stability and the bound estimates with
respect to both the interleaving distance and multiparameter persistence
landscape of multi-GENEO, multi-DGENEO and mix-GENEO persistence module. we
will also show the filtrations of multi-GENEO, multi-DGENEO and mix-GENEO on
discrete function space.

###  3.1. Stability for muli-GENEO

Consider $F$ as an element in the n copies of GENEO written as
$F=(F_{1},F_{2},...,F_{n})\in{\oplus_{i=1}^{n}GENEO}$.

######  Theorem 3.1.

Suppose that $X$ is a non-empty space, $\varphi_{k}\in{\varPhi}$ are the
bounded functions on X for $k=1,2$, and $F=(F^{1},F^{2},...,F^{n})$ is an
element in the n copies of GENEO. Let $V(F(\varphi_{k}))$ be the
multiparameter persistence module of multi-GENEO. The filtration of multi-
GENEO for $1\leq{i}\leq{n}$ can be obtained written as $F(\varphi_{k})$, and
then

$\displaystyle\sup_{F}d_{\lambda}^{(\infty)}(V(F(\varphi_{1}),V(F(\varphi_{2})))$
$\displaystyle\leq\sup_{F}d_{I}(V(F(\varphi_{1}),V(F(\varphi_{2})))$
$\displaystyle\leq\inf_{g\in{G}}{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}$
$\displaystyle\leq D_{\varPhi}(\varphi_{1},\varphi_{2})$

where $G$ is a subgroup of $\textrm{Homeo}_{\varPhi}(X)$.

###### Proof.

For every $F\in{\oplus_{i=1}^{n}GENEO}$, every $g\in{G}$ and $\varphi_{1}$,
$\varphi_{2}\in{\varPhi}$, we have that

$\displaystyle d_{I}(V(F(\varphi_{1}),F(\varphi_{2})))$ $\displaystyle=$
$\displaystyle
d_{I}(V(F^{1}(\varphi_{1}),...,F^{n}(\varphi_{1})),V(F^{1}(\varphi_{2}),...,F^{n}(\varphi_{2})))$
$\displaystyle=$ $\displaystyle
d_{I}(V(F^{1}(\varphi_{1}),...,F^{n}(\varphi_{1})),V(F^{1}(\varphi_{2})\circ
T(g),...,F^{n}(\varphi_{2})\circ T(g)))$ $\displaystyle=$ $\displaystyle
d_{I}(V(F^{1}(\varphi_{1}),...,F^{n}(\varphi_{1})),V(F^{1}(\varphi_{2}\circ
g),...,F^{n}(\varphi_{2}\circ g)))$ $\displaystyle\leq$ $\displaystyle
D_{\varPsi}(V(F^{1}(\varphi_{1}),...,F^{n}(\varphi_{1})),V(F^{1}(\varphi_{2}\circ
g),...,F^{n}(\varphi_{2}\circ g)))$ $\displaystyle=$
$\displaystyle{\|}(F^{1}(\varphi_{1}-\varphi_{2}\circ
g),...,F^{n}(\varphi_{1}-\varphi_{2}\circ g)){\|}_{\infty}$ $\displaystyle=$
$\displaystyle\mathop{max}\limits_{i}{\|}F^{i}(\varphi_{1}-\varphi_{2}\circ
g){\|}_{\infty}$ $\displaystyle\leq$
$\displaystyle{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}$
$\displaystyle=$ $\displaystyle D_{\varPhi}(\varphi_{1},\varphi_{2}\circ{g}).$

The second equality follows from the invariance of multiparameter persistent
homology under action of $\textrm{Homeo}_{\varPhi}(X)$, the third equality and
the seventh inequality follow from that each $F^{i}$ is a GENEO. The fourth
inequality follows from the stability of multiparameter persistent homology
while the sixth equality follows from the definition of the metric
${\|}\cdot{\|}_{\infty}$. Since $\varphi_{1}$, $\varphi_{2}$, $g$ are
arbitrary chosen and $F$ is an element in the n copies of GENEO, we get

$\displaystyle\sup_{F}d_{I}(V(F(\varphi_{1})),V(F(\varphi_{2})))\leq\inf_{g\in{G}}{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}\leq
D_{\varPhi}(\varphi_{1},\varphi_{2}).$

Furthermore, we have that

$\displaystyle\sup_{F}d_{\lambda}^{(\infty)}(V(F(\varphi_{1}),V(F(\varphi_{2})))$
$\displaystyle\leq\sup_{F}d_{I}(V(F(\varphi_{1}),V(F(\varphi_{2})))$
$\displaystyle\leq\inf_{g\in{G}}{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}$
$\displaystyle\leq D_{\varPhi}(\varphi_{1},\varphi_{2}).$

Then we obtain the stability of the $\infty$-landscape distance of the
multiparameter persistence landscapes.

∎

######  Lemma 3.2.

Suppose that $X$ is a non-empty space, $\varphi_{k}\in{\varPhi}$ are the
bounded functions on X for $k=1,2$, and
$L^{i}(\varphi)=F^{1,i}(\varphi)-F^{2,i}(\varphi)$, for which
$F^{1,i}(\varphi)$ and $F^{2,i}$ are two elments in the GENEOs, $i=1,...,n$.
Let $L=(L^{1},L^{2},...,L^{n})$ and $V(L(\varphi_{k}))$ be the multiparameter
persistence module of multi-DGENEO. The filtration of multi-DGENEO for
$1\leq{i}\leq{n}$ can be obtained written as $L^{i}(\varphi_{k})$, and then

$\displaystyle\sup_{L}d_{\lambda}^{(\infty)}(V(L(\varphi_{1}),V(L(\varphi_{2})))$
$\displaystyle\leq\sup_{L}d_{I}(V(L(\varphi_{1}),V(L(\varphi_{2})))$
$\displaystyle\leq
2\inf_{g\in{G}}{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}$
$\displaystyle\leq 2D_{\varPhi}(\varphi_{1},\varphi_{2})$

where $G$ is a subgroup of $Homeo_{\varPhi}(X)$.

###### Proof.

As the same of the calculation in Theorem 3.1, we have

$\displaystyle d_{I}(V(L(\varphi_{1}),L(\varphi_{2})))$ $\displaystyle\leq$
$\displaystyle
D_{\varPsi}(V(L^{1}(\varphi_{1}),...,L^{n}(\varphi_{1})),V(L^{1}(\varphi_{2}\circ
g),...,L^{n}(\varphi_{2}\circ g)))$ $\displaystyle=$
$\displaystyle\mathop{max}\limits_{i}{\|}L^{i}(\varphi_{1}-\varphi_{2}\circ
g){\|}_{\infty}$ $\displaystyle\leq$ $\displaystyle
2{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}$ $\displaystyle=$
$\displaystyle 2D_{\varPhi}(\varphi_{1},\varphi_{2}\circ{g}),$

Since $\varphi_{1}$, $\varphi_{2}$, $g$ are arbitrary chosen and $F$ is
considered as an element in the n copies of GENEO, the conclusion is obtained.
∎

######  Corollary 3.3.

Let $V(M(\varphi_{k}))$ be the multiparameter persistence module of mix-GENEO,
$k=1,2$. Then

$\displaystyle\sup_{M}d_{\lambda}^{(\infty)}(V(M(\varphi_{1}),V(M(\varphi_{2})))$
$\displaystyle\leq\sup_{M}d_{I}(V(M(\varphi_{1}),V(M(\varphi_{2})))$
$\displaystyle\leq
2\inf_{g\in{G}}{\|}\varphi_{1}-\varphi_{2}\circ{g}{\|}_{\infty}$
$\displaystyle\leq 2D_{\varPhi}(\varphi_{1},\varphi_{2})$

###### Proof.

As the same of the proof in Lemma 3.2, we can use the Definition 1.2 to get
the conclusion. ∎

###  3.2. Representation on discrete function spaces

Similar to the representation on discrete function spaces of 1-parameter GENEO
construction in [41], we can construct filtration of multi-GENEO. Let
$\\{\sigma^{i}\\}_{i=1}^{n}$ be a sequence of positive numbers and
$\\{\tau^{i}\\}_{i=1}^{n}$ be a sequence of real numbers, we consider the
$\\{g_{\tau^{i}}\\}_{i=1}^{n}$ for each
$g_{\tau^{i}}:\mathbb{R}\rightarrow\mathbb{R}$ is a 1-dimensional Gaussian
function with width $\sigma^{i}$ and center $\tau^{i}$,

$g_{\tau^{i}}(t):=\exp\left\\{{-\frac{t-\tau^{i}}{2({\sigma^{i}})^{2}}}\right\\}.$

For a positive integer k, we take the set S of $2k$-tuples
$(a_{1},\tau_{1},...,a_{k},\tau_{k})\in{\mathbb{R}^{2k}}$ for which
$\sum_{j=0}^{k}{a_{j}^{2}}=\sum_{j=0}^{k}{\tau_{j}^{2}}$. Let
$p=(p_{1},...,p_{n})$ and
$p_{i}=(a^{i}_{1},\tau^{i}_{1},...,a^{i}_{k},\tau^{i}_{k})\in{S}$ for
$i=1,...,n$. Define the function $G_{p}=(G_{p}^{1},...,G_{p}^{n})$ by

$G^{i}_{p}(x,y):=\sum_{j=0}^{k}{a_{j}g_{\tau^{i}_{j}}(\sqrt{x^{2}+y^{2}})}.$

Define the convolutional operator $F^{i}_{p}$ as follows. For each continuous
map $\varphi:\mathbb{R}^{2}\rightarrow\mathbb{R}$ with compact support,
$F^{i}_{p}(\varphi):\mathbb{R}^{2}\rightarrow\mathbb{R}$ is the continuous map
with compact support in the following form,

$F^{i}_{p}(\varphi)(x,y):=\int_{\mathbb{R}^{2}}\varphi(\alpha,\beta)\frac{G^{i}_{p}(x-\alpha,y-\beta)}{{\|}{G^{i}_{p}{\|}_{L^{1}}}}d\alpha
d\beta.$

Then the operator $F^{i}_{p}$ is a GENEO with respect to the group $I$ of
Euclidean plane isometries. One can see that
$\\{F^{i}_{p}(\varphi)\\}_{i=1}^{n}$ contributes to a filtration of multi-
GENEO. Then by Definition 1.2,
$L^{i}(\varphi)=F^{1,i}_{p}(\varphi)-F^{2,i}_{p}(\varphi)$ for $i=1,...,n$, we
could also get the filtration of multi-DGENEO and mix-GENEO.

##  4\. Experiments

In this section, we aim to demonstrate the effectiveness of the method. We
will use GENEO and DGENEO to extract multiparameter filtration from partial
and complete MNIST dataset, and we will use the tool RIVET and multiparameter
persistence landscape to represent the rank invariants of the multiparameter
persistence module.

RIVET is used to provide the corresponding results in biparameter persistence
module. RIVET software can compute and visualize three such kinds of
invariants, the Hilbert function, the bigraded Betti numbers, and the fibered
barcode. RIVET supports the fast computation of multigraded Betti numbers and
an interactive visualisation for 2-parameter persistence modules. RIVET
approximates multiparameter modules with a discretization in order to reduce
computational cost. These approximations can be taken to arbitrary accuracy
with respect to the interleaving distance. Details of the time and space
complexity of the algorithm may be found in [43, 44, 6].

Multiparameter persistence landscapes are stable with respect to the
interleaving distance and persistence weighted Wasserstein distance which can
be found in
https://github.com/OliverVipond/Multiparameter_Persistence_Landscapes/. Oliver
in [23] provided statistical tests to demonstrate their potential applications
of landscapes.

###  4.1. Generating bifiltrations on digital images

In this subsection, we will provide an algorithm to generate biparameter
filtrations on digital images, which is also suitable for n-parameter
filtrations. We give an example to show how generating biparameter filtration
on digital images.

There have been several methods to construct cubical complexes. In [45], the
authors represented the voxels as vertices of the cubical complexes, and then,
the authors in [46] used this method to build cubical complexes from an image
$\varphi:X\rightarrow\mathbb{R}$. In [7], the authors built lower-star and
upper-star filtrations of the Freudenthal triangulation on a grid. Inspired by
their contributions, we build simplicial complex from two images
$\varphi_{1}$, $\varphi_{2}\in{\mathcal{\varPhi}}$ by considering a unit
square as two 2-simplices.

Recall that such a grayscale image is a function
$\varphi:X\rightarrow\mathbb{R}$, where $X\subset\mathbb{Z}^{2}$ is typically
a rectangular subset of the discrete lattice

$X=\\{(m,n)~{}|~{}0\leq m\leq M,0\leq n\leq N\\}.$

A point $(m,n)\in X$ is called a pixel and the value $\varphi(x)\in\mathbb{R}$
is the grayscale value of $x$. The pixels $x\in X$ are the vertices (0-cells)
of the complex. If two vertices whose coordinates differ by one in a single
axis, then the edge with endpoints of the two vertices is one 1-simplex. If
four vertices form a unit square, then the edge with endpoints located in the
upper left and the lower right is also one 1-simplex. And then, the unit
square is divided into two 2-simplices. An example is given in Figure 1.

Figure 1. The solid dots represent vertices that have already appeared. There
is one edge with two endpoints in the left figure and there are two
2-simplices colored in yellow.

Suppose that two grayscale digital images $\varphi_{1}$ and $\varphi_{2}$ are
represented by the following two matrices

$\begin{bmatrix}7&5&3\\\ 8&6&9\\\ 1&4&2\end{bmatrix}\ \ \ \ \quad\text{and}\ \
\ \ \quad\begin{bmatrix}3&2&7\\\ 4&9&8\\\ 5&6&1\end{bmatrix}.$

Then we use nine letters from $a$ to $i$ to mark the nine vertices as follows,

$\begin{matrix}\bullet&\bullet&\bullet\\\ \bullet&\bullet&\bullet\\\
\bullet&\bullet&\bullet\end{matrix}\qquad\qquad\qquad\ \ \ \ \ \
\begin{matrix}g&h&i\\\ d&e&f\\\ a&b&c\end{matrix}$

By taking sublevelset filtration, a bifiltration could be shown in Figure 2.
The complexes in the position $(p,q)$ are generated by the pixels $x$ which
satisfying

$\varphi_{1}(x)\leq p\ \text{and}\ \varphi_{2}(x)\leq q.$

Notice that the 0-simplices in the position $(4,6)$ are $a$, $b$ and $c$, the
1-simplices are $ab$ and $bc$. The simplices $b$, $ab$ and $bc$ first appear
in the position. Call $(p,q)$ the birth coordinate of them.

Figure 2. Bifiltration Example. The figure records the birth coordinates of
vertices, edges and faces. The vertices and edges in the birth coordinates are
colored in orange, the faces in the birth coordinates are colored in yellow,
the rest are colored in blue.

Furthermore, we can use RIVET to visualize the biparameter persistence modules
of 0-dimensional and 1-dimensional homology in Figure 3. For details of basic
persistent homology, we refer to [43, 47, 48].

$\mathscr{B}_{H(X)^{0}}$

$\mathscr{B}_{H(X)^{1}}$

Figure 3. $\mathscr{B}_{H(X)^{0}}$ reprensents the $H_{0}$ multiparameter
persistence module, $\mathscr{B}_{H(X)^{1}}$ reprensents the $H_{0}$
multiparameter persistence module. One can see 1-loop in
$\mathscr{B}_{H(X)^{1}}$ only birth at the coordinate (9,8) and persist to the
coordinate (9,9).

Notice that the bifiltration is a one-critical multifiltration defined in [49]
since each cell of the multifilter complex has a unique critical coordinate.

Algorithm 1 Build bifiltration

$\mathcal{V}$, vertex ;

$\varphi$, image;

$M=(M^{1},M^{2})$, mix-GENEO;

$\mathscr{F}=(\mathscr{F}_{x},\mathscr{F}_{y})$, bifiltration at $(x,y)$;

$\psi_{1}=M^{1}(\varphi)$, $\psi_{2}=M^{2}(\varphi)$;

$\mathscr{F}\leftarrow$empty;

for $v\in{\mathcal{V}}$ do;

$(\mathscr{F}_{vx},\mathscr{F}_{vy})=(\psi_{1}(v),\psi_{2}(v))$;

$\mathscr{F}\leftarrow{\mathscr{F}\bigcup(\mathscr{F}_{vx},\mathscr{F}_{vy})}$;

end for

$\mathcal{E}\leftarrow$empty;

if $v_{i}$ is adjacent to $v_{j}$ then

$\mathcal{E}\leftarrow{\mathcal{E}\bigcup\\{e_{ij}\\}};$

end if

for $e_{ij}\in{\mathcal{E}}$ do;

$(\mathscr{F}_{e_{ij}x},\mathscr{F}_{e_{ij}y})=(max(\mathscr{F}_{v_{i}x},\mathscr{F}_{v_{j}x}),max(\mathscr{F}_{v_{i}y},\mathscr{F}_{v_{j}y}));$

$\mathscr{F}\leftarrow{\mathscr{F}\bigcup(\mathscr{F}_{e_{ij}x},\mathscr{F}_{e_{ij}y})}$;

end for

$\mathcal{F}\leftarrow$empty;

if four vertices $v_{i}$, $v_{j}$, $v_{k}$, $v_{s}$ form a square, and
$e_{ik}$ is a diagonal line in the square with a fixed direction then

$\mathcal{F}\leftarrow{\mathcal{F}\bigcup f_{ijk}};$

$\mathcal{F}\leftarrow{\mathcal{F}\bigcup f_{isk}};$

end if

for $f_{ijk},f_{isk}\in{\mathcal{F}}$ do;

$(\mathcal{F}_{f_{ijk}x},\mathcal{F}_{f_{ijk}y})=(max(\mathscr{F}_{v_{i}x},\mathscr{F}_{v_{j}x},\mathscr{F}_{v_{k}x},\mathscr{F}_{v_{s}x}$),

$max(\mathscr{F}_{v_{i}y},\mathscr{F}_{v_{j}y},\mathscr{F}_{v_{k}y},\mathscr{F}_{v_{s}y}));$

$(\mathcal{F}_{f_{isk}x},\mathcal{F}_{f_{isk}y})=(max(\mathscr{F}_{v_{i}x},\mathscr{F}_{v_{j}x},\mathscr{F}_{v_{k}x},\mathscr{F}_{v_{s}x}$),

$max(\mathscr{F}_{v_{i}y},\mathscr{F}_{v_{j}y},\mathscr{F}_{v_{k}y},\mathscr{F}_{v_{s}y}));$

$\mathscr{F}\leftarrow{\mathscr{F}\bigcup(\mathscr{F}_{f_{ijk}x},\mathscr{F}_{f_{ijk}y})\bigcup(\mathscr{F}_{f_{isk}x},\mathscr{F}_{f_{isk}y})}$;

end for

return $\mathscr{F}$.

Complexity We now explore the complexity of Algorithm 1. Notice that the
bifiltration we construct is all one-critical. One vertex is computed one time
if it is seemed as 0-simplex or a vertex of a higher simplex. A vertex is
common vertex of at most six 1-simplices and six 2-simplices. The algorithm
requires at most $O(13n)$ time for n vertices.

###  4.2. Example Computations and Machine Learning Applications

In this subsection, we will provide examples of calculating binary
classification and ten-classification. We consider two types of binary
classifications, one is studied by taking 500 samples from the MNIST dataset,
the other one is studied by using the complete MNIST dataset.

####  4.2.1. Binary classification

We will give examples of multi-GENEO, multi-DGENEO and mix-GENEO persistence
filtrations to validate the effectiveness. One can see that mix-GENEO performs
the best on partial MNIST dataset. This method can not only significantly
distinguish data with different topological information such as $\\{0,1\\}$,
but also with almost the same topological and geometric information such as
$\\{6,9\\}$.

Considering the images of the numbers $\\{0,1,3,6,9\\}$, we perform 500
samples for each number according to the order of appearance in the MNIST
dataset. To make the operation faster, we use the parameter bin in RIVET equal
to 10 which coarsen persistence module to obtain an algebraically simpler
module.

We plot the average persistence landscape $\lambda(k,\bm{x})$ for $k=1$ in the
parameter range $[0,255]^{2}$ of the five datasets and the complete MNIST
dataset with stepsize $s=10$ for the $H_{0}$-modules and $H_{1}$-modules (See
Figures 4, 5, 6 and 7). Here the first landscape $\lambda(k,\bm{x})$ detects
the parameter values for which the associated space has at least 1-homological
features together with the persistence of those features.

The figures show that the number $1$ in $H_{1}$ is significantly different
from numbers $s\in\\{0,3,6,9\\}$ since $1$ has different topological and
geometric information. It is worthy to note that although the topological and
geometric information of $6$ and $9$ are almost the same, we can also find
significant differences between them.

All landscapes of numbers from 0 to 9 can be found in our github code.

Figure 4. Multi-GENEO: Average Multiparameter Persistence Landscape for each
number in $\\{0,1,3,6,9\\}$ by taking 500 samples from MNIST dataset
($H_{0}$). Figure 5. Multi-GENEO: Average Multiparameter Persistence Landscape
for each number in $\\{0,1,3,6,9\\}$ by taking 500 samples from MNIST dataset
($H_{1}$). Figure 6. Multi-GENEO: Average Multiparameter Persistence Landscape
for each number in $\\{0,1,3,6,9\\}$ in MNIST dataset ($H_{0}$). Figure 7.
Multi-GENEO: Average Multiparameter Persistence Landscape for each number in
$\\{0,1,3,6,9\\}$ in MNIST dataset ($H_{1}$).

We use machine learning algorithms with landscape functions as a collection of
features for a data set to learn non-linear relationships in our data, and
then apply Principle Components Analysis (PCA) to the collection of $H_{0}$,
$H_{1}$ and $(H_{0},H_{1})$ landscape vectors $\lambda(1,\bm{x})$ to reduce
parameter dimension. The PCA projections make our method better to verify the
topological and geometric information of digital dataset.

Suppose that a digital image is the bounded function $\varphi$. We select five
GENEOs, $G_{0}$, $G_{1}$, $G_{2}$, $G_{3}$ and $G_{4}$, to get bifiltration
$\\{F_{p}^{i}(\varphi)\\}_{i=1}^{2}$. Notice that $G_{0}$ can be seemed as a
Gaussian blur, $G_{1}-G_{2}$ and $G_{3}-G_{4}$ which are called DOG can be
seemed as Laplace operators approximately. Since identity $I$ is also a GENEO,
we could build multi-GENEO filtration by $G_{0}$ and $I$ acting on $\varphi$.
Multi-DGENEO fitration is built by $(G_{3}-G_{4})(\varphi)$ and
$(G_{1}-G_{2})(\varphi)$, and mix-GENEO filtration is built by
$G_{0}(\varphi)$ and $(G_{3}-G_{4})(\varphi)$. To make the parameters in RIVET
and persistent landscape consistency, we resize the value of
$F_{p}^{i}(\varphi)$ into $[0,255]$.

Dataset The MNIST dataset is a classic dataset in the field of machine
learning, consisting of 60000 training samples and 10000 test samples, each of
which is a 28 $\times$ 28 pixel grayscale handwritten digital image and
represents a number from 0 to 9.

Results We obtain the accuracies of binary classifications of 0 and 1 , 1 and
3 , 6 and 9 by persistent landscapes of multi-GENEO, multi-DGENEO and mix-
GENEO. For 500 samples of each number, we perform 100 trials and average the
classification accuracies. For the complete MNIST dataset, we use their train
and test datasets for training and testing, respectively. More details of the
results are provided in Table 1 and Table 2. The accuracy of binary
classification of 0 and 1, which have different topological information, can
achieve $99.5\%$. The accuracy of binary classification of 6 and 9, which have
almost the same topological and geometric information, can achieve $94.1\%$.
The accuracy of binary classification of 1 and 3, which have different
geometric information, can achieve $99.1\%$. Therefore, our methods can
significantly distinguish not only the ones with different topological
information but also the ones with almost the same topological and geometric
information. In our three methods, multi-GENEO is suitable for $H_{0}$, multi-
DGENEO is suitable for $H_{1}$ and mix-GENEO performs well for both $H_{0}$
and $H_{1}$. In general, mix-GENEO performs the best. In particular, $H_{1}$
persistence diagrams of the numbers $\\{1,3\\}$ are almost empty and then it
is unable to distinguish $1$ and $3$ by one parameter persistence homology,
but our methods make sense.

| $H_{0}$ | $H_{1}$ | $H_{0}+H_{1}$
---|---|---|---
L | PL | PS | L | PL | PS | L | PL | PS
$0vs1$ | mul-G | 97.9 | 98.1 | 98.6 | 56.6 | 55.7 | 57.4 | 97.3 | 96.5 | 97.6
mul-D | 50.1 | 48.3 | 48.1 | 92 | 97.2 | 90.4 | 83.9 | 85.7 | 86.5
mix-G | 97.5 | 96.8 | 99.1 | 98.5 | 98.1 | 98.7 | 98.2 | 99.1 | 99.3
$1vs3$ | mul-G | 68.9 | 68.5 | 70.4 | 63.4 | 63.9 | 66.2 | 71.9 | 73.2 | 74.8
mul-D | 50.3 | 50.1 | 50.3 | 89.6 | 89.8 | 85.4 | 81.4 | 81.8 | 82.5
mix-G | 93.3 | 92.6 | 94.3 | 95.7 | 95.7 | 96.8 | 95.7 | 96.9 | 97.6
$6vs9$ | mul-G | 64.8 | 61.2 | 69.8 | 52.4 | 56.5 | 51.5 | 63.5 | 68.2 | 69.7
mul-D | 50 | 48.6 | 48.4 | 69.2 | 85.3 | 86.4 | 76.6 | 86.2 | 85.9
mix-G | 76.6 | 79.7 | 75.7 | 73.3 | 80.4 | 82.1 | 85.8 | 87.3 | 88.7
Table 1. Binary classification of multi-GENEO, multi-DGENEO and mix-GENEO on $1vs0$,$2vs3$ and $6vs9$ with each number of 500 examples using LDA, PCA+LDA, PCA+SVM. In the first row, the following abbreviations are used: L=LDA, PL=PCA+LDA, PS=PCA+SVM. | $H_{0}$ | $H_{1}$ | $H_{0}+H_{1}$
---|---|---|---
L | PL | PS | L | PL | PS | L | PL | PS
$0vs1$ | mul-G | 99.1 | 94.9 | 98.8 | 63.3 | 63 | 63.9 | 99.1 | 97.3 | 98.7
mul-D | 57.7 | 60.3 | 60.2 | 87.8 | 95.6 | 87.5 | 88.8 | 95.6 | 87.8
mix-G | 99.1 | 98.3 | 99.3 | 96.2 | 96.2 | 96.2 | 99.5 | 99.2 | 99.4
$1vs3$ | mul-G | 72.6 | 70 | 72.3 | 68.2 | 67.4 | 67.6 | 69.4 | 66.9 | 70.3
mul-D | 59.7 | 58.4 | 59.4 | 73.4 | 74.5 | 73.6 | 63.7 | 68.3 | 66.5
mix-G | 95.8 | 95.1 | 96.2 | 84.3 | 82.6 | 84.2 | 98.7 | 98.3 | 99.1
$6vs9$ | mul-G | 67.8 | 67.1 | 68.6 | 53.9 | 52.9 | 52.4 | 78.5 | 75.7 | 79
mul-D | 52.1 | 51.7 | 54.3 | 67.1 | 68 | 67 | 75.5 | 76.7 | 75.2
mix-G | 82.9 | 76.8 | 83.9 | 68.3 | 66.2 | 66.6 | 93.4 | 91.7 | 94.1
Table 2. Binary classification results for MNIST dataset using LDA, PCA+LDA,
PCA+SVM. In the first row, the following abbreviations are used: L=LDA,
PL=PCA+LDA, PS=PCA+SVM.

####  4.2.2. Ten-classification

For the completeness of the experiment, we also carry out experiments on the
entire MNIST dataset by our three methods.

Results The accuracies of ten-classification are shown in Table 3. One can see
mix-GENEO performs best, it can effectively identify ten classes and achieve
an accuracy of $78.8\%$.

| $H_{0}$ | $H_{1}$ | $H_{0}+H_{1}$
---|---|---|---
| L | PL | PS | L | PL | PS | L | PL | PS
mul-G | 39.4 | 39.6 | 39.7 | 19.1 | 19.1 | 19.3 | 42.7 | 42.9 | 43.4
mul-D | 9.8 | 14.2 | 14.2 | 30.3 | 32.1 | 29.4 | 9.8 | 33.6 | 31
mix-G | 9.8 | 64.4 | 67.8 | 19.2 | 46.7 | 50.6 | 11.3 | 73 | 78.8
Table 3. Ten-classification results for MNIST dataset using LDA, PCA+LDA,
PCA+SVM. In the first row, the following abbreviations are used: L=LDA,
PL=PCA+LDA, PS=PCA+SVM.

##  5\. Conclusion and future work

In this paper, we introduce three multiparameter persistence filtrations
called multi-GENEO, multi-DGENEO and mix-GENEO which can be chosen flexible.
Moreover, we show the stability of both interleaving distance and
multiparameter persistence landscape of multi-GENEO persistence module. We
also provide estimations of upper bound for multi-DGENEO and mix-GENEO
persistence module with respect to pseudometrics. After giving an algorithm to
build the bifiltrations on digital images, the experiments we conduct
demonstrate that the three methods can significantly distinguish not only the
ones with different topological information but also the ones with almost the
same topological and geometric information.

In the future work, we would like to develop our methods in the following two
aspects. On the one hand, we plan to optimatize our methods to get better
results. For instance, we would obtain multiparameter filtrations by higher
dimensional sublevelset functions or by selecting suitable operators in
another way. On the other hand, we plan to apply our methods to other fields
or problems, for instance, integrating features into deep learning and medical
research.

## References

  * [1] Andrew Blumberg and Michael Lesnick.  Universality of the homotopy interleaving distance.  Transactions of the American Mathematical Society, 376(12):8269–8307, 2023.
  * [2] William Crawley-Boevey.  Decomposition of pointwise finite-dimensional persistence modules.  Journal of Algebra and its Applications, 14(05):1550066, 2015.
  * [3] Ulrich Bauer, Magnus B Botnan, Steffen Oppermann, and Johan Steen.  Cotorsion torsion triples and the representation theory of filtered hierarchical clustering.  Advances in Mathematics, 369:107171, 2020.
  * [4] Michael Lesnick.  The theory of the interleaving distance on multidimensional persistence modules.  Foundations of Computational Mathematics, 15(3):613–650, 2015.
  * [5] David Cohen-Steiner, Herbert Edelsbrunner, and Dmitriy Morozov.  Vines and vineyards by updating persistence in linear time.  In Proceedings of the twenty-second annual symposium on Computational geometry, pages 119–126, 2006.
  * [6] The RIVET Developers.  Rivet.  https://github.com/rivetTDA/rivet/.
  * [7] D. Morozov.  Dionysus 2.  https://mrzv.org/software/dionysus2/.
  * [8] Raúl Rabadán and Andrew J Blumberg.  Topological data analysis for genomics and evolution: topology in biology.  Cambridge University Press, 2019.
  * [9] Barbara Giunti and Janis Lazovskis.  Tda-applications (an online database of papers on applications of tda outside of math), 2021.
  * [10] Rickard Brüel Gabrielsson, Bradley J Nelson, Anjan Dwaraknath, and Primoz Skraba.  A topology layer for machine learning.  In International Conference on Artificial Intelligence and Statistics, pages 1553–1563. PMLR, 2020.
  * [11] Christoph Hofer, Roland Kwitt, Marc Niethammer, and Mandar Dixit.  Connectivity-optimized representation learning via persistent homology.  In International conference on machine learning, pages 2751–2760. PMLR, 2019.
  * [12] Christoph Hofer, Roland Kwitt, Marc Niethammer, and Andreas Uhl.  Deep learning with topological signatures.  Advances in neural information processing systems, 30, 2017.
  * [13] Ephy R Love, Benjamin Filippenko, Vasileios Maroulas, and Gunnar Carlsson.  Topological convolutional layers for deep learning.  Journal of Machine Learning Research, 24(59):1–35, 2023.
  * [14] Olympio Hacquard, Krishnakumar Balasubramanian, Gilles Blanchard, Clément Levrard, and Wolfgang Polonik.  Topologically penalized regression on manifolds.  The Journal of Machine Learning Research, 23(1):7233–7271, 2022\.
  * [15] Leonid Polterovich, Daniel Rosen, Karina Samvelyan, and Jun Zhang.  Topological persistence in geometry and analysis, volume 74.  American Mathematical Soc., 2020.
  * [16] Steve Y Oudot.  Persistence theory: from quiver representations to data analysis, volume 209.  American Mathematical Soc., 2017.
  * [17] Henry Adams, Manuchehr Aminian, Elin Farnell, Michael Kirby, Joshua Mirth, Rachel Neville, Chris Peterson, and Clayton Shonkwiler.  A fractal dimension for measures via persistent homology.  In Topological Data Analysis: The Abel Symposium 2018, pages 1–31. Springer, 2020.
  * [18] Benjamin Schweinhart.  Fractal dimension and the persistent homology of random geometric complexes.  Advances in Mathematics, 372:107291, 2020.
  * [19] Omer Bobrowski and Matthew Kahle.  Topology of random geometric complexes: a survey.  Journal of applied and Computational Topology, 1:331–364, 2018\.
  * [20] Benjamin Schweinhart.  Persistent homology and the upper box dimension.  Discrete & Computational Geometry, 65(2):331–364, 2021.
  * [21] Gunnar Carlsson and Afra Zomorodian.  The theory of multidimensional persistence.  In Proceedings of the twenty-third annual symposium on Computational geometry, pages 184–193, 2007.
  * [22] Peter Bubenik.  Statistical topological data analysis using persistence landscapes.  J. Mach. Learn. Res., 16(1):77–102, 2015.
  * [23] Oliver Vipond.  Multiparameter persistence landscapes.  The Journal of Machine Learning Research, 21(1):2262–2299, 2020\.
  * [24] Henry Adams, Tegan Emerson, Michael Kirby, Rachel Neville, Chris Peterson, Patrick Shipman, Sofya Chepushtanova, Eric Hanson, Francis Motta, and Lori Ziegelmeier.  Persistence images: A stable vector representation of persistent homology.  Journal of Machine Learning Research, 18, 2017.
  * [25] Mathieu Carriere and Andrew Blumberg.  Multiparameter persistence image for topological machine learning.  Advances in Neural Information Processing Systems, 33:22432–22444, 2020.
  * [26] Peter Bubenik, Michael Hull, Dhruv Patel, and Benjamin Whittle.  Persistent homology detects curvature.  Inverse Problems, 36(2):025008, 2020.
  * [27] Henry Adams and Michael Moy.  Topology applied to machine learning: From global to local.  Frontiers in Artificial Intelligence, 4:668302, 2021.
  * [28] Gunnar Carlsson, Tigran Ishkhanov, Vin De Silva, and Afra Zomorodian.  On the local behavior of spaces of natural images.  International journal of computer vision, 76:1–12, 2008.
  * [29] Frédéric Chazal, Leonidas J Guibas, Steve Y Oudot, and Primoz Skraba.  Scalar field analysis over point cloud data.  Discrete & Computational Geometry, 46(4):743–775, 2011.
  * [30] Omer Bobrowski, Sayan Mukherjee, and Jonathan E Taylor.  Topological consistency via kernel estimation.  2017\.
  * [31] Jeff M. Phillips, Bei Wang, and Yan Zheng.  Geometric inference on kernel density estimates.  LIPICS - Leibniz International Proceedings in Informatics, pages 857–871, 2015.
  * [32] Hirokazu Anai, Frédéric Chazal, Marc Glisse, Yuichi Ike, Hiroya Inakoshi, Raphaël Tinarrage, and Yuhei Umeda.  Dtm-based filtrations.  In Topological Data Analysis: The Abel Symposium 2018, pages 33–66. Springer, 2020.
  * [33] Mickaël Buchet, Frédéric Chazal, Steve Y Oudot, and Donald R Sheehy.  Efficient and robust persistent homology for measures.  Computational Geometry, 58:70–96, 2016.
  * [34] Frédéric Chazal, David Cohen-Steiner, and Quentin Mérigot.  Geometric inference for probability measures.  Foundations of Computational Mathematics, 11:733–751, 2011.
  * [35] Leonidas J Guibas, Quentin Mérigot, and Dmitriy Morozov.  Witnessed k-distance.  In Proceedings of the twenty-seventh annual symposium on Computational geometry, pages 57–64, 2011.
  * [36] Andrew J Blumberg, Itamar Gal, Michael A Mandell, and Matthew Pancia.  Robust statistics, hypothesis testing, and confidence intervals for persistent homology on metric measure spaces.  Foundations of Computational Mathematics, 14:745–789, 2014.
  * [37] René Corbet, Michael Kerber, Michael Lesnick, and Georg Osang.  Computing the multicover bifiltration.  Discrete & Computational Geometry, pages 1–30, 2023.
  * [38] Herbert Edelsbrunner and Georg Osang.  The multi-cover persistence of euclidean balls.  Discrete & Computational Geometry, 65:1296–1313, 2021.
  * [39] Andrew J Blumberg and Michael Lesnick.  Stability of 2-parameter persistent homology.  Foundations of Computational Mathematics, pages 1–43, 2022.
  * [40] Yu-Min Chung, Sarah Day, and Chuan-Shen Hu.  A multi-parameter persistence framework for mathematical morphology.  Scientific reports, 12(1):6427, 2022.
  * [41] Mattia G Bergomi, Patrizio Frosini, Daniela Giorgi, and Nicola Quercioli.  Towards a topological–geometrical theory of group equivariant non-expansive operators for data analysis and machine learning.  Nature Machine Intelligence, 1(9):423–433, 2019.
  * [42] Allen Hatcher.  Algebraic topology.  second order equations with nonnegative characteristic form, 2002\.
  * [43] Afra Zomorodian and Gunnar Carlsson.  Computing persistent homology.  In Proceedings of the twentieth annual symposium on Computational geometry, pages 347–356, 2004.
  * [44] Michael Lesnick and Matthew Wright.  Interactive visualization of 2-d persistence modules.  arXiv preprint arXiv:1512.00180, 2015.
  * [45] Vanessa Robins, Peter John Wood, and Adrian P Sheppard.  Theory and algorithms for constructing discrete morse complexes from grayscale digital images.  IEEE Transactions on pattern analysis and machine intelligence, 33(8):1646–1658, 2011.
  * [46] Bea Bleile, Adélie Garin, Teresa Heiss, Kelly Maggs, and Vanessa Robins.  The persistent homology of dual digital image constructions.  In Research in Computational Topology 2, pages 1–26. Springer, 2022\.
  * [47] Afra J Zomorodian.  Topology for computing, volume 16.  Cambridge university press, 2005.
  * [48] Magnus Bakke Botnan and Michael Lesnick.  An introduction to multiparameter persistence.  arXiv preprint arXiv:2203.14289, 2022.
  * [49] Gunnar Carlsson, Gurjeet Singh, and Afra Zomorodian.  Computing multidimensional persistence.  In Algorithms and Computation: 20th International Symposium, ISAAC 2009, Honolulu, Hawaii, USA, December 16-18, 2009. Proceedings 20, pages 730–739. Springer, 2009.


# On-line Remote ekg as a Web Service

Augusto Ciuffoletti
e.mail<EMAIL_ADDRESS>Dept. of Computer Science, Universitá di
Pisa L.go B. Pontecorvo - 56122 Pisa ORCID:0000-0002-9734-2044

###### Abstract

A 3-leads, non-diagnostic ekg has a role in emergency rescue and homecare. In
this paper we introduce the design and a prototype of a service, provided to a
doctor and a patient, for the on-line remote visualization of patient’s
3-leads ekg. The architecture is based on the http protocol, using commercial
off-the-shelf devices to implement the sensor on patient’s side, a browser on
a laptop Personal Computer (PC) on the doctor’s side as viewer, and a cloud
container to connect the two using Websockets. A prototype is built to
evaluate signal latency, power consumption of the patient side device, and the
quality of the rendering. After some experiments, latency is measured below
$1sec$, and power consumption is estimated in the 2A*3.3V range; visualization
is comparable to commercial, non-diagnostic products. The prototype patient
device is portable, and can be operated using rechargeable battery packs. Its
cost is below 100$, and all the required equipment is commercially available.
The architecture is ready for on field evaluation, and we indicate how to
improve power consumption while reducing cost.
Keywords: Non-diagnostic ekg; Websocket; Cloud PaaS; EKG acquisition; Arduino;
Raspberry Pi

##  1 Introduction

There are cases where heartbeat monitoring would improve doctor’s assistance,
but the patient and the physician are not in the same room. What we need in
that case is to transmit the electrocardiogram (EKG) from patient’s heart to
doctor’s display: this is an on-line remote EKG. The number of use cases for
remote EKG is long and includes assistance in rural areas, emergency rescue,
automated processing and alert, long term recording etc. In most cases a
diagnostic 12-leads measurement is inappropriate, since it requires time and a
specific training to be prepared, while a 3-leads, non-diagnostic EKG is
fitting. Two stories illustrate its possible utilization.

One is that of an ambulance with paramedical personnel rescuing a injured
person after a car accident. It is likely that the location is covered by a
ground or satellite broadband provider, and that an Access Point (AP) is
available, e.g. by tethering a smartphone. With an on-line remote EKG service
the 3-leads trace is delivered to the medical staff in the hospital, that
defines the emergency level.

A different homecare story is that of a cardiac patient that is periodically
contacted by the family doctor to check general conditions. In that case the
physicist uses a remote EKG service to receive the trace without leaving its
desk during a phone call, possibly interacting with the patient about
electrodes position or body posture.

This paper addresses the long distance delivery of a non-diagnostic EKG, using
cloud facilities integrated with personal devices.

##  2 Background and previous works

The electrical functionality of the heart is a fundamental diagnostic
information for the medical staff. Its recording dates back to the end of the
19th century. From that time, research addressed first the amplification of
the signal, whose amplitude is in the $\mu V$ range, and next the design of
filters that remove unwanted components (like powerline hum, or signals from
other muscular activity). The current state of the art on EKG filtering is in
[11].

Computers come into play in connection with filters, and today they extract
the fundamental features of the EKG signal. In [10] the authors introduce an
approach and evaluate the computing resources needed, with a survey of other
research results in the field. A recent survey is also in [14].

With the advent of the Web, storage and transport of EKG data emerge as a
practical perspective. In [13] the author investigates its use for human
identification, and incidentally creates a database of EKGs, that later became
a precious resource for many. The availability of historical data is relevant
for medical research, daily health-care use, and educational purposes.

The primary concern with the transport and storage of an EKG is about its
confidential nature. In [18] the authors survey a number of cryptographic
systems that can be used to secure the transport and the storage of biosensor
data. The paper assumes the presence of a number of distinct sensing devices
aggregated in a Medical Sensor Network (MSN).

The ubiquitous presence of wireless networks justifies the realization of
portable (or even wearable[2]) EKG devices that forward the trace to a nearby
PC or tablet. When many such devices are aggregated in a network, we speak of
a Body (or Personal) Data Network (BAN). In 2001, [12] introduces the basic
principles and concepts, and, ten years later, a survey in [3] lists nine BAN
projects, 5 of which include EKG. Detailed designs of BAN devices are found,
for instance, in ([19]), [17], [7]. However they are appropriate for local
area networks, confined in a room or a building.

The availability of cloud servers allows overcoming such limits. In [21] the
authors propose a cloud-based infrastructure: the physician and the patient
are clients of a Web server that processes the EKG, evaluates its quality, and
provides parameter extraction and visualization. The server is in the Amazon
Web Services (AWS) cloud, while the client uses personal devices that interact
with the server using Secure Hypertext Transfer Protocol (HTTPS).

To facilitate the diffusion of the remote EKG service, we need to address the
cost of the hardware devices, which also meets the needs of developing
countries and of rural areas, as in [20] and [8].

In this paper we show how a cloud infrastructure can be used to relay the EKG
from the patient’s device to the doctor’s laptop: in a nutshell, we actualize
the ideas in [21], but with the target of [12]. This result is obtained using
low cost devices and established protocols and infrastructures, without
overlooking security aspects.

##  3 An open service for remote ekg

The design guidelines authorize several alternatives, and we propose the
architecture outlined in figure 1.

Figure 1: Remote ekg service infrastructure layout

The right box represents the devices on patient’s side. We observe the analog
stage with the operational amplifier and the filters, the processor that
encodes the analog signal on a serial line, a Hypertext Transfer Protocol
(HTTP) user agent that further processes the signal and manages the Websocket.
This stage is connected to the Internet with a wireless link, and reaches the
cloud server.

The devices on doctor’s premises are in the left box: a wireless AP, and the
smart device (a PC or smartphone) with a HTTP client that manages the
Websocket, and displays the EKG in a convenient way.

From the security perspective, we highlight that only the routers inside the
APs need to expose a public Internet Protocol (IP) address, therefore both
patient’s and doctor’s devices can be protected by a Network Address
Translation (NAT) and other security techniques implemented on the router,
according with common practice.

The cloud server needs a limited computational capacity, and it may exist just
for the time needed by the physicist to examine the EKG: it is the ideal
candidate for a microservice instance [15]. A Platform as a Service (PaaS)
provider can create a new instance of a microservice in a few seconds, make it
accessible in the Internet with a unique Uniform Resource Locator (URL), and
destroy it after use.

The cloud provider is responsible for the security of the server, whose URL is
known only to the partners. However, server life span is so limited that the
potential intruder has little time to identify and attack it, especially if
communication is encrypted, for instance with Transport Level Security (TLS).

The adoption of HTTP as the transport protocol is a cornerstone, since cloud
services, that are reachable only using this protocol, provide a flexible,
reliable and secure infrastructure for EKG transmission. Although real time
aspects of the remote EKG are incompatible with the http request/response
mechanism, the WebSockets [9] have been recently introduced to allow
unsolicited communication between client and server. WebSockets have a
behavior similar to Transport Control Protocol (TCP) channels, but are
encapsulated in a HTTP session.

In our architecture, the PaaS server acts as a relay point: two separate
WebSockets are created to this end, one with the patient, another with the
doctor, and the data are trasparently transferred from the former to the
latter. This configuration turns out to be easily scalable, with costs that at
the prototype stage can be null, and with excellent security features.

In addition, with HTTP the doctor uses the browser of his laptop to view the
EKG, without the need to install new software. The architecture is therefore
agnostic of the operating system installed on doctor’s premises.

To have similar benefits also on patient’s side, we introduce the following
principles, that, in this paper, are synthesized in the Open Source, Low Cost,
COTS (OpLoC) acronym:

  * •

Open Source: components and protocols are exhaustively documented and freely
reproducible;

  * •

Low Cost: the less expensive option is always preferable. In particular, if a
functionality is already available, it is not re-implemented;

  * •

Commercial Devices: devices must be available on retail (aka Commercial Off-
the-shelf (COTS)).

Put together, the three principles ensure that an implementation is easily
reproducible, and makes a solid ground for further investigation. In addition,
low cost ensures that its applicability is not selective by scale and wealth.

##  4 A prototype for a remote EKG service

This section describes a concrete implementation of the above abstract design.
In figure 3 we see the device on patient’s side, while in figure 4 we see the
display in the browser. Together with the OpLoC hardware components, the
prototype contains also three ad-hoc software products, whose code is publicly
available on the Bitbucket and GitHub platforms:

  * •

the analog-to-serial encoder in the Micro-Controller Unit (MCU) [6]

  * •

the http patient-side User Agent (UA) [4]

  * •

the doctor’s page and the Websocket manager in the cloud container [5]

### The sensor: amplifier and filter

The prototype uses the popular Olimex EMG-EKG sensor board [1]: its size is
$6\times 8\times 3cm$, and the cost is around 30$. It is usually sold with EKG
pads.

The board integrates a 3rd order filter at 40Hz and two high-pass filters to
remove high frequencies and baseline drift.

A single board acquires a non-diagnostic 3-lead EKG through a 3-pole jack. The
design of the board allows to stack up to 6 boards for a diagnostic 12-lead
EKG, but our prototype uses just one of them.

### The MCU: Analog Digital Converter (ADC) and serial encoder

The MCU is a popular Arduino Uno board, whose cost is around 10$. The
connection with the Arduino-compliant Olimex board described above is obtained
by stacking the two boards: the result is mechanically stable and sufficiently
compact. The analog output of the sensor board is converted into a 10-bits
integer by the ADC embedded in the MCU.

    void Timer2OveflowISR( ) {
      int i;
      unsigned long int t;
      if ( full [b] ) {
        Serial.println (fail);
        return;
      }
      for (int Channel = 0 ; Channel < 6 ; Channel++) {
        Data [ Channel ] [ b ] = analogRead (Channel );
      }
      full[b]= true;
      b=b1;
    }

Table 1: The function triggered every $4msecs$ on the mcu

To have an accurate timing, needed for filtering and analysis purposes, a
hardware interrupt triggers data fetch with a frequency of 250Hz [16]. The
code snippet for the interrupt handler is in table 1: note that it is prepared
to collect six analog data for a standard 12-lead EKG, but only one is used in
the prototype. The main loop waits for the buffer to be readable, and encodes
a space-separated line:

    <h>:<m>:<s>.<msec> <v1> <v2> <v3> <v4> <v5> <v6>\n

The first field is a $1msec$ resolution timestamp, and the other six fields,
of which only one is used in the prototype, are integers in the interval
$[0-1023]$ that correspond to a sample. The encoding does not introduce any
rounding or information loss, and its redundancy is functional to data
integrity. The timestamp is not as accurate as the sampling period: it is used
only for rendering.

To avoid interference between data fetch and encoding, a two positions buffer
is introduced, with access regulated by a semaphore.

It is worth saying that Olimex provides a different driver ([1]), but its
accuracy is not sufficient for the task.

###  MCU to Single Board Computer (SBC) interface

The two units communicate using their Universal Asynchronous Receiver-
Transmitter (UART)s, with a baudrate of 115200. Since the maximum length of a
line is 44 8-bit characters, and 250 lines are produced each second, the MCU
outputs 11000 bytes per second. Since one stop bit per byte is added, this
requires a baudrate of 99000, which is consistent with UARTs one.

### Patient side user-agent

Since the sensor unit has tight timing requirements and limited capabilities,
it is more appropriate to decouple the HTTP UA on a distinct platform. The
Raspberry Pi 3 used in the prototype is a SBC that uses production-level
libraries to implement the patient-side user agent. Its cost is about 25$.

The UA is written in Python, and opens a Websocket connection with a container
hosted in the Heroku cloud whose URL is:

    http://<container fqdn>/in/<id>

where `<id>` is a unique id for the device. When the Websocket is opened, the
UA starts sending EKG samples encoded as JavaScript Object Notation (JSON)
objects with two fields: the timestamp, and one value.

###  4.1 The server

The server is a container in the Heroku cloud: for the sake of simplicity, the
prototype envisions one single container hosting several concurrent EKGs
instead of a short lived micro-service for each of them.

The server is implemented using the Python/Flask micro-framework and the
gunicorn Web Server Gateway Interface (WSGI) HTTP server. It provides three
families of application routes:

  * •

/ which returns a presentation page,

  * •

/in/<id> used by a patient’s UA to open a Websocket, as explained above,

  * •

/<id> used by doctor’s UA to open a Websocket and receive the EKG,

  * •

/out/<id> used as endpoint for doctor’s Websocket

The server waits for a patient’s UA request on the `/in/<id>` route, upgrades
the session to a Websocket and prepares to receive a request from doctor’s
side with the same `<id>`: until then, all received data are lost. If doctor’s
request arrives first, the server returns a negative reply.

When the server receives doctor’s request on an `/<id>` path matching
patient’s UA, it delivers a page containing the JavaScript code to open the
WebSocket on `/out/<id>` and displays the EKG.

Since each patient UA has a different identifier, a single server may host
several EKG at the same time, each of which is received by only one doctor UA.

The security mechanisms announced in the 3 are not fully implemented: namely,
in the prototype the server is persistent and communications use plain HTTP.
The implementation of such features appears to be quite straightforward, and
it is a matter for the improvement of the product.

### The display UA

The UA on doctor’s side is a web browser running on a laptop. The doctor opens
the HTTP session using a URL with the Fully Qualified Domain Name (FQDN) of
the server and the `/<id>` route. If the corresponding patient is already
connected, the response contains a resource composed of the familiar EKG
canvas and of a JavaScript application that opens a WebSocket on the
`/in/<id>` route. From it the EKG is received and the Chart.js library is used
to display it.

The JavaScript application filters out the $50Hz$ powerline noise with a
moving average of five values and finds R peaks to compute heartbeat
frequency. This demonstrates the possibility of additional filtering and
feature extraction on doctor’s side, and provides a readable result.

Figure 2:  Two EKGs: one after exercise, using elastic bands on left and right
wrists and right ankle, another at rest, using adhesive electrodes on chest.
They have been captured as screenshots form doctor’s browser using the
prototype described in the paper.

##  5 Results and future work

A relevant parameter is the latency between the production of the signal and
its visualization: it has been measured as lower than one second. Autonomy
with battery operation has an impact on practical usability as well: using a
$2200mAh$ rechargeable power bank an autonomy of 72 minutes has been measured,
and the prototype operates with the 3.3V power supply provided by the
Raspberry.

The Raspberry Pi 3 over-kills its task, and it is a candidate for replacement
with a more focused OpLoC device. Being in charge of implementing the
patient’s WebSocket, it uses 50% of the space, 90% of the power (nearly 1500
mA), and 30% of the cost.

Figure 3: The patient-side prototype device, with three elastic bands, the
$2200mAh$ battery, the Arduino/Olimex ekg boards and the Raspberry Pi 3

The interface on doctor’s side (see figure 4) should be extended with a switch
to select filtering, feature extraction and evaluation tools, and the
possibility to save, and replay, the trace. A Smartphone interface is close to
reach, but the limited size of the display makes the EKG significantly less
readable.

Figure 4: The remote EKG display on doctor’s side

##  6 Discussion

An open-source on-line remote EKG architecture has been introduced. This kind
of service is already offered in return for a payment, as part of health-care
packages, and it is usually based on proprietary hardware/software resources.
In contrast, this paper aims at an architecture based on open source
resources, with an attitude that we summarize in the OpLoC acronym: hardware,
software, and communication protocols are affordable open source products. A
few of them have been designed for the purpose (namely, the client/server
software and the serial protocol), but the rule is to use COTS resources. One
of the resources is in the cloud, the web-server that transfers the EKG from
the patient to the physician: no exception, it is a plain open-source web
server. To demonstrate that the architecture is feasible and to evaluate its
performance, we implemented a prototype that is exhaustively described and
reproducible. The OpLoC principles foster a wider diffusion of a useful device
on a more competitive basis, and make it applicable to disadvantaged or
marginal regions.

##  7 Abbreviations

OpLoC

    Open Source, Low Cost, COTS
COTS

    Commercial Off-the-shelf
EKG

    electrocardiogram
UA

    User Agent
FQDN

    Fully Qualified Domain Name
HTTP

    Hypertext Transfer Protocol
HTTPS

    Secure Hypertext Transfer Protocol
URL

    Uniform Resource Locator
JSON

    JavaScript Object Notation
WSGI

    Web Server Gateway Interface
MCU

    Micro-Controller Unit
SBC

    Single Board Computer
PaaS

    Platform as a Service
AP

    Access Point
WiFi

    Wireless Fidelity
BAN

    Body (or Personal) Data Network
MSN

    Medical Sensor Network
PC

    Personal Computer
AWS

    Amazon Web Services
NAT

    Network Address Translation
TLS

    Transport Level Security
ADC

    Analog Digital Converter
UART

    Universal Asynchronous Receiver-Transmitter
IP

    Internet Protocol
TCP

    Transport Control Protocol

##  8 Bibliography

## References

  * [1] SHIELD EKG-EMG REV.B.
  * [2] Mirza Mansoor Baig, Hamid GholamHosseini, Aasia A. Moqeem, Farhaan Mirza, and Maria Lindén.  A systematic review of wearable patient monitoring systems – current challenges and opportunities for clinical adoption.  Journal of Medical Systems, 41(7):115, Jun 2017.
  * [3] Min Chen, Sergio Gonzalez, Athanasios Vasilakos, Huasong Cao, and Victor C. M. Leung.  Body area networks: A survey.  Mobile Networks and Applications, 16(2):171–193, Apr 2011.
  * [4] Augusto Ciuffoletti.  Ecg over websocket project (client).  https://bitbucket.org/labreti/ecg-client, october 2017.
  * [5] Augusto Ciuffoletti.  Ecg over websocket project (server).  https://bitbucket.org/labreti/ecg-server, october 2017.
  * [6] Augusto Ciuffoletti.  A linux plotter for the Olimex EKG-EMG shield.  https://github.com/AugustoCiuffoletti/Olimex-EKG_plotter, september 2017\.
  * [7] C. Cristea, A. Pasarica, G. Andruseac, B. Dionisie, and C. Rotariu.  A wireless ecg acquisition device for remote monitoring of heart rate and arrhythmia detection.  In 2015 E-Health and Bioengineering Conference (EHB), pages 1–4, Nov 2015.
  * [8] S. Deb, S. M. R. Islam, J. RobaiatMou, and M. T. Islam.  Design and implementation of low cost ecg monitoring system for the patient using smart device.  In 2017 International Conference on Electrical, Computer and Communication Engineering (ECCE), pages 774–778, Feb 2017.
  * [9] I. Fette and A. Melnikov.  The WebSocket Protocol.  RFC 6455 (Proposed Standard), dec 2011.
  * [10] R. Gutierrez-Rivas, J. J. Garcia, W. P. Marnane, and A. Hernandez.  Novel real-time low-complexity qrs complex detector based on adaptive thresholding.  IEEE Sensors Journal, 15(10):6036–6043, Oct 2015.
  * [11] C. Haritha, M. Ganesan, and E. P. Sumesh.  A survey on modern trends in ecg noise removal techniques.  In 2016 International Conference on Circuit, Power and Computing Technologies (ICCPCT), pages 1–7, March 2016.
  * [12] Valerie M. Jones, Richard G.A. Bults, D. Konstantas, and P.A.M. Vierhout.  Healthcare pans: Personal area networks for trauma care and home care.  In Proceedings of the Fourth International Symposium on Wireless Personal Multimedia Communications, volume 3, pages 1369–1374, 2001.  Imported from research group ASNA (ID number 161).
  * [13] Tatiana S. Lugovaya.  Biometric human identification based on electrocardiogram.  Master’s thesis, Faculty of Computing Technologies and Informatics, Electrotechnical University ”LETI”, Saint-Petersburg, Russian Federation, 2005\.
  * [14] G. R. Naik and K. A. Reddy.  A new model for ecg signal filtering and feature extraction.  In 2016 2nd IEEE International Conference on Computer and Communications (ICCC), pages 765–768, Oct 2016.
  * [15] Sam Newman.  Building Microservices: designing fine-grained systems.  O’Reilly, 2015.
  * [16] G.P. Pizzuti, S. Cifaldi, and G. Nolfe.  Digital sampling rate and ECG analysis.  Journal of Biomedical Engineering, 7(3):247 – 250, 1985.
  * [17] E. Plesnik, O. Malgina, J. F. Tasič, and M. Zajc.  Ecg signal acquisition and analysis for telemonitoring.  In Melecon 2010 - 2010 15th IEEE Mediterranean Electrotechnical Conference, pages 1350–1355, April 2010.
  * [18] Duraisamy Sathya and Pugalendhi Ganesh Kumar.  Secured remote health monitoring system.  Healthcare Technology Letters, June 2017.
  * [19] Victor Shnayder, Bor-rong Chen, Konrad Lorincz, Thaddeus R. F. Fulford-Jones, and Matt Welsh.  Sensor networks for medical care.  Technical Report TR-08-05, Harvard Computer Science Group Technical Report, 2005.
  * [20] B. A. Walker, A. H. Khandoker, and J. Black.  Low cost ecg monitor for developing countries.  In 2009 International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP), pages 195–199, Dec 2009.
  * [21] Henian Xia, Irfan Asif, and Xiaopeng Zhao.  Cloud-ecg for real time ecg monitoring and analysis.  Computer Methods and Programs in Biomedicine, 110(3):253 – 259, 2013.

  *[PC]: Personal Computer
  *[EKG]: electrocardiogram
  *[AP]: Access Point
  *[MSN]: Medical Sensor Network
  *[BAN]: Body (or Personal) Data Network
  *[AWS]: Amazon Web Services
  *[HTTPS]: Secure Hypertext Transfer Protocol
  *[HTTP]: Hypertext Transfer Protocol
  *[IP]: Internet Protocol
  *[NAT]: Network Address Translation
  *[PaaS]: Platform as a Service
  *[URL]: Uniform Resource Locator
  *[TLS]: Transport Level Security
  *[TCP]: Transport Control Protocol
  *[OpLoC]: Open Source, Low Cost, COTS
  *[COTS]: Commercial Off-the-shelf
  *[MCU]: Micro-Controller Unit
  *[UA]: User Agent
  *[ADC]: Analog Digital Converter
  *[SBC]: Single Board Computer
  *[UART]: Universal Asynchronous Receiver-Transmitter
  *[JSON]: JavaScript Object Notation
  *[WSGI]: Web Server Gateway Interface
  *[FQDN]: Fully Qualified Domain Name