# On extendability of functionals on Hilbert $C^{*}$-modules

V. Manuilov  Moscow Center for Fundamental and Applied Mathematics, Moscow
State University, Leninskie Gory 1, Moscow, 119991, Russia

Harbin Institute of Technology, Xida St. 92, Harbin, Heilongjiang, 150001,
China<EMAIL_ADDRESS>
###### Abstract.

Let $M\subset N$ be Hilbert $C^{*}$-modules over a $C^{*}$-algebra $A$ with
$M^{\perp}=0$. It was shown recently by J. Kaad and M. Skeide that there
exists a non-zero $A$-valued functional on $N$ such that its restriction onto
$M$ is zero. Here we show that this may happen even if $A$ is monotone
complete. On the other hand, we show that for certain type I $W^{*}$-algebras
this cannot happen.

###### 2010 Mathematics Subject Classification:

46L08

The research was supported by RSF, project No. 21-11-00080

##  1\. Introduction

Hilbert $C^{*}$-modules were introduced by W. Paschke in [8] and became a
useful tool in various aspects of noncommutative geometry and topology. The
idea behind this notion is to generalize the notion of a Hilbert space by
replacing the field of the complex numbers, both as scalars and as the range
for the inner product, by a $C^{*}$-algebra $A$. For details about Hilbert
$C^{*}$-modules we refer to [4, 6, 7]. In some aspects, Hilbert
$C^{*}$-modules behave like Hilbert spaces, but there are three major
differences.

First, while any element $m\in M$ of a Hilbert $C^{*}$-module $M$ determines
an $A$-antilinear map $x\mapsto\langle x,m\rangle=\widehat{m}(x)$,
$\widehat{m}:M\to A$, not any bounded $A$-antilinear map from $M$ to $A$ is of
this form, i.e. there is no Riesz theorem. The $A$-module of all bounded
$A$-antilinear maps is called the dual module and is denoted by $M^{\prime}$.
The map $m\mapsto\widehat{m}$ determines an isometric inclusion $M\subset
M^{\prime}$, which is often not surjective. As a corollary, bounded $A$-linear
operators on Hilbert $C^{*}$-modules need not have an adjoint. This explains
the interest in special classes of Hilbert $C^{*}$-modules like self-dual or
reflexive ones.

The dual module $M^{\prime}$ often has no Hilbert $C^{*}$-module structure.
Nevertheless, when the underlying $C^{*}$-algebra $A$ is a $W^{*}$-algebra,
or, slightly more generally, a monotone complete $C^{*}$-algebra, the inner
product $\langle\cdot,\cdot\rangle$ extends from $M$ to $M^{\prime}$ in a
natural way, [2].

Second, while there is only one (up to an isometric isomorphism) separable
Hilbert space $l_{2}(\mathbb{C})$, a countably generated Hilbert
$C^{*}$-module over $A$ need not be isometrically isomorphic to the standard
Hilbert $C^{*}$-module $l_{2}(A)$.

Third, a Hilbert $C^{*}$-submodule $M\subset N$ in a Hilbert $C^{*}$-module
$N$ need not be (orthogonally) complementable. It may easily happen that the
orthogonal complement $M^{\perp}$ is zero, while $M$ is strictly smaller than
$N$. We call such submodules thick.

Any element $n\subset N$ determines a functional $\widehat{n}|_{M}$ on $M$
given by $\widehat{n}|_{M}(m)=\langle m,n\rangle$. A long-standing problem was
to check whether the norms $\|n\|$ and $\|\widehat{n}|_{M}\|$ coincide. If $A$
is a $W^{*}$-algebra and if $N=M^{\prime}$ (in this case $M^{\prime}$ is a
Hilbert $C^{*}$-module) then it was shown in [8] that these two norms are the
same.

Recently, a surprising counterexample appeared in [3], showing that for
$A=C[0;1]$ and for $M=l_{2}(A)$, there exists $N$ such that $M$ is thick in
$N$, and the two norms differ. As a concequence, it was shown in [3] that
there exist non-zero functionals on $N$ with zero restriction onto $M$, thus
showing that thick submodules may be not so thick.

Hilbert $C^{*}$-modules over $C[0,1]$ are reflexive, i.e. the second dual
module $M^{\prime\prime}$ coincides with $M$, but the first dual module is not
a Hilbert one, which is a reason behind a lot of counterexamples. In this
paper, we focus on the class of Hilbert $C^{*}$-modules, for which
$M^{\prime}=M^{\prime\prime}$ is a Hilbert $C^{*}$-module. In particular, this
is the case when $A$ is a monotone complete $C^{*}$-algebra [2].

We show that even when $A$ is monotone complete, the two norms may be
different. On the other hand, we show that if $A=L^{\infty}(X)$ is a
commutative $W^{*}$-algebra or $A=\mathbb{B}(H)$ is the algebra of all bounded
operators on a Hilbert space then the two norms coincide. We have no idea on
what happens when $A$ is a general $W^{*}$-algebra.

In order to compare the norms, we associate to each element $f$ of the dual
module, a right ideal $J_{f}$ in $A$ and show that the size of this ideal is
related to the coincidence of the two norms.

The author is grateful to M. Frank and E. Troitsky for fruitful discussions.

##  2\. Hilbert $C^{*}$-modules with Hilbert dual

Recall that the map $m\mapsto\widehat{m}$, $M\to M^{\prime}$, defines an
isometric inclusion, so we may identify $M$ with a subset in $M^{\prime}$ via
this map.

######  Definition 1.

A Hilbert $C^{*}$-module $M$ over a $C^{*}$-algebra $A$ is called a module
with a Hilbert dual if the inner product $\langle\cdot,\cdot\rangle$ on $M$
extends to an inner product $\langle\cdot,\cdot\rangle^{\prime}$ on
$M^{\prime}$ such that $\langle\widehat{m},\widehat{n}\rangle^{\prime}=\langle
m,n\rangle$ and $f(m)=\langle\widehat{m},f\rangle^{\prime}$ for any $m,n\in M$
and any $f\in M^{\prime}$.

######  Lemma 2.

Let $M$ be a Hilbert $C^{*}$-module with a Hilbert dual. Then $M^{\prime}$ is
self-dual.

###### Proof.

The identity map extends to the inclusion $M^{\prime\prime}\subset M^{\prime}$
for any Hilbert $C^{*}$-module. If $M^{\prime}$ is a Hilbert $C^{*}$-module
then the identity map extends to the inclusion $M^{\prime}\subset
M^{\prime\prime}$. Hence $M^{\prime\prime}=M^{\prime}$.

∎

######  Proposition 3.

Let $\langle\cdot,\cdot\rangle^{\prime}$ and
$\langle\cdot,\cdot\rangle^{\prime\prime}$ be two such extensions of the inner
product on $M$ to inner products on $M^{\prime}$. Then they coincide.

###### Proof.

Set $\langle f,g\rangle^{+}=\frac{1}{2}(\langle f,g\rangle^{\prime}+\langle
f,g\rangle^{\prime\prime})$, $f,g\in M^{\prime}$. Then
$\langle\cdot,\cdot\rangle^{+}$ is also an extension of the inner product on
$M$. Then $\langle f,f\rangle^{\prime}\leq 2\langle f,f\rangle^{+}$ for any
$f\in M^{\prime}$, hence $\mathop{\rm id}\nolimits:M^{\prime}\to M^{\prime}$
is a bounded map from $(M^{\prime},\langle,\cdot,\cdot\rangle^{+})$ to
$(M^{\prime},\langle,\cdot,\cdot\rangle^{\prime})$. It is also obviously
$A$-linear.

As $M^{\prime}$ with any inner product is self-dual, there exists the adjoint
operator $\mathop{\rm
id}\nolimits^{*}:(M^{\prime},\langle,\cdot,\cdot\rangle^{\prime})\to(M^{\prime},\langle,\cdot,\cdot\rangle^{+})$.
By definition, $\langle\mathop{\rm id}\nolimits f,g\rangle^{\prime}=\langle
f,\mathop{\rm id}\nolimits^{*}g\rangle^{+}$ for any $f,g\in M^{\prime}$. In
particular, this holds for $f=\widehat{m}$, $m\in M$. Then

$\langle\mathop{\rm
id}\nolimits\widehat{m},g\rangle^{+}=\langle\widehat{m},g\rangle=\langle\widehat{m},\mathop{\rm
id}\nolimits^{*}g\rangle=\langle\widehat{m},\mathop{\rm
id}\nolimits^{*}g\rangle^{\prime}$

for any $g\in M^{\prime}$ and any $m\in M$, hence $\mathop{\rm
id}\nolimits^{*}g=\mathop{\rm id}\nolimits g$, i.e. $\mathop{\rm
id}\nolimits^{*}=\mathop{\rm id}\nolimits$, therefore $\langle
f,g\rangle^{\prime}=\langle\mathop{\rm id}\nolimits f,\mathop{\rm id}\nolimits
g\rangle^{+}$ for any $f,g\in M^{\prime}$. Thus, $\langle
f,g\rangle^{\prime}=\langle f,g\rangle^{\prime\prime}$ for any $f,g\in
M^{\prime}$.

∎

Any self-dual Hilbert $C^{*}$-module and any Hilbert $C^{*}$-module over a
monotone complete $C^{*}$-algebra is obviously a module with a Hilbert dual
[2].

##  3\. Right ideals from functionals

Let $f\in M^{\prime}$. Define the subset $J_{f}\subset A$ by $J_{f}=\\{a\in
A:fa\in M\\}$.

######  Lemma 4.

$J_{f}$ is a closed right ideal in $A$.

###### Proof.

Obvious. ∎

Apart from the trivial case when $f\in M$ (then $M_{f}=A$), there are two
extreme cases for the right ideal $J_{f}$ when $f\notin M$:

  * •

$J_{f}$ is zero;

  * •

$J_{f}$ is essential, i.e. $aJ_{f}=0$, $a\in A$, implies $a=0$.

Let $M$ be a Hilbert $C^{*}$-module over a $C^{*}$-algebra $A$, $f\in
M^{\prime}$. Define $M^{0}_{f}$ as the $A$-linear span of $M$ and $f$, i.e.

$M^{0}_{f}=\\{m+fa:m\in M,a\in A\\}\subset M^{\prime},$

and let $M_{f}$ be the norm closure of $M^{0}_{f}$. Usually, $M_{f}$ need not
be a pre-Hilbert $C^{*}$-module, i.e. the inner product on $M$ need not be
extendable to $M_{f}$, but sometimes this happens, e.g. when $A$ is a monotone
complete $C^{*}$-algebra [2]. We are interested in the uniqueness of the inner
product extension from $M$ to $M_{f}$, or to larger modules, e.g. to
$M^{\prime}$.

######  Lemma 5.

Suppose that the inner product on $M$ extends to an inner product on $M_{f}$.
If the right ideal $J_{f}$ is essential then the extended inner product on
$M_{f}$ is unique.

###### Proof.

Let $\langle\cdot,\cdot\rangle_{1}$ and $\langle\cdot,\cdot\rangle_{2}$ be two
extensions of the inner product $\langle\cdot,\cdot\rangle$ on $M$ making
$M_{f}$ a pre-Hilbert $C^{*}$-module. Set $a_{i}=\langle f,f\rangle_{i}$,
$i=1,2$. If $j\in J_{f}$ then $fj\in M$, hence $f(fj)=\langle
fj,f\rangle_{i}$, $i=1,2$, hence $j^{*}a_{1}=j^{*}a_{2}$, or, equivalently,
passing to the adjoints, $(a_{2}-a_{1})j=0$. As this holds for any $j\in
J_{f}$ and as $J_{f}$ is essential then $a_{2}-a_{1}=0$, hence the two inner
products coincide.

∎

######  Theorem 6.

Suppose that $M$ is a Hilbert $C^{*}$-module with a Hilbert dual, and let
$\langle\cdot,\cdot\rangle_{M^{\prime}}$ be the extention of the inner product
$\langle\cdot,\cdot\rangle$ from $M$ to $M^{\prime}$. Suppose also that $M$ is
a thick submodule in a Hilbert $C^{*}$-module $N$ with the inner product
$\langle\cdot,\cdot\rangle_{N}$, which determines an injection $i:N\to
M^{\prime}$ (the inner product on $M$ is supposed to be the restriction of the
inner product on $N$). If the right ideal $J_{f}$ is essential for any $f\in
M^{\prime}$ then the inner product on $N$ coincides with the extended inner
product on $M^{\prime}$, i.e. $\langle n,n^{\prime}\rangle_{N}=\langle
i(n),i(n^{\prime})\rangle_{M^{\prime}}$ for any $n,n^{\prime}\in N$.

###### Proof.

Recall that the injection $j$ is defined as follows: for $n\in N$, set
$i(n)(m)=\langle m,n\rangle_{N}$, $m\in M$, i.e. $i(n)=\widehat{n}|_{M}$.
Thickness of $M$ in $N$ implies that $i$ is injective.

Suppose that there exists $n\in N$ such that $\langle
n,n\rangle_{N}\neq\langle i(n),i(n)\rangle_{M^{\prime}}$. Set $f=i(n)\in
M^{\prime}$, and let $j\in J_{f}$. Then, by definition, $m=i(n)j\in M$, and

$\langle m^{\prime},nj\rangle_{N}=\langle
m^{\prime},nj\rangle_{N}=i(nj)(m^{\prime})=\langle
m^{\prime},i(n)j\rangle_{M^{\prime}},\quad m^{\prime}\in M$

hence $\langle m^{\prime},nj-m\rangle_{N}=0$ for any $m^{\prime}\in M$. As $M$
is thick in $N$, the latter implies that $nj=m\in M$.

Set $a=\langle n,n\rangle_{N}$, $b=\langle i(n),i(n)\rangle_{M^{\prime}}$.
Then

$j^{*}aj=\langle nj,nj\rangle_{N}=\langle m,m\rangle_{M}=\langle
i(n)j,i(n)j\rangle_{M^{\prime}}=j^{*}bj,$

hence $j^{*}(a-b)j=0$ for any $j\in J_{f}$. As $J_{f}$ is essential, this
implies that $a=b$. Thus, $\langle n,n\rangle_{N}=\langle
i(n),i(n)\rangle_{M^{\prime}}$ for any $n\in N$. Polarization identity then
implies that $\langle n^{\prime},n\rangle_{N}=\langle
i(n^{\prime}),i(n)\rangle_{M^{\prime}}$ for any $n,n^{\prime}\in N$.

∎

######  Theorem 7.

Suppose that $m$ is a Hilbert $C^{*}$-module with a Hilbert dual, and let
$\langle\cdot,\cdot\rangle^{\prime}$ be the extension of the inner product
$\langle\cdot,\cdot\rangle$ from $M$ to $M^{\prime}$, hence to $M_{f}\subset
M^{\prime}$. Suppose also that $\langle f,f\rangle^{\prime}\geq 1$. If
$J_{f}=0$ then there exists a Hilbert $C^{*}$-module $N$ such that $M\subset
N$ is thick, $f\in N$, and the norm of $f$ as a functional (i.e. the standard
norm on $M^{\prime}$) is strictly smaller than the norm of $f$ as an element
of $N$.

###### Proof.

The condition $J_{f}=0$ means that the subspaces $M$ and $fA=\\{fa:a\in A\\}$
form a direct sum (note that it need not be a topological direct sum) equal to
$M^{0}_{f}$. Let $c=\langle f,f\rangle^{\prime}\in A$, and, for
$m+fa,m^{\prime}+fa^{\prime}\in M^{0}_{f}$, set

$\langle m+fa,m^{\prime}+fa^{\prime}\rangle_{+}=\langle
m+fa,m^{\prime}+fa^{\prime}\rangle^{\prime}+a^{*}ca^{\prime}.$

Let us check linearity with respect to the first argument:

$\displaystyle\langle
m_{1}{+}m_{2}{+}f(a_{1}{+}a_{2}),m^{\prime}{+}fa^{\prime}\rangle_{+}$
$\displaystyle\\!=\\!$ $\displaystyle\langle
m_{1}{+}m_{2}{+}f(a_{1}{+}a_{2}),m^{\prime}{+}fa^{\prime}\rangle^{\prime}{+}(a_{1}{+}a_{2})ca^{\prime}$
$\displaystyle\\!=\\!$ $\displaystyle\langle
m_{1}{+}fa_{1},m^{\prime}{+}fa^{\prime}\rangle^{\prime}{+}\langle
m_{2}{+}fa_{2},m^{\prime}{+}fa^{\prime}\rangle^{\prime}{+}(a_{1}{+}a_{2})ca^{\prime}$
$\displaystyle\\!=\\!$ $\displaystyle\langle
m_{1}{+}fa_{1},m^{\prime}{+}fa^{\prime}\rangle_{+}{+}\langle
m_{2}{+}fa_{2},m^{\prime}{+}fa^{\prime}\rangle_{+}.$

Other properties of an inner product obviously hold, making thus $M^{0}_{f}$ a
pre-Hilbert $C^{*}$-module with respect to $\langle\cdot,\cdot\rangle_{+}$.

Let us show that $M^{0}_{f}$ is complete with respect to the norm
$\|\cdot\|_{+}$ determined by the inner product
$\langle\cdot,\cdot\rangle_{+}$ (it may be not complete with respect to the
norm on $M^{\prime}$).

Suppose that $\\{m_{k}+fa_{k}\\}_{k\in\mathbb{N}}$ is a Cauchy sequence with
respect to $\|\cdot\|_{+}$, where $m_{k}\in M$, $a_{k}\in A$,
$k\in\mathbb{N}$. Note that $\|fa\|_{+}\leq\|m+fa\|_{+}$ for any $m\in M$,
$a\in A$, therefore, $\\{fa_{k}\\}_{k\in\mathbb{N}}$ is a Cauchy sequence as
well. Then $\\{m_{k}\\}_{k\in\mathbb{N}}$ is Cauchy too.

As $\langle f,f\rangle^{\prime}\geq 1$, we have

$a^{*}a\leq a^{*}\langle f,f\rangle^{\prime}a=\langle fa,fa\rangle^{\prime}$

for any $a\in A$, hence

$\|a_{k}-a_{n}\|^{2}\leq\langle fa_{k}-fa_{n},fa_{k}-fa_{n}\rangle^{\prime},$

so $\\{ak\\}_{k\in\mathbb{N}}$ is a Cauchy sequence too. As both $A$ and $M$
are complete, the limit of the sequence $\\{m_{k}+fa_{k}\\}_{k\in\mathbb{N}}$
is of the form $m+fa$ for some $m\in M$ and $a\in A$, thus $M^{0}_{f}$ is
complete.

Note that

$\langle m,m^{\prime}+fa\rangle_{+}=\langle
m,m^{\prime}+fa\rangle^{\prime}=\langle m,m^{\prime}\rangle+f(m)a,\quad
m,m^{\prime}\in M,a\in A,$

hence $\langle\cdot,\cdot\rangle_{+}$ is an extension of the inner product
$\langle,\cdot,\cdot\rangle$ on $M$ to $N$. As $M$ is thick in $M^{\prime}$,
it is thick in $N$ as well. Indeed, if $\langle m,m^{\prime}+fa\rangle_{+}=0$
for any $m\in M$ then $\langle m,m^{\prime}+fa\rangle^{\prime}=0$, hence
$m^{\prime}+fa\in M^{\prime}$ is orthogonal to the whole $M$, hence
$m^{\prime}+fa=0$.

Finally,

$\|f\|^{\prime}=(\langle f,f\rangle^{\prime})^{1/2}=\|c\|^{1/2},$

while

$\|f\|_{+}=(\langle f,f\rangle_{+})^{1/2}=(\langle
f,f\rangle^{\prime}+c)^{1/2}=\sqrt{2}\|c\|^{1/2}.$

∎

######  Remark 8.

Considering $\widetilde{M}=A\oplus M$ instead of $M$, and
$\widetilde{f}=(\widehat{1}_{A},f)\in\widetilde{M}^{\prime}$ instead of $f\in
M^{\prime}$, we get $\langle\widetilde{f},\widetilde{f}\rangle^{\prime}\geq
1$, and $J_{\widetilde{f}}=J_{f}$, so the condition $\langle
f,f\rangle^{\prime}\geq 1$ is not restrictive.

##  4\. Positive results

For a Hilbert $C^{*}$-module $N$ we write $\widehat{n}$ for the functional
$x\mapsto\langle n,x\rangle$.

######  Theorem 9.

Let $M\subset N$ be Hilbert $C^{*}$-modules over a commutative $W^{*}$-algebra
$A=C(X)$ such that $M$ is countably generated and $M^{\perp}=0$. Then
$\|n\|=\|\widehat{n}|_{M}\|$ for any $n\in N$,i.e. the map
$n\mapsto\widehat{n}|_{M}$, $N\to M^{\prime}$, is an isometry.

###### Proof.

If $M$ is countably generated then it can be considered as a direct summand in
the standard Hilbert $C^{*}$-module $l_{2}(A)$, i.e. $l_{2}(A)=M\oplus K$ for
some Hilbert $C^{*}$-module $K$. The functional $\widehat{n}|_{M}$, $n\in N$,
can be similarly considered as a functional on $l_{2}(A)$ (defined by zero on
$K$). If $\|n\|\neq\|\widehat{n}|_{M}\|$ for some $n\in N$ then the two norms
differ for $(n,0)\in N\oplus K$, so the proof reduces to the case when
$M=l_{2}(A)$.

Let $f\in M^{\prime}$, and let $J_{f}=\\{a\in A:fa\in M\\}$ be an ideal in
$A$. We claim that if $M=l_{2}(A)$ then $J_{f}$ is essential for any $f\in
M^{\prime}$, i.e. $aJ_{f}=0$, $a\in A$, implies that $a=0$. Let
$f=(f_{n})_{n\in\mathbb{N}}$, $f_{n}\in C(X)$. The condition $f\in M^{\prime}$
means that $\sum_{n=1}^{\infty}|f_{n}|^{2}$ is bounded. Set
$g_{n}=\sum_{k=1}^{n}|f_{n}|^{2}$. As $A$ is a $W^{*}$-algebra, the least
upper bound $\bar{g}$ of the monotone sequence $(g_{n})_{n\in\mathbb{N}}$
exists and coincides with the pointwise limit
$\check{g}=\lim_{n\to\infty}g_{n}$ outside a meager subset $Y_{f}$ of $X$. We
claim that $J_{f}$ consists of all functions $a$ vanishing on $Y_{f}$. Recall
that $fa\in M$ iff the sequence $\\{f_{n}a\\}_{n\in\mathbb{N}}$ is norm
convergent.

Supppose that $a\in A$, $a|_{Y_{f}}=0$. Given $\varepsilon>0$, set $U=\\{x\in
X:|a(x)|<\frac{\varepsilon}{\|\bar{g}\|}\\}$. Clearly, $U\subset X$ is open,
and $Y_{f}\subset U$. Then $|g_{n}(x)a(x)-\bar{g}(x)a(x)|\leq\varepsilon$ for
any $x\in U$ and for any $n\in\mathbb{N}$. On the compact set $X\setminus U$,
the sequence $\\{g_{n}a\\}_{n\in\mathbb{N}}$ converges pointwise to the
continuous function $\bar{g}a$, hence the convergence is uniform — both on $U$
and on $X\setminus U$. Thus, $a\in J_{f}$.

Now, let $a\in J_{f}$. Suppose that there exists $x_{0}\in Y_{f}$ such that
$a(x_{0})\neq 0$. The sequence $\\{g_{n}a\\}_{n\in\mathbb{N}}$ is uniformly
convergent on $X$, hence $\check{g}a$ is continuous and coincides with
$\bar{g}a$, i.e. $\bar{g}(x)a(x)=\check{g}(x)a(x)$ for any $x\in X$. But
$\bar{g}(x_{0})\neq\check{g}(x_{0})$. This contradiction shows that
$a|_{Y_{f}}=0$.

Let us show that the ideal $J_{f}$ is essential. Let $aJ_{f}=0$ for some $a\in
C(X)$. Let $V=\\{x\in X:a(x)\neq 0\\}$. Then $V$ is an open subset of $X$. As
$ab=0$ for any $b\in J_{f}$, we should have $b(x)=0$ for any $x\in V$. Recall
that if $C(X)$ is a $W^{*}$-algebra then $X$ is a hyperstonean space. By
Corollary following Proposition 5 of [1], in hyperstonean spaces, any meager
set is nowhere dense, i.e. the interior of its closure is empty. Recall that
functions in $J_{f}$ equal zero on $Y_{f}$, hence, on its closure
$\overline{Y}_{f}$. The condition $aJ_{f}=0$ means that for any $b\in J_{f}$
one has $b|_{V}=0$. We claim that this implies $V\subset\overline{Y}_{f}$.
Indeed, assume the contrary: let $x_{0}\in V\setminus\overline{Y}_{f}$ satisfy
the condition: any continuous function that vanishes on $\overline{Y}_{f}$
vanishes at $x_{0}$. As $X$ is compact and Hausdorff, it is normal, hence
there exists a continuous function that vanishes on $\overline{Y}_{f}$ and
equals 1 at $x_{0}$. This contradiction proves $V\subset\overline{Y}_{f}$. But
$Y_{f}$ is nowhere dense, hence the open set $V$ must be empty. Hence $a=0$.

∎

Let $J$ be of $c_{0}$-$\sum_{i}\oplus\mathbb{K}(H_{i})$-type, i.e. it has a
faithful $*$-representation as a $C^{*}$-algebra of compact operators on some
Hilbert space, and let $A\supset J$ be a $C^{*}$-algebra such that $J$ is an
essential ideal in $A$. As examples, we may consider $A=\mathbb{B}(H)$,
$J=\mathbb{K}(H)$ and $A=l_{\infty}$, $J=c_{0}$.

######  Theorem 10.

Let $M\subset N$ be Hilbert $C^{*}$-modules over $A$ such that $M^{\perp}=0$.
Suppose that $M$ is a Hilbert $C^{*}$-module with a Hilbert dual. If $A$ is a
$C^{*}$-algebra containing an essential ideal $J$ of
$c_{0}$-$\sum_{i}\oplus\mathbb{K}(H_{i})$-type then
$\|n\|=\|\widehat{n}|_{M}\|$ for any $n\in N$, i.e. the map
$n\mapsto\widehat{n}|_{M}$, $N\to M^{\prime}$, is an isometry.

###### Proof.

Denote by $MJ\subset M$ the norm closure of the linear span of $\\{mj:m\in
M,j\in J\\}$. Then $MJ$ is a Hilbert $C^{*}$-module over $J$. Clearly,
$MJ\subset NJ$. It was shown in [5] that any submodule over $J$ is
orthogonally complemented.

We claim that $MJ=NJ$. Indeed, if this is not true then the orthogonal
complement for $MJ$ in $NJ$ is not zero. Let $x\in NJ$ satisfy $x\perp MJ$.
Then $\langle mj,x\rangle=0$ for any $m\in M$ and any $j\in J$. As $J$ is
essential, we can conclude that $\langle m,x\rangle=0$ for any $m\in M$. As
$M^{\perp}=0$ in $N$, we conclude that $x=0$. It follows that $nj\in M$ for
any $n\in N$ and any $j\in J$.

Similarly, consider the inclusion $M\subset M^{\prime}$. Here we have
$M^{\perp}=0$ as well. Passing to $MJ\subset M^{\prime}J$, we can conclude
that $fj\in M$ for any $f\in M^{\prime}$. In particular,
$\widehat{n}|_{M}\cdot j\in M$.

By assumption, $M^{\prime}$ is a Hilbert $C^{*}$-module with the inner product
$\langle\cdot,\cdot\rangle_{M^{\prime}}$. By Theorem 6,
$\langle\widehat{n}|_{M},\widehat{n}|_{M}\rangle_{M^{\prime}}=\langle
n,n\rangle_{N}$ for any $n\in N$.

∎

##  5\. Negative results

Here we show that the norm of extensions of functionals from thick submodules
may be not unique even for commutative monotone complete $C^{*}$-algebras.
Take $X=[0,1]$, and consider the $C^{*}$-algebra $Bor(X)$ of all bounded Borel
functions with the sup-norm. Consider the set $J$ of all functions $a\in
Bor(X)$ such that the set $\\{x\in X:a(x)\neq 0\\}$ is meager. The set $J$ is
an ideal in $Bor(X)$, so let $A=B(X)=Bor(X)/J$ be the quotient
$C^{*}$-algebra. We denote the equivalence class of $a$ by $[a]\in A$. The
$C^{*}$-algebra $B(X)$ was showed to be monotone complete in [1], while
$Bor(X)$ is only monotone $\sigma$-complete [9].

######  Lemma 11.

There exists $f\in l_{2}(A)^{\prime}$ with $\langle f,f\rangle^{\prime}\geq 1$
such that $fa\in l_{2}(A)$ for $a\in A$ implies that $a=0$.

###### Proof.

We use the idea from [3] to construct $f=([f_{1}],[f_{2}],\ldots)\in
l_{2}(A)^{\prime}$ in such a way that the least upper bound for the partial
sums of $\sum_{k=1}^{\infty}|f_{k}|^{2}$ would differ from their pointwise
limit on the set of rationals in $(0,1)$. Here $f_{k}\in Bor(X)$,
$k\in\mathbb{N}$.

Let $(r_{m})_{m\in\mathbb{N}}$ be the sequence of all rational numbers in
$(0,1)$ enumerated by natural numbers. For each $r_{m}$, take a strictly
increasing sequence $\alpha_{m}^{k}\in(0,1)$ such that
$\lim_{k\to\infty}\alpha_{m}^{k}=r_{m}$, and let $h_{m}^{k}$ be the
characteristic function of the interval $[\alpha_{m}^{k},\alpha_{m}^{k+1})$.
Fix a bijection

$\varphi:\mathbb{N}\to\mathbb{N}^{2},\quad\varphi(n)=(m(n),k(n)),$

and set $f_{n}=\frac{1}{2^{m(n)}}h_{m(n)}^{k(n)}$. Then the partial sums
$\sum_{n=1}^{N}|f_{n}|^{2}$ (and hence $\sum_{n=1}^{N}[|f_{n}|]^{2}$) are
uniformly bounded, hence $f=([f_{n}])_{n\in\mathbb{N}}\in l_{2}(A)^{\prime}$.
Note that we could take continuous functions instead of $h_{n}^{k}$ to get a
required $f$. To fit the condition $\langle f,f\rangle^{\prime}\geq 1$ we may
replace the constructed $f=([f_{1}],[f_{2}],\ldots)$ by
$(1_{A},[f_{1}],[f_{2}],\ldots)$. For shortness’ sake set
$g_{m}=\sum_{k=1}^{m}|f_{k}|^{2}\in Bor(X)$.

The series $\sum_{k=1}^{n}[|f_{k}|]^{2}$ is not norm convergent in $B(X)$
(i.e. $f\notin l_{2}(A)$). Indeed, note first that
$\|\chi\|_{Bor(X)}=\|[\chi]\|_{B(X)}=1$ for a characteristic function $\chi$
of any open subset of $X$. The norm in the quotient algebra is given by

$\|[\chi]\|_{B(X)}=\inf_{\psi\in J}\|\chi+\psi\|_{Bor(X)}=1$

as $\psi$ vanishes outside of a meager set. Take any $N\in\mathbb{N}$. There
exists $n_{0}\in\mathbb{N}$ such that $m(N+n_{0})=1$. Then, for any
$M>N+n_{0}$, we have $g_{M}-g_{N}\geq\frac{1}{2}h_{1}^{k(N+n_{0})}$, hence

$\|[g_{M}]-[g_{N}]\|\geq\frac{1}{2}\|[h_{1}^{k(N+n_{0})}]\|=\frac{1}{2}.$

Moreover, this series is not norm convergent on any closed interval
$[c,d]=V\subset X$. Let $\check{q}:Bor(X)\to Bor(V)$ denote the restriction
map. It induces the map of the quotients $q:B(X)\to B(V)$. We claim that the
series $\sum_{k=1}^{\infty}q([|f_{k}|])^{2}$ is not norm convergent. Indeed,
take $m_{0}$ such that $r_{m_{0}}\in V$, then we can repeat the previous
argument. For any $N\in\mathbb{N}$ we can find $n_{0}\in\mathbb{N}$ such that
$m(N+n_{0})=m_{0}$. Then, for any $M>N+n_{0}$, we have
$g_{M}-g_{N}\geq\frac{1}{2^{m_{0}}}h_{m_{0}}^{k(N+n_{0})}$, hence

$\|[g_{M}]-[g_{N}]\|\geq\frac{1}{2^{m_{0}}}\|[h_{m_{0}}^{k(N+n_{0})}]\|=\frac{1}{2^{m_{0}}}.$

Let $a\in B(X)$ such that $a\neq 0$ and $fa\in l_{2}(A)$. Set $b=aa^{*}$, then
$b\neq 0$, $b\geq 0$ and $fb\in l_{2}(A)$, so we may replace $a$ by $b$. If we
show that $b=0$ then $a=0$ as well.

Recall that every equivalence class in $B(X)$ has a lower semicontinuous
representative [1], so there exists a lower semicontinuous $\beta\in Bor(X)$
such that $[\beta]=b$. Let

$f(t)=\left\\{\begin{array}[]{cl}t&\mbox{if\ }t\geq 0;\\\ 0&\mbox{if\
}t<0.\end{array}\right.$

Then $[f\circ\beta]=f([\beta])=f(b)=b$. As $f$ is non-decreasing, it is easy
to see that $\alpha=f\circ\beta\in Bor(X)$ is also lower semicontinuous. We
also have $\alpha(x)\geq 0$ for any $x\in X$. As $b\neq 0$, there exists
$\varepsilon>0$ and a non-empty open set $U\subset X$ such that
$\alpha(x)>\varepsilon$ for any $x\in U$. Let $V\subset U$ be a non-empty
closed subset. If the sequence $\\{[g_{n}]b\\}_{n\in\mathbb{N}}$ is norm
convergent in $B(X)$ (i.e. if $fb\in l_{2}(A)$) then the sequence
$\\{q([g_{n}])\\}_{n\in\mathbb{N}}$ is norm convergent in $B(V)$ too — a
contradiction.

∎

######  Corollary 12.

There exists a monotone complete commutative $C^{*}$-algebra $A$, a Hilbert
$C^{*}$-module $N$ over $A$, a thick submodule $M\subset N$, and $n\in N$ such
that $\|n\|_{N}>\|\widehat{n}|_{M}\|_{M^{\prime}}$.

###### Proof.

Let $A$ be as above, $M=l_{2}(A)$. Take $f\in l_{2}(A)^{\prime}$ as in Lemma
11, i.e. such that $J_{f}=0$. Then apply Theorem 7.

∎

There is an interesting consequence of this result. We show that there exist
functionals in $N^{\prime}$ that equal zero on $M$.

Let $M\subset N$, and suppose that both $M$ and $N$ are Hilbert
$C^{*}$-modules with a Hilbert dual. Denote the inclusion $M\to N$ by
$\alpha$, and define the map $P:N^{\prime}\to M^{\prime}$ by
$P(g)(m)=g(\alpha(m))$, $g\in N^{\prime}$, $m\in M$. As the dual modules are
Hilbert $C^{*}$-modules, this can be written as

(1)
$\langle\widehat{m},P(g)\rangle_{M^{\prime}}=\langle\widehat{\alpha(m)},g\rangle_{N^{\prime}}.$

As the dual modules are Hilbert $C^{*}$-modules, they are self-dual, i.e.
equal to their second duals. Therefore, all bounded operators are adjointable.
Let $I=P^{*}:M^{\prime}\to N^{\prime}$. Then $\langle
f,P(g)\rangle_{M^{\prime}}=\langle I(f),g\rangle_{N^{\prime}}$, $f\in
M^{\prime}$. Taking $f=\widehat{m}$, $m\in M$, we get

(2) $\langle\widehat{m},P(g)\rangle_{M^{\prime}}=\langle
I(\widehat{m}),g\rangle_{N^{\prime}}.$

Comparing (1) and (2), we see that $I(\widehat{m})=\widehat{\alpha(m)}$ for
any $m\in M$.

######  Theorem 13.

Let $M\subset N$ be a thick submodule, let both $M$ and $N$ are Hilbert
$C^{*}$-modules with a Hilbert dual. Then the map $I:M^{\prime}\to N^{\prime}$
is an isometric inclusion. If there exists $n\in N$ such that
$\|n\|_{N}>\|\widehat{n}|_{M}\|_{M^{\prime}}$ then
$N^{\prime}=I(M^{\prime})\oplus K$ for some non-zero $K$.

###### Proof.

Let us calculate the composition $PI:M^{\prime}\to M^{\prime}$. For
$m,m^{\prime}\in M$, we have

$\displaystyle\langle
PI(\widehat{m}),\widehat{m}^{\prime}\rangle_{M^{\prime}}$ $\displaystyle=$
$\displaystyle\langle
I(\widehat{m}),I(\widehat{m}^{\prime})\rangle_{N^{\prime}}$ $\displaystyle=$
$\displaystyle\langle\widehat{\alpha(m)},\widehat{\alpha(m^{\prime})}\rangle_{N^{\prime}}$
$\displaystyle=$ $\displaystyle\langle\alpha(m),\alpha(m^{\prime})\rangle_{N}$
$\displaystyle=$ $\displaystyle\langle m,m^{\prime}\rangle_{M}$
$\displaystyle=$
$\displaystyle\langle\widehat{m},\widehat{m}^{\prime}\rangle_{M^{\prime}},$

hence, $\langle
PI(\widehat{m})-\widehat{m},\widehat{m}^{\prime}\rangle_{M^{\prime}}=0$ for
any $m^{\prime}\in M$, thus, as $M\subset M^{\prime}$ is thick, we conclude
that $PI(\widehat{m})=\widehat{m}$ for any $m\in M$.

Note that $PI$ is selfadjoint, and $(PI)|_{M}=\mathop{\rm id}\nolimits_{M}$.
This suffices to show that $PI=\mathop{\rm id}\nolimits_{M^{\prime}}$:

$\langle PI(f),\widehat{m}\rangle_{M^{\prime}}=\langle
f,PI(\widehat{m})\rangle_{M^{\prime}}=\langle
f,\widehat{m}\rangle_{M^{\prime}},\quad f\in M^{\prime},$

hence, by thickness of $M$ in $M^{\prime}$, we conclude that $PI(f)=f$ for any
$f\in M^{\prime}$.

Then $IP:N^{\prime}\to N^{\prime}$ is a projection with range $M^{\prime}$,
and both $P$ and $I$ are contractions. It follows from (1) that

$\langle\widehat{m},P(\widehat{n})\rangle_{M^{\prime}}=\langle\widehat{\alpha(m)},\widehat{n}\rangle_{N^{\prime}}=\langle\alpha(m),n\rangle_{N}=\widehat{n}|_{M}(m)$

for any $m\in M$, $n\in N$, hence $P(\widehat{n})=\widehat{n}|_{M}$ for any
$n\in N$.

Let $n\in N$ satisfy $\|n\|_{N}>\|\widehat{n}|_{M}\|_{M^{\prime}}$. Then
$\|P(\widehat{n})\|_{M^{\prime}}<\|n\|_{N}$, hence
$\|IP(\widehat{n})\|<\|n\|_{N}$. If $IP=\mathop{\rm id}\nolimits_{N^{\prime}}$
then this cannot be true, hence the projection $IP$ is not the identity map.
Set $K=(\mathop{\rm id}\nolimits_{N^{\prime}}-IP)(N^{\prime})$.

∎

######  Corollary 14.

Let $M\subset N$ be a thick submodule, and let both $M$ and $N$ be Hilbert
$C^{*}$-modules with a Hilbert dual. If there exists $n\in N$ such that
$\|n\|_{N}>\|\widehat{n}|_{M}\|_{M^{\prime}}$ then there exists $g\in
N^{\prime}$ such that $g|_{M}=0$.

###### Proof.

Let $K$ be as in Theorem 13, and let $g\in K$, $g\neq 0$. Then $\langle
g,f\rangle^{\prime}=0$ for any $f\in M^{\prime}$, hence $g|_{M}=0$.

∎

## References

  * [1] J. Dixmier. On certains espaces considérés par M. H. Stone. Sum. Bras. Math. II, 11 (1951), 151–182.
  * [2] M. Frank.  Hilbert $C^{*}$-modules over monotone complete $C^{*}$-algebras.  Math.Nachr. 175(1995), 61–83.
  * [3] J. Kaad, M. Skeide.  Kernels of Hilbert $C^{*}$-module maps: a counterexample. J. Operator Theory, to appear; www.arxiv.org: 2101.03030.
  * [4] E. C. Lance. Hilbert $C^{*}$-Modules — a Toolkit for Operator Algebraists. London Math. Soc. Lecture Note Series 210, Cambridge, 1995.
  * [5] B. Magajna. Hilbert $C^{*}$-modules in which all closed submodules are complemented. Proc. Amer. Math. Soc. 125 (1997), 849–852.
  * [6] V. M. Manuilov, E. V. Troitsky.  Hilbert $C^{*}$-Modules.  AMS, Providence, R.I., 2005.
  * [7] V. M. Manuilov, E. V. Troitsky. Hilbert $C^{*}$\- and $W^{*}$\- Modules and Their Morphisms. J. Math. Sci. 98 (2000), 137–201.
  * [8] W. L. Paschke.  Inner product modules over $B^{*}$-algebras.  Trans. Amer. Math. Soc. 182 (1973), 443–468.
  * [9] K. Saitô, J. D. Maitland Wright. Monotone Complete $C^{*}$-Algebras and Generic Dynamics. Springer, London, 2015.


#  STELAR: Spatio-temporal Tensor Factorization with Latent Epidemiological
Regularization

Nikos Kargas1,2,, Cheng Qian2,*, Nicholas D. Sidiropoulos3, Cao Xiao2,
Lucas M. Glass2,4, Jimeng Sun5
equal contribution

###### Abstract

Accurate prediction of the transmission of epidemic diseases such as COVID-19
is crucial for implementing effective mitigation measures. In this work, we
develop a tensor method to predict the evolution of epidemic trends for many
regions simultaneously. We construct a $3$-way spatio-temporal tensor
(location, attribute, time) of case counts and propose a nonnegative tensor
factorization with latent epidemiological model regularization named STELAR.
Unlike standard tensor factorization methods which cannot predict slabs ahead,
STELAR enables long-term prediction by incorporating latent temporal
regularization through a system of discrete-time difference equations of a
widely adopted epidemiological model. We use latent instead of
location/attribute-level epidemiological dynamics to capture common epidemic
profile sub-types and improve collaborative learning and prediction. We
conduct experiments using both county- and state-level COVID-19 data and show
that our model can identify interesting latent patterns of the epidemic.
Finally, we evaluate the predictive ability of our method and show superior
performance compared to the baselines, achieving up to $21\%$ lower root mean
square error and $25\%$ lower mean absolute error for county-level prediction.

## Introduction

Pandemic diseases such as the novel coronavirus disease (COVID-19) pose a
serious threat to global public health, economy, and daily life. Accurate
epidemiological measurement, modeling, and tracking are needed to inform
public health officials, government executives, policy makers, emergency
responders, and the public at large. Two types of epidemiological modeling
methods are popular today:

  * •

Mechanistic models define a set of ordinary differential equations (ODEs)
which capture the epidemic transmission patterns and predict the long-term
trajectory of the outbreak. These models include the Susceptible-Infected-
Recovered (SIR) model (Kermack and McKendrick 1927), the Susceptible-Exposed-
Infected-Recovered (SEIR) (Cooke and Van Den Driessche 1996) and their
variants e.g, SIS, SIRS, and delayed SIR. These models have a small number of
parameters which are determined via curve fitting. These types of models do
not require much training data, but they are quite restrictive and cannot
leverage rich information.

  * •

Machine learning models such as deep learning, model the epidemiological
trends as a set of regression problems (Yang et al. 2020; Toda 2020; He, Peng,
and Sun 2020; Chimmula and Zhang 2020; Tomar and Gupta 2020). These models are
able to learn the trajectory of the outbreak only from data and can perform
very well especially in short-term prediction. However, they usually require a
large amount of training data.

This paper extends Canonical Polyadic Decomposition (CPD) (Harshman 1970) via
epidemiological model (e.g., SIR and SEIR) regularization, which integrates
spatio-temporal real-world case data and exploits the correlation between
different regions for long-term epidemic prediction. CPD is a powerful tensor
model with successful applications in many fields (Papalexakis, Faloutsos, and
Sidiropoulos 2016; Sidiropoulos et al. 2017). Compared to traditional
epidemiological models which cannot incorporate fine-grain observations or any
kind of side information, tensors offer a natural way of representing
multidimensional time evolving data and incorporate additional information
(Acar, Dunlavy, and Kolda 2009; Araujo, Ribeiro, and Faloutsos 2017). CPD can
capture the inherent correlations between the different modes and thanks to
its uniqueness properties it can extract interpretable latent components
rendering it an appealing solution for modeling and analyzing epidemic
dynamics. Additionally, it can parsimoniously represent multidimensional data
and can therefore learn from limited data. These two important properties
differentiate CPD from neural networks and deep learning models which
typically require a lot of training data and are often treated as black box
models.

In this work, we propose a Spatio-temporal Tensor factorization with
EpidemioLogicAl Regularization (STELAR). STELAR combines a nonnegative CPD
model with a system of discrete-time difference equations to capture the
epidemic transmission patterns. Unlike standard tensor factorization methods
which cannot predict slabs ahead, STELAR can simultaneously forecast the
evolution of the epidemic for a list of regions, and it can also perform long-
term prediction.

In the experiments, we build a spatio-temporal tensor,
$\mathrm{location}\times\mathrm{attribute}\times\mathrm{time}$ of case counts
based on large real-world medical claims datasets, where the first dimension
corresponds to different counties/states, the second dimension corresponds to
different attributes or signals that evolve over time, such as daily new
infections, deaths, number of hospitalized patients and other COVID-19 related
signals, and the third is the time-window of the available signals. This
spatio-temporal tensor is factorized using the proposed low-rank nonnegative
tensor factorization with epidemiological model regularization STELAR. We
observed that the extracted latent time components provide intuitive
interpretation of different epidemic transmission patterns which traditional
epidemiological models such as SIR and SEIR are lacking.

Our main contributions are summarized as follows:

  * •

We propose STELAR, a new data-efficient tensor factorization method
regularized by a disease transmission model in the latent domain to predict
future slabs. We show that by jointly fitting a low-rank nonnegative CPD model
with an SIR model on the time factor matrix we are able to accurately predict
the evolution of epidemic trends.

  * •

Thanks to the uniqueness properties of the CPD, our method produces
interpretable prediction results. Specifically, the tensor is approximated via
$K$ rank-1 components, each of which is associated with a sub-type of the
epidemic transmission patterns in a given list of regions. The latent time
factor matrix includes $K$ different patterns of the epidemic evolution and
using the latent location and signal factor matrices we can identify the
corresponding locations and signals associated with each pattern.

  * •

We perform extensive experimental evaluation on both county- and state-level
COVID-19 case data for $10$ and $15$ days-ahead prediction and show that our
method outperforms standard epidemiological and machine learning models in
COVID-19 pandemic prediction. Our method achieves up to $21\%$ lower root mean
square error and $25\%$ lower mean absolute error for county-level prediction.

(a) SIR model.

(b) Component $1$.

(c) Component $2$.

(d) Component $3$.

Figure 1: An example of an SIR model with parameters
$N=1,S(0)=0.95,I(0)=0.05,R(0)=0,\beta=0.4,\gamma=0.1$ is shown in Figure 1(a).
The new infections curve has been scaled by $10$ for visualization purposes.
Figures 1(b), 1(c), 1(d) show $3$ latent time components i.e., columns of the
factor matrix $\mathbf{C}$ when we compute a plain nonnegative CPD of the
spatio-temporal tensor.

## Related Work

Many epidemiological and deep learning models have been applied for modeling
the COVID-19 pandemic evolution. Methods based on traditional epidemic
prediction models such as SIR (Kermack and McKendrick 1927) and SEIR and their
variants (Cooke and Van Den Driessche 1996), rely on a system of differential
equations which describe the dynamics of the pandemic (Yang et al. 2020; Toda
2020; He, Peng, and Sun 2020). These models are trained for each location
independently by curve fitting.

Machine learning approaches and specifically deep learning models treat the
problem as a time series prediction problem. Given as input a time series of
length $L_{w}$, $x_{t-L_{w}-1},\ldots,x_{t-1},x_{t}$, the goal is to output
$L_{o}$ future time points $x_{t+1},x_{t+2},\ldots,x_{L_{o}}$. A popular and
very successful method for time series prediction is the Long Short Term
Memory (LSTM) network (Hochreiter and Schmidhuber 1997). Recent studies have
applied LSTMs for COVID-19 prediction (Chimmula and Zhang 2020; Tomar and
Gupta 2020; Yang et al. 2020). To increase the data for LSTM training, Yang et
al. used the 2003 SARS-CoV data to pretrain the model before using it for
COVID-19 prediction. DEFSI (Wang, Chen, and Marathe 2019) is a deep learning
method based on LSTMs for influenza like illness forecasting which combines
synthetic epidemic data with real data to improve epidemic forecasts.

Graph Neural Network (GNN) (Kipf and Welling 2017) is a type of neural network
which operates on a graph. Kapoor et al. proposed creating a graph with
spatial and temporal edges. They leverage human mobility data to improve the
prediction of daily new infections (Kapoor et al. 2020). STAN (Gao et al.
2020) is an attention-based graph convolutional network which constructs edges
based on geographical proximity of the different regions and regularizes the
model predictions based on an epidemiological model.

Methods based on tensor factorization have been applied for various time
series prediction tasks. CP Forecasting (Dunlavy, Kolda, and Acar 2011), is a
tensor method which computes a low-rank CPD model and uses the temporal factor
to capture periodic patterns in data. TENSORCAST (Araujo, Ribeiro, and
Faloutsos 2017) is a method that forecasts time-evolving networks using
coupled tensors. Both methods are not suitable for COVID-19 pandemic
prediction and rely on a two-step procedure – fitting a low-rank model and
then performing forecasting based on the temporal factor matrix – which as we
will see later leads to performance degradation relative to our joint
optimization formulation.

Notation | Description
---|---
$\underline{\mathbf{X}}\in\mathbb{R}^{M\times N\times L}$ | spatio-temporal tensor
$\mathbf{A}\in\mathbb{R}^{M\times K}$ | location factor matrix
$\mathbf{B}\in\mathbb{R}^{N\times K}$ | signal factor matrix
$\mathbf{C}\in\mathbb{R}^{L\times K}$ | temporal factor matrix
$\underline{\mathbf{X}}^{(i)}$ | mode-$i$ unfolding of $\underline{\mathbf{X}}$
$\beta$; $\gamma$; | contact rate; recovery rate
$K$ | # of components
$M$ | # of locations
$N$ | # of signals
$L$ | # of time points
$T$ | transpose
$\circ$ | outer product
$\otimes$ | Kronecker product
$\odot$ | Khatri-Rao product
$\circledast$ | Hadamard product
$\|\cdot\|_{F}$ | Frobenius norm
$\rm{diag}(\mathbf{x})$ | diagonal matrix
Table 1: Notation

## Background

In this section, we review necessary background on epidemiological models and
tensor decomposition that will prove useful for developing our method. Table 1
contains the notation used throughout the paper.

### Epidemiological Models

Epidemiological models have been popular solutions for pandemic modeling. For
example, The SIR model (Kermack and McKendrick 1927) is one of the most famous
and paradigmatic models in mathematical epidemiology. In this model, a
population is divided into susceptible, infected and recovered subpopulations.
The evolution of these quantities is described by the following equations:

$\displaystyle S(t)-S(t-1)=-\beta S(t-1)I(t-1)/N,$ (1a) $\displaystyle
I(t)-I(t-1)=\frac{\beta S(t-1)I(t-1)}{N}-\gamma I(t-1),$ (1b) $\displaystyle
R(t)-R(t-1)=\gamma I(t-1),$ (1c)

where $S(t)$, $I(t)$ and $R(t)$ stand for the size of susceptible, infected
and recovered populations at time $t$, respectively and $N=S(t)+I(t)+R(t)$ is
the total population. The parameter $\beta$ controls the rate of spread and
the parameter $\gamma$ is the recovery rate, so its inverse $1/\gamma$
represents the average time period that an infected individual remains
infectious. In Equation (1a), the quantity $\beta S(t)/N$ is the fraction of
susceptible individuals that will contact an infected individual at time $t$.
Therefore,

$C(t):=\beta S(t)I(t)/N,$ (2)

denotes the new infections at time $t$. In Equation (1b), $\gamma I(t)$
denotes the number of individuals recovered. Therefore, the change of the
infected cases at time $t$ will be the difference between new infected and
recovered cases as shown in Equation (1b). Given $S(0)$, $I(0)$ and $R(0)$, we
can compute $S(t)$, $I(t)$ and $R(t)$ for $t=1,2,\ldots$, $L$, where $L$ is
the size of the prediction window we are interested in. An example is shown in
Figure (1(a)) where we have set
$N=1,S(0)=0.95,I(0)=0.05,R(0)=0,\beta=0.4,\gamma=0.1$ i.e., we have normalized
the different populations by the total population number $N$ and set the
window size to be $L=50$.

The SEIR model is a variant of the SIR model. In this model, a population is
divided into susceptible, exposed, infected and recovered subpopulations. The
exposed population consists of individuals who have been infected but are not
yet infectious. The SEIR model includes an additional parameter $\sigma$ which
is the rate at which the exposed individuals becoming infectious.

### Canonical Polyadic Decomposition

The Canonical Polyadic Decomposition (CPD) expresses a $3$-way tensor
$\underline{\mathbf{X}}\in\mathbb{R}^{M\times N\times L}$ as a sum of rank-$1$
components, i.e.,

$\underline{\mathbf{X}}=[\\![\mathbf{A},\mathbf{B},\mathbf{C}]\\!]=\sum_{k=1}^{K}\mathbf{a}_{k}\circ\mathbf{b}_{k}\circ\mathbf{c}_{k},$
(3)

where $\mathbf{A}=[\mathbf{a}_{1},\ldots,\mathbf{a}_{K}]\in\mathbb{R}^{M\times
K}$, $\mathbf{B}=[\mathbf{b}_{1},\ldots,\mathbf{b}_{K}]\in\mathbb{R}^{N\times
K}$, $\mathbf{C}=[\mathbf{c}_{1},\ldots,\mathbf{c}_{K}]\in\mathbb{R}^{L\times
K}$. The rank $K$ is the minimum number of components needed to synthesize
$\underline{\mathbf{X}}$. We can express the CPD of a tensor in many different
ways. For example,
$\underline{\mathbf{X}}=[\\![\mathbf{A},\mathbf{B},\mathbf{C}]\\!]$ can be
represented by the matrix unfolding
$\underline{\mathbf{X}}^{(1)}=(\mathbf{C}\odot\mathbf{B})\mathbf{A}^{T}$ where
the mode-$1$ fibers are the rows of the resulting matrix. Using ‘role
symmetry’, the mode-$2$ and mode-$3$ matrix unfoldings are given by
$\underline{\mathbf{X}}^{(2)}=(\mathbf{C}\odot\mathbf{A})\mathbf{B}^{T}$,
$\underline{\mathbf{X}}^{(3)}=(\mathbf{B}\odot\mathbf{A})\mathbf{C}^{T}$
respectively. CPD can parsimoniously represent tensors of size $M\times
N\times L$ using only $(M+N+L)\times K$ parameters. The CPD model has two very
important properties that make it a very powerful tool for data analysis 1) it
is universal, i.e., every tensor admits a CPD of finite rank, and 2) it is
unique under mild conditions i.e., it is possible to extract the true latent
factors that synthesize $\underline{\mathbf{X}}$.

## Method

### Problem Formulation

Consider a location where we monitor $N$ signals related to a pandemic over
time, e.g., number of new infections, hospitalized patients, intensive care
unit (ICU) patients, etc. At time $t$, the value of the $n$th signal at
location $m$ is denoted by $x_{m,n,t}$. Assuming that there are $N$ signals,
$M$ locations and $L$ time points, then, the dataset can be naturally
described by a $3$-way spatio-temporal tensor
$\underline{\mathbf{X}}\in\mathbb{R}^{M\times N\times L}$, where
$\underline{\mathbf{X}}(m,n,t):=x_{m,n,t}$. Tensor $\underline{\mathbf{X}}$
includes the evolution of all signals and regions for times $1$ through $L$
and we are interested in estimating the signals at time $L+1,\ldots,L+L_{o}$.
In other words, we want to predict the frontal slabs
$\underline{\mathbf{X}}(:,:,t)$ for $L_{o}$ timesteps ahead. However, standard
tensor factorization methods cannot predict slabs ahead. It is evident from
Equation 3 and the mode-$3$ unfolding that is impossible to impute when an
entire slab $\underline{\mathbf{X}}(:,:,t)$ is missing since we have no
information regarding the corresponding row of $\mathbf{C}$. To address this
challenge, we take into consideration the transmission law dynamics of the
disease. The key idea is to decompose the tensor using a CPD model and impose
SIR constraints on the latent time factor.

To illustrate the key idea, we perform a preliminary experiment using a
rank-$5$ plain nonnegative CPD on a spatio-temporal tensor
$\underline{\mathbf{X}}\in\mathbb{R}^{140\times 15\times 82}$ with case
counts, constructed from real COVID-19 data. As shown in Figures 1(b), 1(c),
1(d), CPD is able to extract meaningful latent components from the data.
Specifically, the figures show $3$ columns of the latent time factor
$\mathbf{C}$ where an SIR model has been fitted after obtaining the
decomposition. We observe that each columns depicts a curve similar to the one
in Figure 1(a) i.e., the CPD can unveil the principal patterns of the epidemic
evolution, and each pattern corresponds to a different pandemic phase.

Therefore, we propose solving the following constrained nonnegative CPD
problem

$\displaystyle\min_{\begin{subarray}{c}\mathbf{A},\mathbf{B},\mathbf{C},\\\
\boldsymbol{\beta},\boldsymbol{\gamma},\mathbf{s},\mathbf{i}\end{subarray}}~{}$
$\displaystyle\left\|\underline{\mathbf{X}}-[\\![\mathbf{A},\mathbf{B},\mathbf{C}]\\!]\right\|_{F}^{2}+\mu\left(\|\mathbf{A}\|_{F}^{2}+\|\mathbf{B}\|_{F}^{2}+\|\mathbf{C}\|_{F}^{2}\right)$
$\displaystyle+\nu\sum_{k=1}^{K}\sum_{t=1}^{L}\left(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1)\right)^{2}$
s. t.
$\displaystyle\mathbf{A}\geq\mathbf{0},\mathbf{B}\geq\mathbf{0},\mathbf{C}\geq\mathbf{0},$
$\displaystyle\boldsymbol{\beta}\geq\mathbf{0},\boldsymbol{\gamma}\geq\mathbf{0},\mathbf{s}\geq\mathbf{0},\mathbf{i}\geq\mathbf{0},$
(4) $\displaystyle S_{k}(t)=S_{k}(t-1)-\beta_{k}S_{k}(t-1)I_{k}(t-1),$
$\displaystyle I_{k}(t)=I_{k}(t-1)+\beta_{k}S_{k}(t-1)I_{k}(t-1)$
$\displaystyle\qquad\quad-\gamma_{k}I_{k}(t-1),$ $\displaystyle
s_{k}=S_{k}(0),i_{k}=I_{k}(0).$

The first term is the data fitting term. We fit a CPD model of rank-$K$ with
nonnegativity constraints on the factors. The second term is Frobenius norm
regularization which is typically used to avoid overfitting and improve
generalization of the model. We introduce a third term which regularizes each
column of the factor matrix $\mathbf{C}$ according to Equation (2) i.e., we
learn $K$ different SIR models, each of which is fully described by parameters
$S_{k}(0),I_{k}(0),R_{k}(0)$ and $\beta_{k}$, $\gamma_{k}$ according to
Equations (1a), (1b), (1c). We aim at estimating these parameters such that
each column of $\mathbf{C}$ follows the new infections curve of an SIR model.
Vectors $\boldsymbol{\beta}\in\mathbb{R}^{K}$ and
$\boldsymbol{\gamma}\in\mathbb{R}^{K}$ hold the $\beta$ and $\gamma$
parameters of each SIR model and vectors $\mathbf{s}=\in\mathbb{R}^{K}$,
$\mathbf{i}\in\mathbb{R}^{K}$ hold the initial conditions of each SIR model.
Note that the recovered cases curve can be safely removed because it does not
affect the optimization cost and that the parameter $N_{k}$ corresponding to
the total population has been absorbed in the vector $\boldsymbol{\beta}$.

### Prediction

After the convergence of the above optimization algorithm, we have some
estimates of $\hat{\mathbf{A}}$, $\hat{\mathbf{B}}$, $\hat{\mathbf{C}}$ and
parameters $\\{\beta_{1},\cdots,\beta_{K}\\}$,
$\\{\gamma_{1},\cdots,\gamma_{K}\\}$, $\\{s_{1},\cdots,s_{K}\\}$,
$\\{i_{1},\cdots,i_{K}\\}$ where the pair of $\beta_{k}$ and $\gamma_{k}$
describes the epidemic transmission of the $k$th component in the time factor
matrix and $s_{k}$, $i_{k}$ the initial values of the subpopulations. Using
Equations (1a),(1b) and (2) we can predict “future” values for the $k$th
column of $\mathbf{C}$. We repeat the same procedure for all columns of
$\mathbf{C}$ such that we can predict the entire “future” rows. Let
$\hat{\mathbf{C}}(t,:)\in\mathbb{R}^{K}$ be the prediction of the temporal
information at a future time point $t$ using estimates
$\hat{\boldsymbol{\beta}}$, $\hat{\boldsymbol{\gamma}}$,
$\hat{\boldsymbol{s}},\hat{\boldsymbol{i}}$. Since $\mathbf{A}$ and
$\mathbf{B}$ do not depend on $t$, the prediction of all signals in the tensor
at time $t$ is given by

$\displaystyle\widehat{\underline{\mathbf{X}}}(:,:,t)=\hat{\mathbf{A}}\mathrm{diag}(\hat{\mathbf{C}}(t,:)\hat{\mathbf{B}}^{T}.$
(5)

Adding latent temporal regularization through an SIR model offers significant
advantages compared to having separate SIR models for each location. Our model
can capture correlations between different locations and signals through their
latent representations and therefore improve the prediction accuracy.
Additionally, it enables expressing the evolution of a signal as weighted sum
of $K$ separate SIR models e.g., the prediction of the $n$th signal for the
$m$th location for time point $t$, is given by

$\widehat{\underline{\mathbf{X}}}(m,n,t)=\sum_{k=1}^{K}\hat{a}_{m,k}\hat{b}_{n,k}\hat{\beta}_{k}S_{k}(t-1)I_{k}(t-1),$
(6)

which makes it much more flexible and expressive.

### Optimization

The optimization problem (Problem Formulation) is a nonconvex and very
challenging optimization problem. To update factor matrices
$\mathbf{A},\mathbf{B},\mathbf{C}$ we rely on alternating optimization. Note
that by fixing all variables except for $\mathbf{A}$, the resulting subproblem
w.r.t. $\mathbf{A}$, is a nonnegative least squares problem, which is convex.
Similarly for the factor matrices $\mathbf{B}$ and $\mathbf{C}$. We choose to
solve each factor matrix subproblem via the Alternating Direction Method of
Multipliers (ADMM) (Gabay and Mercier 1976), which is a very efficient
algorithm that has been successfully applied to nonnegative tensor
factorization problems (Huang, Sidiropoulos, and Liavas 2016).

Let us first consider the subproblem w.r.t. $\mathbf{A}$. Assume that at the
$\ell$th iteration, we have some estimates of all the variables available.
Fixing all variables except for $\mathbf{A}$, we have

$\min_{\mathbf{A}\geq\mathbf{0}}\|\underline{\mathbf{X}}^{(1)}-\mathbf{\Phi}_{A}^{(\ell)}\mathbf{A}^{T}\|_{F}^{2}+\mu\|\mathbf{A}\|_{F}^{2}$
(7)

where
$\mathbf{\Phi}_{A}^{(\ell)}=\mathbf{C}^{(\ell)}\odot\mathbf{B}^{(\ell)}$. The
ADMM updates for optimization problem (7) are the following

$\displaystyle\widehat{\mathbf{A}}=\operatorname*{arg\,min}\|\underline{\mathbf{X}}^{(1)}-\mathbf{\Phi}_{A}^{(\ell)}\widehat{\mathbf{A}}\|_{F}^{2}+\mu\|\widehat{\mathbf{A}}\|_{F}^{2}$
(8a)
$\displaystyle\quad\quad\quad+\rho\|\mathbf{A}-\widehat{\mathbf{A}}^{T}+\mathbf{A}_{d}\|_{F}^{2},$
$\displaystyle\mathbf{A}=\operatorname*{arg\,min}_{\mathbf{A}\geq\mathbf{0}}\rho\|\mathbf{A}-\widehat{\mathbf{A}}^{T}+\mathbf{A}_{d}\|_{F}^{2},$
(8b)
$\displaystyle\mathbf{A}_{d}=\mathbf{A}_{d}+\mathbf{A}-\widehat{\mathbf{A}}^{T}.$
(8c)

Equation (8a) is a least squares problem. Because ADMM is an iterative
algorithm and the update is performed multiple times, we save computations by
caching (Huang, Sidiropoulos, and Liavas 2016). Equation (8b) is a simple
element-wise nonnegative projection operator and Equation (8c) is the dual
variable update. The updates for $\mathbf{B}$ are similar.

$\displaystyle\widehat{\mathbf{B}}=\operatorname*{arg\,min}\|\underline{\mathbf{X}}^{(2)}-\mathbf{\Phi}_{B}^{(\ell)}\widehat{\mathbf{B}}\|_{F}^{2}+\mu\|\widehat{\mathbf{B}}\|_{F}^{2}$
(9a)
$\displaystyle\quad\quad\quad+\rho\|\mathbf{B}-\widehat{\mathbf{B}}^{T}+\mathbf{B}_{d}\|_{F}^{2},$
$\displaystyle\mathbf{B}=\operatorname*{arg\,min}_{\mathbf{B}\geq\mathbf{0}}\rho\|\mathbf{B}-\widehat{\mathbf{B}}^{T}+\mathbf{B}_{d}\|_{F}^{2},$
(9b)
$\displaystyle\mathbf{B}_{d}=\mathbf{B}_{d}+\mathbf{B}-\widehat{\mathbf{B}}^{T}.$
(9c)

where
$\mathbf{\Phi}_{B}^{(\ell)}=\mathbf{C}^{(\ell)}\odot\mathbf{A}^{(\ell)}$. Now
let us consider the update of $\mathbf{C}$. The related optimization problem
takes the form of

$\min_{\mathbf{C}\geq
0}\|\underline{\mathbf{X}}^{(3)}-\mathbf{\Phi}_{C}^{(\ell)}\mathbf{C}^{T}\|_{F}^{2}+\mu\|\mathbf{C}\|_{F}^{2}+\nu\|\mathbf{C}-\bar{\mathbf{C}}^{(\ell)}\|_{F}^{2},$
(10)

where
$\mathbf{\Phi}_{C}^{(\ell)}=\mathbf{B}^{(\ell)}\odot\mathbf{A}^{(\ell)}$,
$\bar{\mathbf{C}}^{(\ell)}=(\mathbf{P}^{(\ell)}\circledast\mathbf{Q}^{(\ell)})\mathrm{diag}(\boldsymbol{\beta}^{(\ell)})$
and we define $\mathbf{P}^{(\ell)}(t,k):=S_{k}(t-1)$ and
$\mathbf{Q}^{(\ell)}(t,k):=I_{k}(t-1)$. Optimization problem (10) is also a
nonnegative least squares problem. Therefore the updates for $\mathbf{C}$ are

$\displaystyle\widehat{\mathbf{C}}=\operatorname*{arg\,min}\|\underline{\mathbf{X}}^{(3)}-\mathbf{\Phi}_{C}^{(\ell)}\widehat{\mathbf{C}}\|_{2}^{2}+\mu\|\widehat{\mathbf{C}}\|_{F}^{2}$
(11a)
$\displaystyle\quad\quad\quad+\nu\|\mathbf{C}-\bar{\mathbf{C}}^{(\ell)}\|_{F}^{2}+\rho\|\mathbf{C}-\widehat{\mathbf{C}}^{T}+\mathbf{C}_{d}\|_{F}^{2}$
$\displaystyle\mathbf{C}=\operatorname*{arg\,min}_{\mathbf{C}\geq\mathbf{0}}\rho\|\mathbf{C}-\widehat{\mathbf{C}}^{T}+\mathbf{C}_{d}\|_{F}^{2}$
(11b)
$\displaystyle\mathbf{C}_{d}=\mathbf{C}_{d}+\mathbf{C}-\widehat{\mathbf{C}}^{T}$
(11c)

We observed that running a few ADMM inner iterations ($\sim 10$) for each
factor suffices for the algorithm to produce satisfactory results.

Algorithm 1 STELAR Method

Input: Tensor $\underline{\mathbf{X}}$, rank $K$, max. outer iterations
$\rm{iters}_{\rm outer}$, max. inner iterations $\rm{iters}_{\rm inner}$,
gradient steps $\rm iters_{\rm grad}$, prediction window $L_{o}$

repeat

Update $\mathbf{A}$ using (8) for $\rm{iters}_{\rm inner}$ iterations

Update $\mathbf{B}$ using (9) for $\rm{iters}_{\rm inner}$ iterations

Update $\mathbf{C}$ using (11) for $\rm{iters}_{\rm inner}$ iterations

Perform $\rm iters_{\rm grad}$ projected gradient steps for
$\boldsymbol{\beta}$, $\boldsymbol{\gamma}$, $\mathbf{s}$, $\mathbf{i}$, using
(14), (15), (16), (17)

until $\rm{iters}_{\rm outer}$ or validation RMSE increases

Predict $L_{o}$ future slabs using Equation (5)

By fixing the factor matrices, we have

$\displaystyle\min_{\boldsymbol{\beta},\boldsymbol{\gamma},\mathbf{s},\mathbf{i}}{}$
$\displaystyle\sum_{k=1}^{K}\sum_{t=1}^{L}\left(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1)\right)^{2}$
(12) s. t.
$\displaystyle\boldsymbol{\beta}\geq\mathbf{0},\boldsymbol{\gamma}\geq\mathbf{0},\mathbf{s}\geq\mathbf{0},\mathbf{i}\geq\mathbf{0},$
$\displaystyle S_{k}(t)=S_{k}(t-1)-\beta_{k}S_{k}(t-1)I_{k}(t-1),$
$\displaystyle I_{k}(t)=I_{k}(t-1)+\beta_{k}S_{k}(t-1)I_{k}(t-1)$
$\displaystyle\qquad\quad-\gamma_{k}I_{k}(t-1),$ $\displaystyle
s_{k}=S_{k}(0),i_{k}=I_{k}(0).$

Both $S_{k}(t)$ and $I_{k}(t)$ are functions of $\boldsymbol{\beta}$ and
$\boldsymbol{\gamma}$, and are calculated in a recursive manner. Therefore the
optimization problem w.r.t. $\boldsymbol{\beta}$ and $\boldsymbol{\gamma}$ is
nonconvex. Optimization problem (12) corresponds to $K$ independent one-
dimensional curve fitting problems and since there are only $4$ optimization
variables for each problem, we can use off-the-shelf curve fitting methods.
Alternatively, we can perform a few projected gradient descent steps. Focusing
on the $k$th subproblem we have

$\displaystyle
f(\beta_{k})=\nu\sum_{t=1}^{L}(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1))^{2}.$
(13)

The derivative of $f(\beta_{k})$ w.r.t. $\beta_{k}$ is

$\displaystyle\frac{\partial
f}{\partial\beta_{k}}=-2\nu\sum_{t=1}^{L}(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1))\times$
(14) $\displaystyle\Bigl{(}S_{k}(t-1)I_{k}(t-1)+\beta_{k}\frac{\partial
S_{k}(t-1)}{\partial\beta_{k}}I_{k}(t-1)+$
$\displaystyle\quad\quad\quad\quad\quad\quad\beta_{k}S_{t-1,k}\frac{\partial
I_{t-1,k}}{\partial\beta_{k}}\Bigr{)}.$

Note that both $S_{k}(t)$ and $I_{k}(t)$ are recursive functions. Thus, their
respective derivatives w.r.t. $\beta$ are computed recursively in $L$ steps.
Similarly, for $\gamma_{k}$ we have

$\displaystyle\frac{\partial
f}{\partial\gamma_{k}}=-2\nu\sum_{t=1}^{L}(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1))\times$
(15) $\displaystyle\left(\beta_{k}\frac{\partial
S_{t-1}}{\partial\gamma}I_{t-1}+\beta_{k}S_{k}(t-1)\frac{\partial
I_{k}(t-1)}{\partial\gamma}\right).$

Finally for $\mathbf{s},\mathbf{i}$

$\displaystyle\frac{\partial f}{\partial
s_{1}}=-2\nu\sum_{t=1}^{L}(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1))\times$ (16)
$\displaystyle\Bigl{(}\beta_{k}\frac{\partial S_{k}(t-1)}{\partial
s_{1}}I_{k}(t-1)+\beta_{k}S_{k}(t-1)\frac{\partial I_{k}(t-1)}{\partial
s_{1}}\Bigr{)},$ $\displaystyle\frac{\partial f}{\partial
i_{1}}=-2\nu\sum_{t=1}^{L}(c_{t,k}-\beta_{k}S_{k}(t-1)I_{k}(t-1))\times$ (17)
$\displaystyle\left(\beta_{k}\frac{\partial S_{k}(t-1)}{\partial
i_{1}}I_{k}(t-1)+\beta_{k}S_{k}(t-1)\frac{\partial I_{k}(t-1)}{\partial
i_{1}}\right).$

The overall procedure is summarized in Algorithm 1.

| $L_{o}=10$ | $L_{o}=15$
---|---|---
Model | RMSE | MAE | RMSE | MAE
Mean | $304.1$ | $122.0$ | $269.5$ | $108.5$
SIR | $156.2$ | $62.2$ | $159.1$ | $63.6$
SEIR | $177.1$ | $72.9$ | $163.2$ | $69.7$
LSTM (w/o feat.) | $203.6$ | $77.1$ | $191.0$ | $81.7$
LSTM (w/ feat.) | $162.3$ | $68.2$ | $187.6$ | $78.3$
STAN | $164.2$ | $61.1$ | $152.6$ | $61.8$
STELAR ($\nu=0$) | $149.2$ | $61.5$ | $152.8$ | $66.9$
STELAR | $\mathbf{127.5}$ | $\mathbf{55.6}$ | $\mathbf{136.1}$ | $\mathbf{61.7}$
| $L_{o}=10$ | $L_{o}=15$
---|---|---
Model | RMSE | MAE | RMSE | MAE
Mean | $125.0$ | $77.0$ | $123.3$ | $77.1$
SIR | $46.5$ | $27.2$ | $48.7$ | $27.7$
SEIR | $39.1$ | $23.9$ | $41.1$ | $25.7$
LSTM (w/o feat.) | $45.6$ | $23.6$ | $54.8$ | $31.2$
LSTM (w/ feat.) | $42.5$ | $23.3$ | $47.5$ | $26.8$
STAN | $30.6$ | $17.3$ | $42.8$ | $24.2$
STELAR ($\nu=0$) | $28.6$ | $16.6$ | $46.8$ | $21.1$
STELAR | $\mathbf{24.0}$ | $\mathbf{15.1}$ | $\mathbf{36.0}$ | $\mathbf{18.0}$

Table 2: County-level prediction for new infections (left) and hospitalized
patients (right).

| $L_{o}=10$ | $L_{o}=15$
---|---|---
Model | RMSE | MAE | RMSE | MAE
Mean | $309.0$ | $258.7$ | $325.8$ | $273.1$
SIR | $186.1$ | $133.8$ | $186.9$ | $134.5$
SEIR | $162.4$ | $127.0$ | $162.6$ | $130.2$
LSTM (w/o feat.) | $187.5$ | $138.1$ | $419.7$ | $356.0$
LSTM (w/ feat.) | $197.9$ | $151.6$ | $359.2$ | $286.5$
STAN | $\mathbf{74.1}$ | $\mathbf{60.1}$ | $\mathbf{100.5}$ | $79.6$
STELAR ($\nu=0$) | $140.8$ | $104.0$ | $127.8$ | $95.0$
STELAR | $117.8$ | $89.8$ | $107.3$ | $\mathbf{79.4}$
| $L_{o}=10$ | $L_{o}=15$
---|---|---
Model | RMSE | MAE | RMSE | MAE
Mean | $685.5$ | $553.9$ | $729.2$ | $586.0$
SIR | $343.4$ | $252.4$ | $367.8$ | $266.0$
SEIR | $109.0$ | $97.6$ | $192.5$ | $154.3$
LSTM (w/o feat.) | $280.9$ | $187.5$ | $416.1$ | $308.7$
LSTM (w/ feat.) | $295.3$ | $182.3$ | $276.0$ | $208.5$
STAN | $100.3$ | $73.6$ | $177.7$ | $144.7$
STELAR ($\nu=0$) | $118.3$ | $84.4$ | $126.6$ | $\mathbf{75.2}$
STELAR | $\mathbf{56.8}$ | $\mathbf{43.9}$ | $\mathbf{113.6}$ | $83.8$

Table 3: State-level prediction for new infections (left) and hospitalized
patients (right).

(a) Palm Beach, Florida

(b) Jefferson, Alabama

(c) Franklin, Ohio

Figure 2: $10$ days county-level prediction for new infections.

## Experiments

### Dataset and Baselines

We use US county-level data from the Johns Hopkins University (JHU) (Dong, Du,
and Gardner 2020) and a large IQVIA patient claims dataset, which can be
publicly accessible upon request. JHU data includes the number of active
cases, confirmed cases and deaths for different counties in the US. The total
number of counties was $133$. We use the reported active cases to compute the
daily new infections. The claims dataset was created from 582,2748 claims from
732,269 COVID-19 patients from 03-24-2020 to 06-26-2020 (95 days). It contains
the daily counts of $12$ International Classification of Diseases ICD-10 codes
observed in each county and the Current Procedural Terminology (CPT) codes
related to hospitalization and utilization of intensive care unit (ICU). The
size of the constructed tensor is $133\times 15\times 95$. Using this data, we
also construct a tensor which includes the aggregated data for $5$ states, New
York, New Jersey, Massachusetts, Connecticut, Pennsylvania and is of size
$5\times 15\times 95$.

We perform two different experiments. Initially we use $85$ days for training
and validation and use the remaining $L_{o}=10$ as the test set. In the second
experiment we use $80$ days for training and validation and $L_{o}=15$ days
for test. We compare our method against the following baselines:

  1. 1.

Mean. We use the mean of the last $5$ days of the training set as our
prediction.

  2. 2.

SIR. The susceptible-infected-removed model.

  3. 3.

SEIR. The susceptible-exposed-infected-removed epidemiological model.

  4. 4.

LSTM (w/o feat.) LSTM model without additional features. We use one type of
time series as our input.

  5. 5.

LSTM (w feat.) LSTM with additional features. We use $15$ different time
series as input.

  6. 6.

STAN (Gao et al. 2020) A recently proposed GNN with attention mechanism.

  7. 7.

STELAR ($\nu=0$) The proposed method without regularization (two-step
approach).

We use the Root Mean Square Error (RMSE) and Mean Absolute Error (MAE) as our
evaluation metrics.

### Results

Table 2 shows the county-level results for $10$ and $15$ day prediction for
daily new infections and hospitalized patients. For new infections and
$L_{o}=10$, our method achieves $18\%$ lower RMSE and $9\%$ lower MAE compared
to the best performing baselines which are the SIR and STAN model
respectively. When $L_{o}=15$ our method achieves $10\%$ lower RMSE and the
same MAE compared to the STAN model. For hospitalized patients and $L_{o}=10$,
our method achieves $21\%$ lower RMSE and $12\%$ lower MAE compared to the
best performing baseline which is STAN. When $L_{o}=15$ our method achieves
$12\%$ lower RMSE and $25\%$ lower MAE compared to the best baseline. For
state-level prediction of new infections, the best performing model is STAN
but for hospitalized patients our method again outperforms all the baselines.
Note that in all cases except one, joint optimization and SIR model fitting
always improves the performance of our method compared to the two-step
procedure.

Figure 2 shows some examples of county-level prediction. Figure 2(a) shows
simple case where one can observe an increasing pattern of the new infections.
Almost all models are able to capture this trend except for the LSTM (w/feat.)
model. Figure 2(b) shows a more challenging scenario where it is not obvious
if the curve will continue increasing but our method makes an accurate
prediction, and is better than the baselines. Finally in Figure 2(c) we
observe an example where a peak was already observed in the past and therefore
SIR and SEIR models fail. On the other hand, our method is again able to make
reasonable predictions.

Component 1 | Component 2 | Component 3
---|---|---
New York (NY) | L.A (CA) | Nassau (NY)
Westchester (NY) | Cook (IL) | L.A (CA)
Nassau (NY) | Milwaukee (WI) | Essex (NJ)
Bergen (NJ) | Fairfax (VA) | Wayne (MI)
Miami-Dade (FL) | Hennepin (MN) | Oakland (MI)
Hudson (NJ) | Montg. (MD) | Middlesex (NJ)
Union (NJ) | P. George’s (MD) | New York (NY)
Phila. (PA) | Dallas (TX) | Phila. (PA)
Passaic (NJ) | Orange (CA) | Cook (IL)
Essex (NJ) | Harris (TX) | Bergen (NJ)

Table 4: Counties that contribute more to each of the strongest $3$ rank-$1$
components of a rank-$30$ STELAR model.  Figure 3: The strongest $3$ temporal
components of a rank-$30$ STELAR model.

Component 1 | Component 2 | Component 3
---|---|---
New infections | New infections | Hosp. patients
Hosp. patients | Hosp. patients | ICU patients
ICU patients | ICU patients | J96
J96 | J96 | N17
R09 | R05 | R06

Table 5: Signals that contribute more to each of the strongest $3$ rank-$1$
components of a rank-$30$ STELAR model. Definitions of ICD-10 codes:
J96–Respiratory failure, not elsewhere classified; N17–Acute kidney failure;
R05–Cough. R06–Abnormalities of breathing; R09–Other symptoms and signs
involving the circulatory and respiratory system.

Finally, we demonstrate the ability of our model to produce interpretable
results. We train our model on county-level data using $K=30$. After the
algorithm converges we normalize each factor matrix such that each column has
unit norm and absorb the scaling to a vector $w$. Using this vector, we
extract the $3$ rank-$1$ components with the highest weights i.e., by
selecting the corresponding columns of $\mathbf{A},\mathbf{B},\mathbf{C}$.
Figure 3 depicts the $3$ temporal profiles (columns of factor $\mathbf{C}$)
that corresponds to the selected rank-$1$ components. We observe that the
different curves depict different stages of the disease transmission. For
example, the $1$st component has captured an initial wave of the disease, the
$2$nd has captured a more recent increasing trend and the $3$rd component is
very similar to the first but shifted to the right. To gain some intuition
regarding the results, we sort each column of factor matrices $\mathbf{A}$
(counties) and $\mathbf{B}$ (signals) and present the top $10$ counties and
top $5$ signals that contribute to each one of the $3$ strongest components in
Tables 4, 5. We can clearly see that the counties which correspond to the
early wave ($1$st component) are from the states of New York and New Jersey as
we would expect. Also the $1$st component is mostly associated with new
infections, hospitalized patients and ICU. On the other hand, the $3$rd
component which is very similar to the $1$st is associated with hospitalized
patients and ICU but not with new infections. Some counties appear in both the
$1$st and $3$rd component which means that hospitalized patients and ICU cases
started to increase (or getting reported) slightly after the new infections
were reported. Finally, the second component depicts a later increase to the
number of infections and hospitalized patients for some counties.

## Conclusion

In this paper, we propose STELAR a data efficient and interpretable method
based on constrained nonnegative tensor factorization. Unlike standard tensor
factorization methods, our method enables long-term prediction of future slabs
by incorporating latent epidemiological regularization. We demonstrated the
ability of our method to make accurate predictions on real county- and state-
level COVID-19 data. Our method achieves $18\%$ and $10\%$ lower RMSE compared
to the best baseline, when predicting county-level daily new infections for
$10$ and $15$ days-ahead respectively and $21\%$ and $12\%$ lower RMSE when
predicting county-level hospitalized patients.

## Acknowledgments

This work is in part supported by National Science Foundation award
IIS-1704074, Army Research Office award ARO W911NF1910407, National Science
Foundation award SCH-2014438, IIS-1418511, CCF-1533768, IIS-1838042, the
National Institute of Health award NIH R01 1R01NS107291-01 and R56HL138415.

## References

  * Acar, Dunlavy, and Kolda (2009) Acar, E.; Dunlavy, D. M.; and Kolda, T. G. 2009.  Link prediction on evolving data using matrix and tensor factorizations.  In _2009 IEEE International Conference on Data Mining Workshops (ICDMW)_ , 262–269.
  * Araujo, Ribeiro, and Faloutsos (2017) Araujo, M. R.; Ribeiro, P. M. P.; and Faloutsos, C. 2017.  TensorCast: Forecasting with context using coupled tensors.  In _2017 IEEE International Conference on Data Mining (ICDM)_ , 71–80.
  * Chimmula and Zhang (2020) Chimmula, V. K. R.; and Zhang, L. 2020.  Time series forecasting of COVID-19 transmission in Canada using LSTM networks.  _Chaos, Solitons & Fractals_ 135\.
  * Cooke and Van Den Driessche (1996) Cooke, K. L.; and Van Den Driessche, P. 1996.  Analysis of an SEIRS epidemic model with two delays.  _Journal of Mathematical Biology_ 35(2): 240–260.
  * Dong, Du, and Gardner (2020) Dong, E.; Du, H.; and Gardner, L. 2020.  An interactive web-based dashboard to track COVID-19 in real time.  _The Lancet infectious diseases_ 20(5): 533–534.
  * Dunlavy, Kolda, and Acar (2011) Dunlavy, D. M.; Kolda, T. G.; and Acar, E. 2011.  Temporal link prediction using matrix and tensor factorizations.  _ACM Transactions on Knowledge Discovery from Data (TKDD)_ 5(2): 1–27.
  * Gabay and Mercier (1976) Gabay, D.; and Mercier, B. 1976.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation.  _Computers & Mathematics with Applications_ 2(1): 17–40.
  * Gao et al. (2020) Gao, J.; Sharma, R.; Qian, C.; Glass, L. M.; Spaeder, J.; Romberg, J.; Sun, J.; and Xiao, C. 2020.  STAN: Spatio-Temporal Attention Network for Pandemic Prediction Using Real World Evidence.  _arXiv preprint arXiv:2008.04215_ .
  * Harshman (1970) Harshman, R. A. 1970.  Foundations of the PARAFAC procedure: Models and conditions for an ”explanatory” multimodal factor analysis.  _UCLA Working Papers Phonetics_ 16: 1–84.
  * He, Peng, and Sun (2020) He, S.; Peng, Y.; and Sun, K. 2020.  SEIR modeling of the COVID-19 and its dynamics.  _Nonlinear Dynamics_ 1–14.
  * Hochreiter and Schmidhuber (1997) Hochreiter, S.; and Schmidhuber, J. 1997.  Long short-term memory.  _Neural computation_ 9(8): 1735–1780.
  * Huang, Sidiropoulos, and Liavas (2016) Huang, K.; Sidiropoulos, N. D.; and Liavas, A. P. 2016.  A flexible and efficient algorithmic framework for constrained matrix and tensor factorization.  _IEEE Transactions on Signal Processing_ 64(19): 5052–5065.
  * Kapoor et al. (2020) Kapoor, A.; Ben, X.; Liu, L.; Perozzi, B.; Barnes, M.; Blais, M.; and O’Banion, S. 2020.  Examining COVID-19 Forecasting using Spatio-Temporal Graph Neural Networks.  _arXiv preprint arXiv:2007.03113_ .
  * Kermack and McKendrick (1927) Kermack, W. O.; and McKendrick, A. G. 1927.  A contribution to the mathematical theory of epidemics.  _Proceedings of the royal society of london. Series A, Containing papers of a mathematical and physical character_ 115(772): 700–721.
  * Kipf and Welling (2017) Kipf, T. N.; and Welling, M. 2017.  Semi-Supervised Classification with Graph Convolutional Networks.  In _2017 International Conference on Learning Representations (ICLR)_.
  * Papalexakis, Faloutsos, and Sidiropoulos (2016) Papalexakis, E. E.; Faloutsos, C.; and Sidiropoulos, N. D. 2016.  Tensors for data mining and data fusion: Models, applications, and scalable algorithms.  _ACM Transactions on Intelligent Systems and Technology (TIST)_ 8(2): 1–44.
  * Sidiropoulos et al. (2017) Sidiropoulos, N. D.; De Lathauwer, L.; Fu, X.; Huang, K.; Papalexakis, E. E.; and Faloutsos, C. 2017.  Tensor Decomposition for Signal Processing and Machine Learning.  _IEEE Transactions on Signal Processing_ 65(13): 3551–3582.
  * Toda (2020) Toda, A. A. 2020.  Susceptible-infected-recovered (SIR) dynamics of COVID-19 and economic impact.  _arXiv preprint arXiv:2003.11221_ .
  * Tomar and Gupta (2020) Tomar, A.; and Gupta, N. 2020.  Prediction for the spread of COVID-19 in India and effectiveness of preventive measures.  _Science of The Total Environment_ 18\.
  * Wang, Chen, and Marathe (2019) Wang, L.; Chen, J.; and Marathe, M. 2019.  DEFSI: Deep learning based epidemic forecasting with synthetic information.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 33, 9607–9612.
  * Yang et al. (2020) Yang, Z.; Zeng, Z.; Wang, K.; Wong, S.-S.; Liang, W.; Zanin, M.; Liu, P.; Cao, X.; Gao, Z.; Mai, Z.; et al. 2020.  Modified SEIR and AI prediction of the epidemics trend of COVID-19 in China under public health interventions.  _Journal of Thoracic Disease_ 12(3): 165.