Homology group of branched cyclic covering over a $2$-bridge knot of genus two

Ilya Mednykh

###### Abstract

The structure of the first homology group of a cyclic covering of a knot is an
important invariant well known in the knot theory. In the last century, H.
Seifert developed a general approach to compute the homology group of the
covering. Based on his ideas R. Fox found explicit form for
$H_{1}(M_{n},\mathbb{Z}),$ where $M_{n}$ is an $n$-fold cyclic covering over a
knot $K$ admitting genus one Seifert surface.

The aim of the present paper is to find the structure of
$H_{1}(M_{n},\mathbb{Z})$ for $2$-bridge knots admitting genus two Seifert
surface. The result is given explicitly in terms of Alexander polynomial of
the knot.

Primary: 57M12; Secondary: 57K14, 39A06.

Keywords: Alexander polynomial, knot, branched covering, homology group.

##  1 Introduction

Let $K$ be a knot in the three dimensional sphere $\mathbb{S}^{3}.$ Denote by
$M_{n}$ a cyclic $n$-fold covering of $\mathbb{S}^{3}$ branched over $K.$ The
general description of the first homology group $H_{1}(M_{n},\mathbb{Z})$ was
done by Seifert [17]. To do this he introduced matrix $\Gamma$ which can be
expressed via Seifert matrix $V$ of the knot $K$ as
$\Gamma=(V-V^{\prime})^{-1}V$ where $V^{\prime}$ is a transpose of $V.$ Then
$H_{1}(M_{n},\mathbb{Z})$ as an Abelian group is represented by the matrix
$\Gamma^{n}-(\Gamma-I)^{n}.$

Recall that the Alexander polynomial of a knot $K$ is given by equality
$A(z)=\det(V-zV^{\prime}).$ We note that even in the class of $2$-bridge knots
Alexander polynomial does not determine the congruence class of matrix $V.$
For example, two $2$-bridge knots with rational slopes $17/2$ and $15/4$ share
the same Alexander polynomial $A(z)=4-7z+4z^{2},$ but have non-congruent
Seifert matrices $\left(\begin{array}[]{cc}4&1\\\ 0&1\end{array}\right)$ and
$\left(\begin{array}[]{cc}2&1\\\ 0&2\end{array}\right).$ In spite of this,
these two knots have the same first homology group of a cyclic $n$-fold
covering. See the Theorem 5.1 below.

In general, the Alexander polynomial does not designate the structure of the
first homology group $H_{1}(M_{n},\mathbb{Z}).$ Indeed, the stevedore’s knot
$6_{1}$ and the knot $9_{46}$ have the same Alexander polynomial, but in the
first case, $H_{1}(M_{2},\mathbb{Z})\cong\mathbb{Z}_{9},$ where as in the
second, $H_{1}(M_{2},\mathbb{Z})\cong\mathbb{Z}_{3}\oplus\mathbb{Z}_{3}.$

However, if we restrict ourselves to the class of $2$-bridge knots, the
situation became much more plausible. It was shown by A. Cattabriga and M.
Mulazzani [2] that the polynomial associated with $n$-fold cyclic covering
$M_{n}$ of a $2$-bridge knot coincides with the Alexander polynomial of this
knot. That is, $H_{1}(M_{n},\mathbb{Z})$ is completely determined through
Alexander polynomial of $2$-bridge knot.

In this paper, we present explicit formulas for the group
$H_{1}(M_{n},\mathbb{Z})$ for $2$-bridge knots of genus one and two by making
use of coefficients of their Alexander polynomials.

##  2 Cyclic $n$-fold covering of 2-bridge knots

For the basic definitions and facts from the knot theory we refer the reader
to the D. Rolfsen book [15] and A. Kawauchi survey [7].

Recall the following basic notions and definitions. A $2$-bridge knot (or
rational) is a knot admitting a diagram with two bridges. A full
classification of $2$-bridge knots was done by H. Schubert [16]. These knots
are uniquely defined by their rational slope $p/q,$ where $p$ and $q$ are
positive integers and $1\leq q\leq p-1.$ Two knots with rational slopes $p/q,$
and $p^{\prime}/q^{\prime},$ are equivalent if and only if $p=p^{\prime}$ and
$q\equiv-q^{\prime}$ or $q\,q^{\prime}\equiv 1\mod p.$ By M. Boileau and B.
Zimmermann [1], a knot is $2$-bridge if and only if the fundamental group of
it complement in the three dimensional sphere is generated by two meridians.

The general approach to calculate Conway and Alexander polynomials of a
$2$-bridge knot is given in [8].

A Seifert surface of a knot $K$ is an orientable surface with one boundary
component which coincides with the knot $K.$

A genus of the knot $K$ is a minimum genus of any Seifert surface for $K.$

The degree Alexander polynomial of genus $g$ knot never exceed $2g$ [17].

Let $K$ be a $2$-bridge knot and $M_{n}$ is an $n$-fold cyclic covering of the
three dimensional sphere branched over $K.$ We note that the fundamental group
$\pi_{1}(\mathbb{S}^{3}\setminus K)$ is generated by two meridians
$\texttt{x}_{0},\,\texttt{x}_{1}$ of knot $K$ and has exactly one relation
$r_{1}(\texttt{x}_{0},\,\texttt{x}_{1})=0$ whose structure is completely
determined by the slope of the knot [16]. Denote by
$A(z)=\sum_{\ell=0}^{s}a_{\ell}z^{\ell}$ the Alexander polynomial of knot $K$
normalizied by conditions $a_{0}\neq 0,\,a_{s}\neq 0$ and $A(1)=1.$ According
to ([4], p. 416) the first homology group of $M_{n}$ is an Abelian group
represented by matrix $A(T_{n}),$ where $T_{n}=circ(0,1,\ldots,0)$ is an
$n\times n$ circulant matrix. Thereby, $\mathcal{H}=H_{1}(M_{n},\mathbb{Z})$
has the following representation

$\mathcal{H}=\big{\langle}x_{1},\,x_{2},\ldots,x_{n}|A(T_{n})(x_{1},x_{2},\ldots,x_{n})^{t}=0\big{\rangle}.$

This leads to equivalent infinite representations

$\mathcal{H}=\big{\langle}x_{j},\,j\in\mathbb{Z}|\sum_{\ell=0}^{s}a_{\ell}x_{i+\ell}=0,\,x_{i+n}=x_{i},\,i\in\mathbb{Z}\big{\rangle}$

We set

$U=\left(\begin{array}[]{cccc}0&-1&\ldots&0\\\ 0&0&&0\\\
\vdots&&\ddots&\vdots\\\ 0&0&&-1\\\
a_{0}&a_{1}&\ldots&a_{s-1}\end{array}\right)\text{ and
}V=\left(\begin{array}[]{ccccc}1&0&\ldots&0&0\\\ 0&1&&0&0\\\
\vdots&&\ddots&&\vdots\\\ 0&0&&1&0\\\ 0&0&\ldots&0&a_{s}\end{array}\right).$

Then we can rewrite the representation of group $\mathcal{H}$ as

$\mathcal{H}=\big{\langle}x_{j},\,j\in\mathbb{Z}|\,U\,X_{i}+V\,X_{i+1}=0,X_{i+n}=X_{i},\,i\in\mathbb{Z}\big{\rangle},$

where $X_{i}=(x_{i},x_{i+1},\ldots,x_{i+s-1})^{t}$ is a column vector of
length $s.$ We note that $U+V=\left(\begin{array}[]{ccccc}1&-1&\ldots&0&0\\\
0&1&&0&0\\\ \vdots&&\ddots&&\vdots\\\ 0&0&&1&-1\\\
a_{0}&a_{1}&\ldots&a_{s-2}&a_{s-1}+a_{s}\end{array}\right).$ By induction we
have

$\det(U+V)=a_{0}+a_{1}+\ldots+a_{s-1}+a_{s}=A(1)=1.$

Therefore both $U+V$ and $(U+V)^{-1}$ are integer unimodular matrices. Next,
we multiply each of the relations $U\,X_{i}+V\,X_{i+1}=0,\,i\in\mathbb{Z}$
from left-hand side by $(U+V)^{-1}$ to obtain

$\mathcal{H}=\big{\langle}x_{j},\,j\in\mathbb{Z}|\Gamma\,X_{i}-(\Gamma-I)\,X_{i+1}=0,X_{i+n}=X_{i},\,i\in\mathbb{Z}\big{\rangle},$

where $\Gamma=(U+V)^{-1}U.$

We finish the section with the following lemma attributed to H. Seifert ([17],
p. 577, Satz 1).

######  Lemma 2.1.

Let $K$ be a two-bridge knot with the Alexander polynomial $A(z).$ Denote by
$\mathcal{H}=H_{1}(M_{n},\mathbb{Z})$ the first homology group of the cyclic
$n$-fold covering $M_{n}$ of the three-dimensional sphere $\mathbb{S}^{3}$
branched over $K.$ Then the Abelian group $\mathcal{H}$ has the following
matrix of relations $\Gamma^{n}-(\Gamma-I)^{n}.$

One can consider the matrices $U$ and $V$ as combinatorial analogues of the
Seifert matrix introduced in [17]. The matrix $\Gamma$ was also introduced by
Seifert as $\Gamma=(U+V)^{-1}U$ by setting $V=-U^{t}.$ We also note that the
Alexander polynomial can be expressed as $A(z)=\det(U+z\,V).$

##  3 Preliminary observations

Consider a $2$-bridge knot $K$ whose Alexander polynomial has the form
$A(z)=a+(b-a)z+(1-2b)z^{2}+(b-a)z^{3}+az^{4}.$ This is a general form for a
knot of Seifert genus $2.$ (See, for example, [3]). Denote by $M_{n}$ a cyclic
$n$-fold covering of the three sphere $\mathbb{S}^{3}$ branched over knot $K.$
The aim of this paper is to describe explicitly the structure of the first
homology group $H_{1}(M_{n},\mathbb{Z})$ for any natural $n.$

We refer to the content of the previous section to get the following
combinatorial version of Seifert matrix

$\Gamma=\left(\begin{array}[]{cccc}a&-1+b&-b&-a\\\ a&b&-b&-a\\\ a&b&1-b&-a\\\
a&b&1-b&1-a\end{array}\right).$

By making use of Lemma 2.1, we conclude that the Abelian group
$H_{1}(M_{n},\mathbb{Z})$ is represented by the matrix

$B(n)=\Gamma^{n}-(\Gamma-I)^{n}.$ (1)

See also [3] for more detailed explanation. One can consider matrix $B(n)$ as
$\mathbb{Z}$-linear operator from $\mathbb{Z}^{4}$ to $\mathbb{Z}^{4}$ acting
by the rule $B(n):x\rightarrow B(n)x,\,x\in\mathbb{Z}^{4}.$ Define
$\textrm{coker}\,B(n)$ as the factor space $\mathbb{Z}^{4}/\textrm{im}\,B(n),$
where $\textrm{im}\,B(n)$ is the image of $B(n)$ in $\mathbb{Z}^{4}.$ Then one
has

$H_{1}(M_{n},\mathbb{Z})=\textrm{coker}\,B(n).$

The structure of the Abelian group $\textrm{coker}\,B(n)$ is completely
determined by the Smith Normal Form of matrix $B(n).$ More precisely,

$\textrm{coker}\,B(n)=\mathbb{Z}_{d_{1}}\oplus\mathbb{Z}_{{d_{2}}/{d_{1}}}\oplus\mathbb{Z}_{{d_{3}}/{d_{2}}}\oplus\mathbb{Z}_{{d_{4}}/{d_{3}}},$

where $d_{k}$ is the greatest common divisor of all $k\times k$ minors of
matrix $B(n).$ By definition, $\mathbb{Z}_{d}$ is the factor group
$\mathbb{Z}/d\,\mathbb{Z},$ where $d$ is an integer number. It can be
positive, negative or zero. In particular, $\mathbb{Z}_{0}=\mathbb{Z}$ and
$\mathbb{Z}_{-5}=\mathbb{Z}_{5}.$

##  4 Recurrent sequences associated with Alexander polynomial.

Consider the matrix $\Gamma$ from previous section. Characteristic polynomials
for $\Gamma$ and $\Gamma-I$ are
$P_{0}(\lambda)=a-(3a+b)\lambda+(1+3a+b)\lambda^{2}-2\lambda^{3}+\lambda^{4}$
and
$P_{1}(\lambda)=a+(3a+b)\lambda+(1+3a+b)\lambda^{2}+2\lambda^{3}+\lambda^{4}$
respectively. Their product is
$P_{0}(\lambda)P_{1}(\lambda)=a^{2}+(2a-3a^{2}-4ab-b^{2})\lambda^{2}+(1-4a+9a^{2}-2b+6ab+b^{2})\lambda^{4}+(-2+6a+2b)\lambda^{6}+\lambda^{8}.$

Hence, each entry of matrix $B(n)$ and the matrix $B(n)$ itself satisfy the
following recursion

$\displaystyle a^{2}u(n)+(2a-3a^{2}-4ab-b^{2})u(n+2)$
$\displaystyle+(1-4a+9a^{2}-2b+6ab+b^{2})u(n+4)$ (2)
$\displaystyle+(-2+6a+2b)u(n+6)+u(n+8)=0.$

We make the following important observation. Consider eigenvalues
$\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4}$ of the matrix $\Gamma.$ They
are

$\lambda_{1,2}=\frac{1}{2}(1\pm\sqrt{1-6a-2b-2\sqrt{-4a+(3a+b)^{2}}})$

and

$\lambda_{3,4}=\frac{1}{2}(1\pm\sqrt{1-6a-2b+2\sqrt{-4a+(3a+b)^{2}}}).$

The list of eigenvalues of $\Gamma-I$ coincides with
$\\{-\lambda_{1},-\lambda_{2},-\lambda_{3},-\lambda_{4}\\},$ where
$\\{\lambda_{1},\lambda_{2},\lambda_{3},\lambda_{4}\\}$ are eigenvalues of
$\Gamma.$

Now we introduce four integer sequences.

$s_{even}(n)=\frac{\lambda_{2}^{n}-\lambda_{1}^{n}}{\lambda_{2}-\lambda_{1}}+\frac{\lambda_{4}^{n}-\lambda_{3}^{n}}{\lambda_{4}-\lambda_{3}},$

$t_{even}(n)=\left(\frac{\lambda_{2}^{n}-\lambda_{1}^{n}}{\lambda_{2}-\lambda_{1}}-\frac{\lambda_{4}^{n}-\lambda_{3}^{n}}{\lambda_{4}-\lambda_{3}}\right)/\left(\Big{(}\frac{\lambda_{4}-\lambda_{3}}{2}\Big{)}^{2}-\Big{(}\frac{\lambda_{2}-\lambda_{1}}{2}\Big{)}^{2}\right),$

$s_{odd}(n)=\lambda_{1}^{n}+\lambda_{2}^{n}+\lambda_{3}^{n}+\lambda_{4}^{n},$

$t_{odd}(n)=(\lambda_{1}^{n}+\lambda_{2}^{n}-\lambda_{3}^{n}-\lambda_{4}^{n})/\left(\Big{(}\frac{\lambda_{4}-\lambda_{3}}{2}\Big{)}^{2}-\Big{(}\frac{\lambda_{2}-\lambda_{1}}{2}\Big{)}^{2}\right).$

We note that
$\left((\frac{\lambda_{4}-\lambda_{3}}{2})^{2}-(\frac{\lambda_{2}-\lambda_{1}}{2})^{2}\right)=\sqrt{-4a+(3a+b)^{2}}.$

The fact that all of them are expressed as linear combinations of $n$-th
powers of eigenvalues of $\Gamma$ and $\Gamma-I$ implies that they satisfy the
same recursive relation (4).

Then we also introduce two sequences.

$s(n)=\begin{cases}s_{even}(n),\textrm{ if }n\textrm{ is even}\\\
s_{odd}(n),\textrm{ if }n\textrm{ is odd}\end{cases}\textrm{ and }\
t(n)=\begin{cases}t_{even}(n),\textrm{ if }n\textrm{ is even}\\\
t_{odd}(n),\textrm{ if }n\textrm{ is odd}\end{cases}.$

The structure of matrix $B(n)$ is given by the following lemmas.

######  Lemma 4.1.

Suppose $n$ is odd. Then the matrix $B(n)=s(n)L+t(n)R$ is a sum of two
centrosymmetric matrices, where $L$ and $R$ are

$L=\left(\begin{array}[]{cccc}\frac{1}{2}&0&0&0\\\ 0&\frac{1}{2}&0&0\\\
0&0&\frac{1}{2}&0\\\ 0&0&0&\frac{1}{2}\end{array}\right)$

and

$R=\left(\begin{array}[]{cccc}\frac{a-b}{2}&-1+a+2b&a-b&-a\\\
a&\frac{-a+b}{2}&a&0\\\ 0&a&\frac{-a+b}{2}&a\\\
-a&a-b&-1+a+2b&\frac{a-b}{2}\end{array}\right).$

###### Proof.

We note that both sides of equality $B(n)=s(n)L+t(n)R$ satisfy the linear
recursive equation (4). So, it is sufficient to prove Lemma for $n=1,3,5,7.$
By making use of symbolic calculation with Wolfram Mathematica 12.0 we deduce
that

$\displaystyle s(1)$ $\displaystyle=2,\,s(3)=2-9a-3b,$ $\displaystyle s(5)$
$\displaystyle=2-25a+45a^{2}-5b+30ab+5b^{2},$ (3) $\displaystyle s(7)$
$\displaystyle=2-49a+189a^{2}-189a^{3}-7b+105ab$
$\displaystyle-189a^{2}b+14b^{2}-63ab^{2}-7b^{3}$

and

$\displaystyle t(1)=0,\,t(3)=-3,$ $\displaystyle t(5)=5(-1+3a+b),$ (4)
$\displaystyle t(7)=-7(1-7a+9a^{2}-2b+6ab+b^{2}).$

Then the result follows by the direct substitution. ∎

######  Lemma 4.2.

Suppose $n$ is even. Then the matrix $B(n)=s(n)L+t(n)R$ is a sum of two
anticentrosymmetric matrices, where $L$ and $R$ are

$L=\left(\begin{array}[]{cccc}\frac{-1+2a}{2}&-1+b&-b&-a\\\
a&\frac{-1+2b}{2}&-b&-a\\\ a&b&\frac{1-2b}{2}&-a\\\
a&b&1-b&\frac{1-2a}{2}\end{array}\right)$

and $R$ is block matrix of the form $\left(\begin{array}[]{cc}L&M\\\
\hat{M}&\hat{L}\end{array}\right),$ where

$L=\left(\begin{array}[]{cc}\frac{-5a+6a^{2}+b+2ab}{2}&1-2a-3b+3ab+b^{2}\\\
a(-1+3a+b)&\frac{(3a+b)(-1+2b)}{2}\end{array}\right),$

$M=\left(\begin{array}[]{cc}-a+b-3ab-b^{2}&-a(-1+3a+b)\\\
a-3ab-b^{2}&-a(3a+b)\end{array}\right)$

and $\widehat{\left(\begin{array}[]{cc}x&y\\\
z&w\end{array}\right)}=\left(\begin{array}[]{cc}-w&-z\\\
-y&-x\end{array}\right).$

###### Proof.

Since matrix $B(n)=s(n)L+t(n)R$ is a solution of linear recursive equation
(4), it is sufficient to prove Lemma for $n=0,2,4,6.$ We have

$\displaystyle s(0)=0,\,s(2)=2,$ $\displaystyle s(4)=2(1-3a-b),$ (5)
$\displaystyle s(6)=2-18a+27a^{2}-4b+18ab+3b^{2}$

and

$t(0)=0,\,t(2)=0,\,t(4)=-2,\,t(6)=-4+9a+3b.$ (6)

Then the result follows by the direct calculations. ∎

In what follows, we will call the sequences $s(n)$ and $t(n)$ to be associated
with Alexander polynomial $A(z).$ They are unique solutions of of linear
recursive equation (4) with initial data (4) – (6).

##  5 Prelminary results

The next theorem is reformulation of a result previously proved by various
authors ([3], [11], [12]).

######  Theorem 5.1.

Let $K$ be a $2$-bridge knot of genus $1$ with Alexander polynomial
$A(z)=b+(1-2b)z+b\,z^{2}.$ We set
$\alpha(n)=\left(\frac{1+x}{2}\right)^{n}+\left(\frac{1-x}{2}\right)^{n}$ and
$\beta(n)=\frac{1}{x}\left(\left(\frac{1+x}{2}\right)^{n}-\left(\frac{1-x}{2}\right)^{n}\right),$
where $x=\sqrt{1-4b}.$

Then the first homology group of cyclic $n$-fold covering $M_{n}$ of the three
sphere $\mathbb{S}^{3}$ branched over knot $K$ has the following structure

$H_{1}(M_{n},\mathbb{Z})=\begin{cases}\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\alpha(n)}\textit{
if }n\textit{ is odd,}\\\
\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{(4b-1)\beta(n)}\textit{ if }n\textit{
is even.}\end{cases}$

###### Proof.

By K. Funcke ([5], Sats 5.1), any $2$-bridge knot of genus $1$ has a slope
$p/q=2\lambda_{1}+\frac{1}{-2\lambda_{2}}$ for some integers
$\lambda_{1},\lambda_{2}.$ In this case, Seifert matrix of the knot has a form
$V=\left(\begin{array}[]{cc}\lambda_{1}&1\\\ 0&\lambda_{2}\end{array}\right).$
One can recognize Alexander polynomial $A(z)$ as determinant of the matrix
$V-zV^{t},$ where $V^{t}$ is transpose of matrix $V.$ Therefore,
$A(z)=\lambda_{1}\lambda_{2}+(1-2\lambda_{1}\lambda_{2})z+\lambda_{1}\lambda_{2}z^{2}.$
That is $b=\lambda_{1}\lambda_{2}.$

Following Seifert we set $\Gamma=(V-V^{t})^{-1}V.$ Then by Seifert theorem
([17], Satz 1) $H_{1}(M_{n},\mathbb{Z})$ is an Abelian group given by the
matrix $H(n)=\Gamma^{n}-(\Gamma-\mathtt{1})^{n}.$ By direct calculations we
have $H(n)=\left(\begin{array}[]{cc}\alpha(n)&0\\\
0&\alpha(n)\end{array}\right)$ if $n$ is odd and
$H(n)=\beta(n)\left(\begin{array}[]{cc}1&2\lambda_{2}\\\
-2\lambda_{1}&-1\end{array}\right)$ if $n$ is even. Since the Smith Normal
Form of matrix $\left(\begin{array}[]{cc}1&2\lambda_{2}\\\
-2\lambda_{1}&-1\end{array}\right)$ is $\left(\begin{array}[]{cc}1&0\\\
0&4\lambda_{1}\lambda_{2}-1\end{array}\right)$ the result follows. ∎

Theorem 5.1 shows that the structure of the first homology group
$H_{1}(M_{n},\mathbb{Z})$ completely defined by the Alexander polynomial of
the $2$-bridge knot $K.$ This fails to be true for $3$-bridge knots. Indeed,
consider two knots $6_{1}$ and $9_{46}$ shown on figures $50$ and $51$ in the
book [3]. The first one is a $2$-bridge knot while the other is a $3$-bridge
knot. They share the same Alexander polynomial $A(z)=-2+5z-2z^{2}$ but have
the different Seifert matrices $\left(\begin{array}[]{cc}1&1\\\
0&-2\end{array}\right)$ and $\left(\begin{array}[]{cccc}1&0&0&0\\\ 0&-1&0&0\\\
1&0&1&1\\\ -1&-1&0&0\end{array}\right)$ respectively. Consequently, the first
homology group of their cyclic coverings are

$6_{1}:\begin{cases}\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\alpha(n)}\textit{
if n is odd,}\\\ \mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{9\beta(n)}\textit{ if
n is even,}\end{cases}\textrm{ and
}9_{46}:\begin{cases}\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\alpha(n)}\textit{
if n is odd,}\\\ \mathbb{Z}_{3\beta(n)}\oplus\mathbb{Z}_{3\beta(n)}\textit{ if
n is even.}\end{cases}$

Here $\alpha(n)=2^{n}+(-1)^{n}$ and $\beta(n)=\frac{1}{3}(2^{n}-(-1)^{n}).$

##  6 Main results

Our first result is given by the following theorem.

######  Theorem 6.1.

Let $K$ be a $2$-bridge knot with Alexander polynomial
$A(z)=a+(b-a)z+(1-2b)z^{2}+(b-a)z^{3}+az^{4}.$ Let $s(n)$ and $t(n)$ be the
associated with $A(z)$ recurrent sequences. Suppose that $n$ is odd. Consider
the two following sequences
$\hat{\alpha}(n)=\gcd(t(n),\frac{s(n)+(5a-b)t(n)}{2})$ and
$\hat{\beta}(n)=\frac{s(n)^{2}+(4a-(3a+b)^{2})t(n)^{2}}{4\hat{\alpha}(n)}.$

Then the first homology group of cyclic $n$-fold covering $M_{n}$ of the three
sphere $\mathbb{S}^{3}$ branched over knot $K$ has the following structure

$H_{1}(M_{n},\mathbb{Z})=\mathbb{A}\oplus\mathbb{A},\textit{ where
}\mathbb{A}=\mathbb{Z}_{\hat{\alpha}(n)}\oplus\mathbb{Z}_{\hat{\beta}(n)}.$

###### Proof.

Taking into account lemma 4.1 we conclude that matrix $B(n)$ which represents
the first homology group $H_{1}(M_{n},\mathbb{Z})$ has the form

$B(n)=\left(\begin{array}[]{cccc}\alpha&\beta&\gamma&\delta\\\
-\delta&\alpha-\gamma&-\delta&0\\\ 0&-\delta&\alpha-\gamma&-\delta\\\
\delta&\gamma&\beta&\alpha\end{array}\right),$

where
$\alpha=\frac{s(n)+(a-b)t(n)}{2},\beta=(-1+a+2b)t(n),\gamma=(a-b)t(n),\delta=-a\,t(n).$
To describe the structure of the first homology group of we have to find the
Smith Normal Form of its relation matrix $B(n).$ The following sequence of
elementary row-column transformations significantly simplify our task.

$\left(\begin{array}[]{cccc}\alpha&\beta&\gamma&\delta\\\
-\delta&\alpha-\gamma&-\delta&0\\\ 0&-\delta&\alpha-\gamma&-\delta\\\
\delta&\gamma&\beta&\alpha\end{array}\right)\to\left(\begin{array}[]{cccc}\alpha&\beta-\delta&\gamma&\delta\\\
-\delta&\alpha-\gamma&-\delta&0\\\ 0&0&\alpha-\gamma&-\delta\\\
\delta&\gamma-\alpha&\beta&\alpha\end{array}\right)\to$

$\left(\begin{array}[]{cccc}\alpha&\beta-\delta&\gamma-\alpha&\delta\\\
-\delta&\alpha-\gamma&0&0\\\ 0&0&\alpha-\gamma&-\delta\\\
\delta&\gamma-\alpha&\beta-\delta&\alpha\end{array}\right)\to\left(\begin{array}[]{cccc}\alpha&\beta-\delta&0&0\\\
-\delta&\alpha-\gamma&0&0\\\ 0&0&\alpha-\gamma&-\delta\\\
0&0&\beta-\delta&\alpha\end{array}\right).$

So $B(n)$ splits into two $2\times 2$ blocs
$\left(\begin{array}[]{cc}\alpha&\beta-\delta\\\
-\delta&\alpha-\gamma\end{array}\right)$ and
$\left(\begin{array}[]{cc}\alpha-\gamma&-\delta\\\
\beta-\delta&\alpha\end{array}\right).$ They have the same Smith Normal Form.
The block $\left(\begin{array}[]{cc}\alpha&\beta-\delta\\\
-\delta&\alpha-\gamma\end{array}\right)$ is equivalent to
$\left(\begin{array}[]{cc}d_{1}&0\\\ 0&d_{2}\end{array}\right),$ where
$d_{1}=\gcd(\alpha,\,\beta-\delta,\,-\delta,\,\alpha-\gamma)=\gcd(\alpha,\,\beta,\,\gamma,\,\delta)$
and $d_{2}=\det\left(\begin{array}[]{cc}\alpha&\beta-\delta\\\
-\delta&\alpha-\gamma\end{array}\right)/d_{1}=\frac{\alpha^{2}-\alpha\,\gamma+\beta\,\delta-\delta^{2}}{d_{1}}.$
After substituting $\alpha,\,\beta,\,\gamma$ and $\delta$ with their
expressions through $a,\,b,\,s,\,t$ we get
$d_{1}=\gcd(t(n),\frac{s(n)+(5a-b)t(n)}{2})$ and
$d_{2}=\frac{s(n)^{2}+(4a-(3a+b)^{2})t(n)^{2}}{4d_{1}}.$ Finally, the Smith
Normal Form of $B(n)$ is diagonal matrix
$\textrm{diag}(d_{1},\,d_{1},\,d_{2},\,d_{2}).$ ∎

The second result described the structure of the first homology group for the
case of even $n.$ For the sake of simplicity we will write
$s,\,t,\,\mu,\,\zeta$ instead of $s(n),\,t(n),\,\mu(n),\,\zeta(n).$

######  Theorem 6.2.

Let $K$ be a knot with Alexander polynomial
$A(z)=a+(b-a)z+(1-2b)z^{2}+(b-a)z^{3}+az^{4}.$ Let $s=s(n)$ and $t=t(n)$ be
the associated with $A(z)$ recurrent sequences. Suppose that $n$ is even. We
set $k=1+4a-4b,\,\zeta=\frac{s+(5a-b)t}{2}$ and
$\mu=\frac{s^{2}+(4a-(3a+b)^{2})t^{2}}{4}.$ Then the first homology group of
cyclic $n$-fold covering $M_{n}$ of the three sphere $\mathbb{S}^{3}$ branched
over knot $K$ has the following structure

$H_{1}(M_{n},\mathbb{Z})=\mathbb{Z}_{d_{1}}\oplus\mathbb{Z}_{d_{2}/d_{1}}\oplus\mathbb{Z}_{d_{3}/d_{2}}\oplus\mathbb{Z}_{d_{4}/d_{3}},$

where
$d_{1}=\gcd(t,\,\zeta),d_{2}=\gcd(\mu,\,\zeta\,t,k\,t^{2}),d_{3}=\mu\,\gcd(k\,t,\zeta)$
and $d_{4}=k\,\mu^{2}.$

###### Proof.

We note that

$\det(B(n))=(1+4a-4b)(\frac{s^{2}+(4a-(3a+b)^{2})t^{2}}{4})^{2}=k\,\mu^{2},$

where $k=1+4a-4b$ and $\mu=\frac{s^{2}+(4a-(3a+b)^{2})t^{2}}{4}.$

The number $k=A(-1)$ is well known in the knot theory. It absolute value is
called a determinant of the knot $K$ and was introduced firstly by K.
Reidemeister in his book [14].

Since $k=1+4a-4b$ is odd we conclude that $\mu$ is an integer number.
Moreover, since both sequences $s=s(n)$ and $t=t(n)$ satisfy the recursive
relation (4) the same holds for $\zeta(n)=\frac{s(n)+(5a-b)t(n)}{2}.$ Taking
into account the initial data for $s(n)$ and $t(n)$ we have
$\zeta(0)=0,\,\zeta(2)=1,\,\zeta(4)=1-8a,\,\zeta(6)=1-19a+36a^{2}+12a\,b.$
Hence, $\zeta(n)$ is integer number for all even $n.$

As we have expression of the matrix $B=B(n)$ from lemma 4.2, one can easily
find two integer matrices $C$ and $D$ such that $B=\zeta\,C+t\,D.$ So, each
entry $B_{i,j}$ of matrix $B$ is an integer linear combination $\zeta$ and
$t.$ The converse is also true. Namely, $\zeta=-B_{1,1}-B_{2,4}$ and
$t=(-2-8a+2b+4a\,b)B_{1,1}+B_{1,2}+2(2-b)B_{1,3}+(5-8a+6b+4a\,b-4b^{2})B_{1,4}-2(2-b)B_{2,3}-(3+6b-4b^{2})B_{2,4}.$
This leads to the following result.

######  Proposition 6.3.

The greatest common divisor $d_{1}$ of all entries of matrix $B$ is equal to
the greatest common divisor of $t$ and $\zeta.$

In order to find the second element of the Smith Normal Form of $B$ we need to
find the greatest common divisor of all $2\times 2$ minors of $B.$

######  Proposition 6.4.

The greatest common divisor $d_{2}$ of all $2\times 2$ minors of matrix $B$ is
equal to $\gcd(\mu,\,\zeta\,t,\,k\,t^{2}).$

###### Proof.

Following notation from Wolfram Mathematica [10] denote by
$P=\textrm{Minors[B,\,2]}$ the six by six matrix consisting of $2\times 2$
minors of $B.$ By $P_{i,j}$ we denote the $(i,\,j)$-entry of matrix $P.$

By making use of lemma 4.2 one can find three integer matrices $E,\,F$ and $G$
such that $P=\mu\,E+t\,\zeta\,F+k\,t^{2}\,G.$ This can be done by the
following procedure. First, we express each $P_{i,j}$ in the form
$P_{i,j}=\mu\,E_{i,j}+R_{i,j},$ where $E_{i,j}$ and $R_{i,j}$ are the
polynomial quotient and polynomial remainder of $P_{i,j}$ and $\mu$ with
respect to $s.$ Then we set $E=\\{E_{i,j}\\}_{i,j=1,\ldots,6}$ and note that
all the entries of $E$ are integer polynomials in $a$ and $b.$ Next, we
present each $R_{i,j}$ as $R_{i,j}=t\,\zeta\,F_{i,j}+k\,t^{2}\,G_{i,j},$ where
$F_{i,j}$ and $k\,t^{2}\,G_{i,j}$ are the polynomial quotient and polynomial
remainder of $R_{i,j}$ and $t\,\zeta$ with respect to $s.$ Again, all the
entries of matrices $F=\\{F_{i,j}\\}_{i,j=1,\ldots,6}$ and
$G=\\{G_{i,j}\\}_{i,j=1,\ldots,6}$ are integer polynomials in $a$ and $b.$

Set $x=\mu,\,y=t\,\zeta$ and $z=k\,t^{2}.$ From now on we can consider the
elements $P_{i,j}$ as integer linear combinations of $x,\,y$ and $z$ whose
coefficients are integer polynomials in $a$ and $b.$

Consider $J=Span_{\mathbb{Z}}(P_{i,j},\,i,j=1,\ldots,6)$ as a
$\mathbb{Z}$-module. As it shown before, each $P_{i,j}$ is an integer linear
combination of $x,\,y$ and $z.$ Hence the same is true for all elements of
$J.$ Now we will prove that $J$ is generated by $x,\,y,\,z$ as a
$\mathbb{Z}$-module. This is equivalent to the statement of proposition under
proof. To do this we need only to show that all three of $x,\,y,\,z$ belong to
$J.$ The inclusion of $y$ in $J$ follows from the identity

$\displaystyle y$ $\displaystyle=$ $\displaystyle
2(-1-6a+2ab)(2P_{1,3}+P_{1,6}-P_{3,3})+2(2-b)(P_{6,5}+P_{6,6})$
$\displaystyle+$ $\displaystyle P_{3,5}+(4+a-7b+2b^{2})(2P_{1,5}-P_{2,5})$
$\displaystyle+$
$\displaystyle(15+14a-28b-4ab+8b^{2})(P_{4,3}-P_{5,1}+P_{6,3}).$

To prove that $x$ and $z$ lie in $J$ we make some small observation

$\displaystyle P_{1,3}$ $\displaystyle-P_{1,6}=2a\,x,$ $\displaystyle P_{4,3}$
$\displaystyle-P_{5,1}+P_{6,3}=a\,y,$ $\displaystyle 2P_{1,5}$ $\displaystyle-
P_{2,5}=-a\,z+2a\,y.$

Hence, $2a\,x,\,a\,y$ and $a\,z$ lie in $J.$

Next, we take the following $4$-tuple of elements of
$J:(P_{1,1}+P_{1,2},P_{1,4},P_{2,4},P_{3,4}).$ Reducing each element of this
tuple modulo polynomials $2a\,x,\,a\,z$ and $y$ we get much more simple
expression whose parts belong to $J:$

$(x,-b^{2}z+2bx,bz-2b^{2}z+(-2+6b)x,-z+4bz-3b^{2}z+(-4+8b)x).$

So, consequently we conclude that $x\in J,\,b^{2}z\in J,\,b\,z\in J$ and
lastly, $z\in J.$ ∎

######  Proposition 6.5.

The greatest common divisor $d_{3}$ of all $3\times 3$ minors of matrix $B$ is
equal to $\mu\,\gcd(k\,t,\zeta).$

###### Proof.

As before, following notation from Wolfram Mathematica [10] denote by
$P^{\prime}=\textrm{Minors[B,\,3]}$ the four by four matrix consisting of
$3\times 3$ minors of $B.$ By $P_{i,j}^{\prime}$ we denote the $(i,\,j)$
-entry of matrix $P^{\prime}.$

From lemma 4.2 one can derive the following identity

$P^{\prime}=\mu(k\,t\,C^{\prime}+\zeta\,D^{\prime}),$

where

$C^{\prime}=\left(\begin{array}[]{cccc}-a+b&a&0&a\\\ -1-a+2b&0&a&a-b\\\
-a+b&-a&0&1+a-2b\\\ -a&0&-a&a-b\end{array}\right)$

and

$D^{\prime}=\left(\begin{array}[]{cccc}1+2a-4b&-2a&-2a&-2a\\\
2+4a-6b&1-2b&-4a+2b&-4a+2b\\\ 4a-2b&4a-2b&-1+2b&-2-4a+6b\\\
2a&2a&2a&-1-2a+4b\end{array}\right).$

So, all $3\times 3$ minors of $B$ are linear combinations of the form
$\alpha\,k\,t\,\mu+\beta\,\zeta\,\mu.$ Moreover, one can see that
$\zeta\,\mu=P_{2,2}^{\prime}+P_{2,3}^{\prime}-P_{1,2}^{\prime}-P_{1,3}^{\prime}$
and
$k\,t\,\mu=(-1+4a+2b)P_{1,2}^{\prime}+(-3+4a+2b)P_{1,3}^{\prime}-2(-1+2a+b)(P_{2,2}^{\prime}+P_{2,3}^{\prime})-2P_{2,4}^{\prime}+P_{3,4}^{\prime}.$
Hence, the greatest common divisor of all $3\times 3$ minors of $B$ coincides
with the greatest common divisor
$\gcd(k\,t\,\mu,\zeta\,\mu)=\mu\gcd(k\,t,\zeta).$ ∎

There is only one $4$ minor of $B$ which is $\det(B)=k\,\mu^{2}.$ Hence, the
greatest common divisor $d_{4}$ of all $4\times 4$ minors is equal to
$k\,\mu^{2}.$ Now, we use all the above propositions to finish the proof of
the theorem. ∎

The connection between Theorem 1 and Theorem 2 is given by the following
refined version of the second Plans theorem ([13], [18]).

######  Theorem 6.6.

Let $K$ be a knot with Alexander polynomial
$A(z)=a+(b-a)z+(1-2b)z^{2}+(b-a)z^{3}+az^{4}.$ Let $s(n)$ and $t(n)$ be the
associated with $A(z)$ recurrent sequences. Suppose that $n$ is even. Consider
the two following sequences
$\hat{\alpha}(n)=\gcd(t(n),\frac{s(n)+(5a-b)t(n)}{2})$ and
$\hat{\beta}(n)=\frac{s(n)^{2}+(4a-(3a+b)^{2})t(n)^{2}}{4\alpha(n)}$ and set
$k=1+4a-4b.$ Then $H_{1}(M_{2},\mathbb{Z})=\mathbb{Z}_{k}$ and we have the
following exact sequence of Abelian groups

$0\longrightarrow\mathbb{A}\oplus\mathbb{A}\longrightarrow
H_{1}(M_{n},\mathbb{Z})\longrightarrow\mathbb{Z}_{k}\longrightarrow 0,$

where
$\mathbb{A}=\mathbb{Z}_{\hat{\alpha}(n)}\oplus\mathbb{Z}_{\hat{\beta}(n)}.$

###### Proof.

Let $n$ be even. Direct calculation from formula (1) leads to
$B(2)=\Gamma^{2}-(\Gamma-I)^{2}=2\Gamma-I.$ Hence

$B(2)=\left(\begin{array}[]{cccc}-1+2a&-2+2b&-2b&-2a\\\ 2a&-1+2b&-2b&-2a\\\
2a&2b&1-2b&-2a\\\ 2a&2b&2-2b&1-2a\end{array}\right).$

Its determinant is $1+4a-4b.$ Since $a$ and $b$ are whole numbers $B(2)$ is
invertible.

It easy to see that the greatest common divisor of all entries of $B(2)$ is
$1,$ The same is true for minors of orders $2$ and $3.$ So, the Smith Normal
Form of $B(2)$ is $\textrm{diag}\\{1,1,1,1+4a-4b\\}$ and
$H_{1}(M_{2},\mathbb{Z})=\textrm{coker}(B(2))={Z}_{1+4a-4b}.$

By making use of Lemma 4.2 we have

$B(2)^{-1}B(n)=\left(\begin{array}[]{cccc}\alpha&\beta&\gamma&\delta\\\
-\delta&\alpha-\gamma&-\delta&0\\\ 0&-\delta&\alpha-\gamma&-\delta\\\
\delta&\gamma&\beta&\alpha\end{array}\right),$

where
$\alpha=\frac{s(n)+(a-b)t(n)}{2},\beta=(-1+a+2b)t(n),\gamma=(a-b)t(n),\delta=-a\,t(n).$
We note for $n$ odd, the matrix of this form appeared early in Theorem 6.1.

From here we consider two matrices $W_{1}=B(2)$ and $W_{2}=B(2)^{-1}B(n)$ as
$\mathbb{Z}$-linear operators from $\mathbb{Z}^{4}$ into itself. Dealing with
$W_{2}$ in the same way as in the proof of Theorem 6.1 we deduce that the
Abelian group represented by $W_{2}$ has the form
$\mathbb{A}\oplus\mathbb{A},$ where
$\mathbb{A}=\mathbb{Z}_{\hat{\alpha}(n)}\oplus\mathbb{Z}_{\hat{\beta}(n)}.$
Here $\hat{\alpha}(n)$ and $\hat{\beta}(n)$ are the same as in the statement
under proof.

Moreover, there exist an exact sequence of the type

$ker\,W_{1}\longrightarrow coker\,W_{2}\longrightarrow
coker\,W_{1}W_{2}\longrightarrow coker\,W_{1}\longrightarrow 0.$

For the proof see, for example ([9], Snowflake Lemma 2.3.16) or ([6], Theorem
2.1).

From the above we have $\textrm{ker}\,W_{1}=0$ and
$\textrm{coker}\,W_{1}=\mathbb{Z}_{1+4a-4b}.$ Also,
$\textrm{coker}\,W_{2}=\mathbb{A}\oplus\mathbb{A}$ and
$\textrm{coker}\,W_{1}W_{2}=H_{1}(M_{n},\mathbb{Z}).$ Hence, the previous
exact sequence morphs into

$0\longrightarrow\mathbb{A}\oplus\mathbb{A}\longrightarrow
H_{1}(M_{n},\mathbb{Z})\longrightarrow\mathbb{Z}_{k}\longrightarrow 0.$

∎

##  7 Examples

###  7.1 Knot $6_{2}$

Alexander polynomial: $A(z)=-1+3z-3z^{2}+3z^{3}-z^{4}.$ Here $a=-1,b=2.$

We set $\alpha(n)=\gcd(t(n),\frac{-7t(n)+s(n)}{2})$ and
$\beta(n)=\frac{s(n)^{2}-5t(n)^{2}}{4\alpha(n)}.$

If $n$ is odd then
$H_{1}(M_{n}(6_{3}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}^{2}\oplus\mathbb{Z}_{\beta(n)}^{2}.$

If $n$ is even and is not a multiple of $12$ then

$H_{1}(M_{n}(6_{3}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{11\beta(n)}$

otherwise

$H_{1}(M_{n}(6_{3}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{11\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{\beta(n)}.$

Here is a table for initial values of $\alpha(n)$ and $\beta(n).$

Table 1: Data for $H_{1}(M_{n}(6_{2}),\mathbb{Z})$ $n$ | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12
---|---|---|---|---|---|---|---|---|---|---|---|---
$\alpha(n)$ | 1 | 1 | 1 | 1 | 2 | 1 | 1 | 3 | 1 | 4 | 1 | 1
$\beta(n)$ | 1 | 1 | 5 | 1 | 2 | 5 | 29 | 3 | 5 | 4 | 131 | 55

###  7.2 Knot $6_{3}$

Alexander polynomial: $A(z)=1-3z+5z^{2}-3z^{3}+z^{4}.$ Here $a=1,b=-2.$

We set $\alpha(n)=\gcd(t(n),\frac{3t(n)+s(n)}{2})$ and
$\beta(n)=\frac{s(n)^{2}-21t(n)^{2}}{4\alpha(n)}.$

If $n$ is odd then
$H_{1}(M_{n}(6_{3}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}^{2}\oplus\mathbb{Z}_{\beta(n)}^{2}.$

If $n$ is even and is not a multiple of $14$ then

$H_{1}(M_{n}(6_{3}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{13\beta(n)}$

otherwise

$H_{1}(M_{n}(6_{3}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{13\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{\beta(n)}.$

Here is a table for initial values of $\alpha(n)$ and $\beta(n).$

Table 2: Data for $H_{1}(M_{n}(6_{3}),\mathbb{Z})$ $n$ | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---
$\alpha(n)$ | 1 | 1 | 1 | 1 | 4 | 1 | 1 | 1 | 1 | 8 | 1 | 3 | 1 | 1
$\beta(n)$ | 1 | 1 | 7 | 3 | 4 | 7 | 43 | 21 | 133 | 8 | 397 | 63 | 1171 | 559

###  7.3 Knot $7_{7}$

Alexander polynomial: $A(z)=1-5z+9z^{2}-5z^{3}+z^{4}.$ Here $a=1,b=-4.$

We set $\alpha(n)=\gcd(t(n),\frac{9t(n)+s(n)}{2})$ and
$\beta(n)=\frac{s(n)^{2}+3t(n)^{2}}{4\alpha(n)}.$

If $n$ is odd then
$H_{1}(M_{n}(7_{7}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}^{2}\oplus\mathbb{Z}_{\beta(n)}^{2}.$

If $n$ is even we have the following cases

  1. 1.

$n\equiv 0\mod 12$

$H_{1}(M_{n}(7_{7}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{21\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{\beta(n)};$

  2. 2.

$n\equiv 2,10\mod 12$

$H_{1}(M_{n}(7_{7}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{21\beta(n)};$

  3. 3.

$n\equiv 4,8\mod 12$

$H_{1}(M_{n}(7_{7}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{7\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{3\beta(n)};$

  4. 4.

$n\equiv 6\mod 12$

$H_{1}(M_{n}(7_{7}),\mathbb{Z})=\mathbb{Z}_{\alpha(n)}\oplus\mathbb{Z}_{3\alpha(n)}\oplus\mathbb{Z}_{\beta(n)}\oplus\mathbb{Z}_{7\beta(n)}.$

Here is a table for initial values of $\alpha(n)$ and $\beta(n).$

Table 3: Data for $H_{1}(M_{n}(7_{7}),\mathbb{Z})$ $n$ | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12
---|---|---|---|---|---|---|---|---|---|---|---|---
$\alpha(n)$ | 1 | 1 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 8 | 1 | 5
$\beta(n)$ | 1 | 1 | 13 | 7 | 38 | 39 | 421 | 217 | 2353 | 152 | 13201 | 1365

## Acknowledgment

The author was supported by Mathematical Center in Akademgorodok under
agreement No. 075-15-2019-1613 with the Ministry of Science and Higher
Education of the Russian Federation.

## References

  * [1] M. Boileau, and B. Zimmermann, The $\pi$-orbifold group of a link. Math. Z. 200 (1989), 187–208. Zbl 0663.57006 MR 978294
  * [2] A. Cattabriga, and M. Mulazzani, Strongly-cyclic branched coverings of (1, 1)-knots and cyclic presentations of groups. Math. Proc. Cambridge Phil. Soc. 135 (2003), no. 1, 137–146. Zbl 1050.57003 MR 1990837
  * [3] R. H. Fox, The homology characters of the cyclic coverings of the knots of genus one. Ann. Math. 71 (1960), no. 1, 187–196. Zbl 0122.41801 MR 0119210
  * [4] R. H. Fox, Free Differential Calculus III. Subgroups. Ann. Math. (2) 64 (1956), no. 3, 407–419. Zbl 0073.25401 MR 0095876
  * [5] K. Funke, Geschlecht von Knoten und die Faserbarkeit ihrer Aussenräume. Math. Z. 159 (1978), 3–24. Zbl 0734.55001
  * [6] D. Huang, A cyclic six-term exact sequence for block matrices over a PID. Linear Multilinear Algebra 49 (2001), no. 2, 91–114. Zbl 0991.15008 MR 1885668
  * [7] A. Kawauchi, A survey of knot theory. Basel: Birkhauser Verlag, 1996. Zbl 0861.57001 MR 1417494
  * [8] P.-V. Koseleff, and D. Pecker, On AlexanderÐConway polynomials of two-bridge links. J. Symbolic Comp. 68 (2015), no. 2, 215–229. Zbl 1305.57015 MR 3283863
  * [9] S. S. Kutateladze, Fundamentals of functional analysis. Vol. 12. Netherlands: Springer Science & Business Media, 2013. Zbl 0852.46001 MR 1384068
  * [10] Wolfram Research, Inc., Mathematica, Version 12.3.1, Champaign, IL (2021).
  * [11] M. Mulazzani, All Lins-Mandel spaces are branched cyclic coverings of $S^{3}.$ J. Knot Theory Ramif. 5 (1996), 239–263. Zbl 0858.57004 MR 1395781
  * [12] M. Mulazzani, and A. Vesnin, The many faces of cyclic branched coverings of 2-bridge knots and links. Atti Sem. Mat. Fis. Univ. Modena, Supplemento al Vol. IL (2001), 177–215. Zbl 1221.57009 MR 1881097
  * [13] A. Plans, Aportacion al estudio de los grupos de homologia de los recubrimientos ciclicos ramificados correspondientes a un nudo. Revista Acad. Ci. Madrid 47 (1953), 161–193. Zbl 0051.14603 MR 0056923
  * [14] K. Reidemeister, Knotentheorie. New York: Chelsea Pub. Co., New York, 1948. Zbl 0277.55007 MR 0345089 Originally published as: K. Reidemeister, Knotentheorie., Ergebnisse der Mathematik und ihrer Grenzgebiete, Alte Folge, Band 1, Heft 1, Berlin: Springer, 1932. Zbl JFM 58.1202.04
  * [15] D. Rolfsen, Knots and Links. UK ed. Edition: AMS Chelsea Publishing, 2003. Zbl 0339.55004 MR 1277811
  * [16] H. Schubert, Uber Knoten mit zwei Brücken. Math. Z. 65 (1956), 133–170. Zbl 0071.39002
  * [17] H. Seifert, Uber das Geschlecht von Knoten. Math. Ann. 110 (1934), 571–592. Zbl 0010.13303 MR 1512955
  * [18] W. H. Stevens, On the Homology of Branched Cyclic Covers of Knots Louisiana State University. PhD thesis, 1996. $https://digitalcommons.lsu.edu/gradschool{\\_}disstheses/6282$


# Action Recognition Using Volumetric Motion Representations

Michael Peven, Gregory D. Hager, Austin Reiter
Department of Computer Science, Johns Hopkins University, Baltimore, MD, USA

###### Abstract

Traditional action recognition models are constructed around the paradigm of
2D perspective imagery. Though sophisticated time-series models have pushed
the field forward, much of the information is still not exploited by confining
the domain to 2D. In this work, we introduce a novel representation of motion
as a voxelized 3D vector field and demonstrate how it can be used to improve
performance of action recognition networks. This volumetric representation is
a natural fit for 3D CNNs, and allows out-of-plane data augmentation
techniques during training of these networks. Both the construction of this
representation from RGB-D video and inference can be run in real time. We
demonstrate superior results using this representation with our network design
on the open-source NTU RGB+D dataset where it outperforms state-of-the-art on
both of the defined evaluation metrics. Furthermore, we experimentally show
how the out-of-plane augmentation techniques create viewpoint invariance and
allow the model trained using this representation to generalize to unseen
camera angles. Code is available here: https://github.com/mpeven/ntu_rgb.

##  1 Introduction

High-quality, low-cost RGB-D sensors are already commonplace in today’s world.
Color, depth, and often articulated pose data (3D body configuration) can be
collected with ease in real time. The success of action recognition methods
depend on how these multi-modal data sources can be used.

Articulated pose data is a popular choice as input to action recognition
classifiers as it has the advantage of being an easy-to-use, low-dimensional
representation of a human. Pose data lies in 3D Cartesian space, therefore
out-of-plane data augmentation techniques (3D rotations and translations) can
be used to infer novel viewpoints and increase training set size. However,
skeletons alone do not provide enough context for activities involving human-
object interaction. Furthermore, pose information is often noisy and can only
perform as well as the middleware in RGB-D sensors which compute it - by
definition limiting performance. Despite the fact that pose is often used for
action recognition, it remains to be seen whether this intermediate
representation alone is enough to recover a diverse set of human activities
from RGB-D data.

The field of action recognition has benefited from the substantial performance
improvements of Convolutional Neural Networks (CNNs) over the past decade. We
have seen the ability of these networks, trained on millions of images for
object recognition, to have considerable generalization when applied to other
tasks, including the problem of action recognition in video. However, the
performance gains for action recognition have not been commensurate with
object detection results - the pre-trained object recognition networks lack
the ability to model the temporal structure present in actions. To deal with
this, two-stream convolutional networks [24] use pre-trained networks on a
single image in conjunction with 2D optical flow over multiple RGB frames to
represent motion. Despite these successes, methods using 2D video lack the
important 3D spatial representation obtained from RGB-D sensors. 2D data
augmentation approaches are limited either to in-plane operations or
techniques like color jittering that do not represent legitimate physical
phenomena. Furthermore, traditional 2D imagery is viewpoint dependent -
without context, optical illusions such as “forced perspective” causes an
affiliation between overlapping objects that are in fact separated.

In this work, we propose a novel method for classification of actions in
video. We design an input representation that takes advantage of 3D motion
signals captured from RGB-D sensors without relying on pose data. Using this
volumetric input, we investigate an extension to the successful two-stream CNN
architecture [24] and experimentally demonstrate how our input representation
improves performance over the original 2D formulation. Furthermore, using 3D
data augmentation techniques, we show that we can improve performance by
generalizing to unseen views. Finally, to capture the long-term structure of
human activities, we apply a recurrent neural network which has shown promise
in modeling global temporal signals [20].

To summarize, our contribution is threefold. First, we introduce a novel input
to action recognition network using a volumetric representation of 3D motion
by projecting 2D optical flow from RGB video into 3D using the z-coordinate
captured in a depth-map video. Next, we show that out-of-plane data
augmentation (translations and rotations) that can be performed over this
voxel grid input helps to create view-point invariance. Finally, we obtain
state-of-the-art activity classification results using a two-stream CNN
network with 3D convolutions over this volumetric representation and further
extend it with a long-short term memory network (LSTM) on the output of the
convolutional layers. The two-stream network captures spatial and short-term
temporal signals from the video while the LTSM models the longer-term temporal
structure.

We have released the code in an open-source repository located at
https://github.com/mpeven/ntu_rgb. It contains everything needed for creating
the input representations and training the network on these inputs. Also
included is a custom OpenGL application for viewing the input representations.
This was built in order to manually validate an interpretable input
representation, demonstrated in Figure 1.

The rest of this paper is organized as follows. In Section 2, we discuss
related work in the field of action recognition. In Section 3, we introduce
our method of representing 3D motion from RGB-D data and define our proposed
neural network architecture in detail. Implementation and evaluation details
are given in Section 4 with a presentation and discussion of our results.

Figure 1: Our volumetric representation of motion during the ’shaking hands’
action. The red points show the point cloud and the blue vectors represent the
3D motion vectors used as input to the 3D convolutional network. The code to
visualize this type of representation from any RGB+D video is provided as an
OpenGL application in the released source code.

##  2 Related Work

A broad range of action recognition methods have been studied in the computer
vision community over the past few decades. Previous works related to ours
fall under the following categories: i) Convolutional networks for action
recognition, ii) Long-term temporal modeling for action recognition, and iii)
Input representations for action recognition.

###  2.1 CNNs for Action Recognition

The success of CNNs in the image domain has led to a variety of methods when
applied toward action recognition. To incorporate the temporal information in
video, early work investigated methods involving 3D CNNs [2, 11, 12, 28] which
perform 3D convolutions over snippets of images to learn spatiotemporal
patterns. However, [12] showed similar performance between a 3D CNN over
multiple frames and a 2D CNN over a single frame in a large-scale action
recognition setting. This result indicates that the spatiotemporal filters in
3D CNNs may not learn a useful temporal signal without a considerable number
of training videos. To correct for this, [27] propose using a 1D temporal
convolution to be placed on top of 2D spatial convolutions. Recent work in
[4], show impressive results using 3D CNNs with a deep temporal receptive
field. Interestingly, [4] showed that explicitly representing motion using 2D
optical flow still improved performance, even when using 3D kernels which are
claimed to learn motion signals across video frames.

In this work, we also employ 3D CNNs; however, ours differs in that the three
input dimensions are not $XYT$ (where $T$ is the temporal dimension of a
video), but $XYZ$, as the input representations we create lie in 3D Cartesian
space. Because the convolutions in our 3D CNN are applied over a volume, it is
reasonable to assume a symmetric receptive field. Specifically, this means we
avoid having to either estimate or experimentally evaluate the receptive field
of convolutions in the temporal dimension as is done in [4] and [28], where
the choice may not be optimal when generalizing to a new dataset that has
different frame-rates or actions that happen at an unfamiliar pace. Following
on the success of 3D CNNs for the object recognition domain in [26], we
investigate its application towards action recognition by using a volumetric
motion field.

Our proposed method is most closely related to the two-stream CNNs introduced
in [24]. This approach has a spatial CNN, leveraging the success of object
recognition networks and pre-trained models and it has a temporal CNN using
perspective-based 2D optical flow to represent motion. The substantial success
of two-stream approaches has led to numerous successors [4, 7, 15, 18, 30, 31]
into one of the more effective methods for action recognition to date.

###  2.2 Modeling Longer-term Structure

A limitation of both 3D CNNs and the original two-stream approach is the
inability to model the long-term temporal structure present in video. The work
introduced in [1] propose recurrent neural networks (RNNs), specifically LSTM
cells, to model this long-term structure. This work is expanded upon in [2],
where it uses the outputs from a 3D CNN as the input to the RNN. The work in
[5, 15, 20, 25, 32] all propose alternative methods of modeling a longer-term
signal on top of the outputs of either the original two-stream approach or
similar techniques of separating spatial and short-term temporal features.

###  2.3 Input Representations for Action Recognition

With the emergence of low-cost RGB-D sensors, many datasets for action
recognition also include frame-wise depth maps and articulated pose data. A
variety of methods look at using more than just the RGB video frames, for
example, in [13, 17, 18, 21, 22, 36, 37] articulated pose data is used for
action recognition; either alone or in addition to RGB and depth video. As the
pose output from Microsoft Kinect is calculated using the depth map only, work
in [8, 18] ignore this intermediate output and try to learn a representation
with depth-maps only. These works have shown that the 3D representation from
pose or depth contains a useful signal; however, state-of-the-art results for
action recognition use RGB data in some context (illustrated in Table 2).

Similar to our work, [33] use scene flow to get a representation of 3D motion
for action recognition. However, they constrain this representation to a 2D
image, rather than as a volumetric motion field. Furthermore, their work
differs from ours as dynamics of these motion fields in the temporal dimension
are not explicitly modeled. Although these design choices made in [33] are to
make use of pre-trained object detectors, the approach we take in this work is
to model 3D motion in a voxel grid in order to perform both 3D convolutions
and out-of plane augmentation techniques directly on this input.

Figure 2: The network architecture is composed of three main parts: (1) the
spatial stream from RGB frames; (2) the local-temporal stream using our 3D
motion representation (see Fig. 3 for more detail on the design of this
stream); (3) a global-temporal network to model temporal structure over an
entire action.

##  3 Methods Overview

We describe here in detail the methods used to transform raw RGB and depth
video (from an RGB-D sensor) into a volumetric representation of 3D motion.
Formally, the notation we use is as follows: We look at 3 representations of
each video $V$, the RGB frames $V_{RGB}\in R^{T\times I\times J\times 3}$, the
depth frames $V_{depth}\in R^{T\times U\times W}$, and the voxelized
combination $V_{voxel}\in R^{T\times X\times Y\times Z\times 3}$. The height
and width of the RGB video and depth video are $I,J$ and $U,W$, respectively.
The values $X$, $Y$, and $Z$ are the Cartesian coordinates in the voxel grid
and $T$ is the number of frames in the video. The last dimension in the voxel
grid is the 3D motion vectors $(dx,dy,dz)$ that we will describe in detail
below.

###  3.1 3D Voxel Flow

First, we run dense optical flow over the RGB video to get an estimate of 2D
motion over the video. The output of dense optical flow at frame $t$ in
$V_{RGB}$ is a displacement vector field, where each vector represents the
motion of the scene at pixel $(I_{t},J_{t})$ to pixel $(I_{t+1},J_{t+1})$.
However, these vectors are in the RGB image system. Using standard camera
calibration techniques, we can represent each value in this vector field in
terms of the depth image by using the extrinsics between the color and depth
cameras and computing a look-up table to map pixels in the RGB image to pixels
in the depth image. Finally, given the intrinsic parameters of the depth
camera, we can project each pixel in the depth image to its corresponding 3D
camera coordinate:

$\begin{bmatrix}X_{camera}\\\\[10.00002pt] Y_{camera}\\\\[10.00002pt]
Z_{camera}\end{bmatrix}=\begin{bmatrix}\displaystyle\frac{(i-c_{x})D_{i,j}}{f_{x}}\\\\[10.00002pt]
\displaystyle\frac{(j-c_{y})D_{i,j}}{f_{y}}\\\\[10.00002pt]
D_{i,j}\end{bmatrix}$ (1)

where $D_{i,j}$ is the depth value at pixel coordinates $(i,j)$ and
$(c_{x},c_{y},f_{x},f_{y})$ are the intrinsic parameters (principle point and
focal length, respectively) of the depth camera (note: if the depth camera
produces invalid data at a particular pixel, we skip it). This allows us to
project the 2D vector field from the color image into the 3D camera coordinate
system of the depth image. Thus, we have a dense 3D vector field representing
motion across frames:

$\begin{bmatrix}dX\\\ dY\\\ dZ\end{bmatrix}=\begin{bmatrix}X_{t+1}\\\
Y_{t+1}\\\ Z_{t+1}\end{bmatrix}-\begin{bmatrix}X_{t}\\\ Y_{t}\\\
Z_{t}\end{bmatrix}$ (2)

We rescale the 3D motion vectors to fit within a voxel grid. The size of the
voxel grid was chosen to be large enough to clearly represent the scene (see
Figure 1) but also kept small enough for computational reasons, as discussed
below.

###  3.2 Voxel Grid Augmentations

The benefit of a volumetric representation is that we can model realistic
augmentations, e.g. a change in viewing angle, in comparison to forced 2D
perspective augmentation methods. Two of these out-of-plane data augmentation
techniques to the voxel grid input were investigated. The first method was to
translate all of the voxels in the voxel grid in the $X$, $Y$, and $Z$
directions. The second was performing an out-of-plane rotation of the motion
vectors (rotation about the axis normal to the ground plane).

Figure 3: The Local-Temporal Stream. This is the network designed for the
volumetric motion representation described in Section 3.1. This input is
visualized here as images with a red point-cloud and green motion vectors.

###  3.3 3D Two-Stream Model

Inspired by the two-stream approach [24] of decomposing a video into a
discriminative representation of an action, we wish to segment a video into
both spatial and short-term temporal components. This is a powerful approach
as it leverages the success of object recognition networks for the spatial
stream and explicit modeling of motion in the temporal stream. Furthermore,
the spatial stream is important to pick up on the discriminative spatial cues
for identical motion patterns, (e.g. mopping vs. sweeping). Equivalently, the
temporal motion stream can do the same in the case of identical spatial
features (e.g. standing up vs. sitting down). Formally, we would like to
represent video $V$ as a snippet $S$, which represents a set of $L$
consecutive frames, i.e. the snippet starting at frame $\tau$ would be
represented by $S=(V_{\tau},V_{\tau+1},\dots,V_{\tau+(L-1)})$. In this
approach, the motion field throughout $S$ represents a localized motion
(gesture) and the color image at $S_{L/2}$ represents the median spatial
representation of the snippet.

However, this representation of a video is insufficient. It is unable to model
the entire structure of an action longer than the number of frames in the
video snippet representing the temporal feature [32]. We solve this issue by
sampling multiple segments across the entire video. Formally, if $S$ is the
input to the temporal network and video $V$ has $T$ frames, we would like to
get $K$ sections $S=(S_{0},\dots,S_{K})$, each of $L$ frames, where the frames
for snippet $k$ are chosen as follows:

$\displaystyle S_{k}$
$\displaystyle=(V_{\tau},V_{\tau+1},\cdots,V_{\tau+(L-1)})$ (3) where:
$\displaystyle\tau$ $\displaystyle=k\frac{T-L}{K-1}$ (4)

####  3.3.1 Spatial Stream

The spatial appearance stream responds to static information in individual
video frames. Thus, we build upon the substantial success of residual networks
(ResNets) [9] for object recognition tasks and apply it over the RGB frames
sampled from our video. We can leverage the models pre-trained on the millions
of images in the ImageNet dataset and fine-tune it towards our task. Training
and configuration details are specified in Section 4.2. As input we use the
center image in each sampled section of the video; for snippets
$S=(s_{0},\dots,s_{K})$ of length $L$, input to the spatial stream are the
images $(S_{(0,l=L/2)},S_{(1,l=L/2)},\dots,S_{(K,l=L/2)})$.

Stream | Input | Cross-Subject | Cross-View
---|---|---|---
2D Spatial Stream | 2D images | 81.48% | 80.60%
2D Local-Temporal Stream | 2D optical flow | 71.27% | 71.09%
3D Local-Temporal Stream | 3D motion field | 78.97% | 89.58%
2D 2-Stream | 2D images & 2D optical flow | 88.15% | 84.24%
3D 2-Stream | 2D images & 3D motion field | 89.84% | 94.54%
Table 1: Results of an architecture ablation study on the NTU RGB+D dataset.
This table shows the results from the different streams of the network. The
first section shows the results from using only the RGB video as input. The
second section shows a comparison of the motion stream between a 2D optical
flow input and the volumetric input developed here. The final section shows
the results of a product of experts voting scheme when combining the output of
each stream.

####  3.3.2 Local-Temporal Stream

The temporal stream of our network is used to learn features from short-term
motion patterns. Using the sampling technique described in Section 3.3, we
create $K$ short snippets from an input video. For each snippet, we build the
motion representation using the method described in Section 3.1 to collect the
3D motion vectors at each frame in the snippet, constructing a voxel grid per
snippet, and concatenating the motion vectors in the snippet into their
respective voxels. The final representation of a single snippet is
$\mathbf{S}_{temporal}\in R^{3L\times X\times Y\times Z}$, where $X$, $Y$, $Z$
are the dimensions of the voxel grid, and $3L$ represents stacked 3D motion
vectors across an entire snippet of length $L$.

We propose a fully 3D CNN (Fig. 3) using this representation of motion as
input. 3D convolutions are performed in a sliding window style search over
each voxel grid. Because these voxel grids are the optical flow based
representations of motion, this can be seen as the 3D equivalent of the two-
stream temporal network defined in [24] if we choose $K=1$. We consider the
effectiveness of a 3D CNN compared to the 2D equivalent, which we describe
below.

####  3.3.3 Global-Temporal Model

Modeling the global structure over compositions of the local features computed
above is crucial to discriminate between certain actions. For example, taking
off a shoe contains the same spatial and temporal patterns as putting on a
shoe, albeit in the opposite order. A solution to this, as described in [2, 5,
19, 20] is to model the global structure of an action using recurrent neural
networks. We employ an LSTM over the sequences of activations coming from both
the spatial and temporal stream. As we do not wish to investigate methods of
feature fusion in this work, we use separate LSTMs for the spatial and
temporal stream. Therefore, we avoid attempting to model the relationship
between space and time, and as can be seen in Figure 2, these networks can
ultimately be trained separately. A product of experts voting scheme is used
to produce a final classification across streams.

##  4 Experiments and Results

The methods introduced are evaluated here. We first present the evaluation
dataset. We follow with a description of the implementation and training
details. Then, we explore the method used to evaluate the performance of our
3D representation over the 2D baseline and we compare the full two-stream
method to current state of the art results. In the end we show in an ablative
study the performance gains using 3D data augmentation techniques.

###  4.1 NTU RGB+D Action Recognition Dataset

We evaluate our approach in the NTU RGB+D dataset [22] which to the best of
our knowledge, is the largest open-source RGB-D action recognition dataset.
This dataset contains RGB, depth, IR, and 3D articulated pose data for each
action. There are 56.9 thousand videos split into 60 individual action
classes. Of these 60 classes, 40 of which are daily activities, 9 are health
related, and 11 are mutual (two-person) actions. The dataset is collected
using 3 separate Kinect V2 cameras positioned at angles $-45^{\circ}$,
$0^{\circ}$, and $+45^{\circ}$ relative to the subject. This dataset defines
two methods for evaluation, which we follow as described in [22]. The first
one is the cross-subject split where a portion of the subjects are assigned to
a train split and the rest are reserved for testing. The second one is the
cross-view split where videos from two of the camera angles are used for
training and the other is used for testing. This corresponds to 40,320
training videos and 16,560 testing videos for the cross-subject split and
37,920 and 18,960 for the cross-view splits.

Model | Pose | Depth | RGB | Cross-Subject | Cross-View
---|---|---|---|---|---
Part-Aware LSTM [22] | X |  |  | 62.9% | 70.3%
ED-LSTM [19] |  | X |  | 66.2% | -
LSTM with Trust Gates [17] | X |  |  | 69.2% | 77.7%
Res-TCN [13] | X |  |  | 74.3% | 83.1%
DSSCA-SSLM [23] |  | X | X | 74.9% | -
D+S CNN [21] | X | X |  | 75.2% | 83.1%
D-Pose Traversal [35] | X |  |  | 76.8% | 84.9%
ESV-CNN [18] | X |  | X | 80.0% | 87.2%
Chained-MSN [37] | X | X | X | 80.8% | -
Unsupervised [16] |  | X | X | 80.9% | 83.4%
TS-RNNCNN [36] | X |  | X | 83.7% | 93.6%
CT-DAN [34] |  | X | X | 86.4% | 89.0%
Glimpse Clouds [3] | X |  | X | 86.6% | 93.2%
DA-Net [29] |  |  | X | 88.1% | 92.0%
Ours |  | X | X | 89.8% | 94.5%
Table 2: Results of our model compared to state-of-the-art methods on the NTU
RGB+D dataset. We present only results from the full 2-stream network, our
best performing model.

###  4.2 Training and Implementation Details

####  4.2.1 Sampling parameters

As input to the network, $L=5$ frames were chosen to create the volumetric
motion representations from $K=10$ segments of the video. In the NTU RGB+D
dataset, we found this choice to be reasonable as it captures nearly 60% of
the average video. The size of the voxel grid was chosen to be $54\times
54\times 54$. These values were chosen for a number of reasons. Most
importantly, when visualized, it is a large enough voxel grid for a human to
clearly recognize the actions being performed in the scene. Furthermore, as
discussed below, it allows 3D pooling layers after each of the 4 convolution
layers to reduce the final output to a vector of size $1\times N$ where $N$ is
the number of convolutions in the final layer.

####  4.2.2 CNN parameters

We selected the size and configuration of the convolutional layers in our
temporal stream to be similar to both [24] and [26]. These are enumerated on
the right hand side of Fig. 3. There are 4 layers of 3D convolutions with
stride 1, 3D batch-normalization, ReLU activation, and Max Pooling. The kernel
sizes were $3^{3}$, $3^{3}$, $3^{3}$, and $2^{3}$ respectively. The output
after the final pooling operation is a vector of size $512$. The global
temporal model (seen in Fig. 2) is an LSTM with input size 512 and hidden size
256. Dropout is not applied in this network. The final logits layer has an
output equal to the number of classes (60 in NTU RGB+D). The 2D equivalent
model, discussed below, has the exact same configuration except with 2D
convolutions and 2D batch-normalization. The predictions of the two-stream
model uses the product of experts method [10] of multiplying each stream’s
softmax output, however, gradient calculations for backprop were performed
before the fusion of the two networks.

####  4.2.3 Training details

We use cross-entropy loss to optimize our models. The ADAM [14] optimizer is
used with a learning rate of 0.001, reducing it by 50% every 10 epochs. We
train all models for 40 epochs or until the training accuracy has saturated.
For the spatial stream, the ResNet-18 implementation in Pytorch was used. We
use pre-trained ImageNet weights, however the weights of the first half of the
network were frozen to avoid overfitting. All experiments were run on a
machine with dual Titan-RTX GPUs. The batch size was 8 (videos) when training
the motion stream and 128 for the spatial stream. These were chosen because of
GPU memory limitations. For training it took on average 33 minutes per epoch
for the motion stream and 6 minutes per epoch for the spatial stream (each
epoch being a full pass over the training split). Inference over the entire
test set took an average of 12 minutes, approximately 23 videos per second.
This greatly exceeds real-time performance as it runs at over 1,000 frames per
second. In order to determine stopping criteria during training, 5% of the
training samples were reserved as a validation set.

####  4.2.4 Data augmentation and pre-processing

During training, random translations, as described in Section 3.2 are applied
up to $\pm 10\%$ the total size of the grid. Random out-of-plane rotations of
voxels and motion vectors are applied at $\pm 30^{\circ}$ about the axis
normal to the ground-plane. We examine the performance of these techniques in
an ablation study over the NTU RGB+D dataset (Table 3). For the spatial stream
over color images, the images need to be pre-processed before being passed
through ResNet by resizing the images to (224,224) and normalizing the color
values to the same mean and standard deviation. During training, we augment
images through random crops, adding random color jitter, and rotating them
$\pm 15^{\circ}$. The random parameters of these pre-processing and
augmentation techniques are consistent across an individual sample (video),
varying only between samples and across training iterations. During inference,
no augmentation is applied, only rescaling and normalization. To obtain the 2D
optical flow images, the off-the-shelf implementation of [6] in the OpenCV
toolkit is used.

###  4.3 Experimental Results and Analysis

An ablative experiment designed to evaluate our input representation is
presented in Table 1. We first investigate the results from the spatial and
temporal stream alone. Additionally, we investigate the 3D representation of
motion in our temporal stream with a comparison to the 2D equivalent. We can
see that the individual spatial and temporal streams achieve competitive
results on their own, with performance rivaling or better than many of the
state of the art results presented in Table 2. However, the best results come
from the full 2-stream model, showing an increase in accuracy of up to 14%
above its respective individual stream. This corroborates that motion and
spatial features are discriminative on a disjoint set of features and the full
two-stream approach is the most effective.

Using a 3D representation of motion shows a significant performance increase
in comparison to the 2D equivalent. The largest increase comes in the cross-
view split with an 18.5% performance increase in classification accuracy. We
hypothesize these results show that out-of-plane data augmentation techniques
with a 3D representation better capture geometric (view-point) invariance than
the 2D equivalent. To substantiate these claims, we design an ablative
experiment of the augmentation presented in Table 3 which we discuss in detail
below. Of note, we can see that there is no significant difference between the
results of using a 3D representation without data augmentation techniques and
its 2D equivalent. This shows that a 3D representation alone is not enough.

###  4.4 Comparison with the State of the Art

Table 2 presents the results of previous works compared to our proposed 3D
two-stream approach. For sake of comparison, we also include the modalities
the various models use for training their respective models. Although various
modalities were used in the methods we compare against, we felt it fair to
compare across all of them, as all outputs from an RGB-D sensor (depth, RGB,
and pose) are available for download. These results show that our methodology
improves over the previous state-of-the-art results on this dataset.

###  4.5 Exploration - Data Augmentation

We conduct further experiments to understand the difference in performance
when using 3D data augmentation techniques. Table 3 presents the results of
this ablation study. Applying random translations to the voxel grids we see a
$5-8\%$ increase in the classification rate of each evaluation metric.
Applying random out-of-plane rotations to the volumetric representation boosts
performance by another $2-9\%$. It is interesting to note that the the
relative performance gain on the cross-view split (17.73%) is much higher than
the cross-subject split (7.79%). Because the cross-view split is from a
different camera angle, we believe this illustrates that out-of-plane
rotations and translations during training help the network become invariant
to view-point as evidenced by this significant perfomance gain.

Augmentation Technique | Cross-Subject | Cross-View
---|---|---
No augmentation | 71.18% | 71.85%
Translations | 76.70% | 80.08%
Translations + Rotations | 78.97% | 89.58%
Table 3: Results on NTU RGB+D: augmentation ablation study. This shows
classification accuracy when using only the volumetric motion representation
(the temporal stream of the network) with the various augmentation techniques
applied to the input.

The volumetric representation created in this work is from a single RGB-D
camera. This is limiting because the type of video, referred to as 2.5D, can
only capture partial surfaces from the depth-map, i.e. the camera-facing
surface. Without complete surface reconstruction, the data augmentations
through virtually rotating and translating the scene is limiting due to
missing information. However, our results show a clear benefit towards
applying this technique nonetheless, evidenced by the significant performance
gain shown in Table 3.

##  5 Conclusions

We propose a method for activity recognition using a volumetric representation
of 3D motion. The method achieves state of the art performance on both the
cross-subject and cross-view evaluation metrics of the NTU RGB+D dataset. A
novel representation of motion is created from RGB-D video by projecting a
dense optical flow field, calculated over RGB frames, into a 3D voxel grid
using the corresponding depth map. A two-stream convolutional network is
applied over short snippets of video and an LSTM is used to model the temporal
structure over the snippets. In the two-steam network, the spatial stream uses
a pre-trained object recognition network for the RGB frames and for the
temporal stream we define a 3D CNN formulation over the volumetric 3D motion
field. In our experiments, we show that this 3D representation outperforms the
equivalent 2D representation. Furthermore, we show that the out-of-plane data
augmentation techniques that are possible over a 3D representation can
significantly improve performance. Future work will investigate approaches to
fuse the temporal and spatial stream to allow learning of complementary
filters across streams.

## References

  * [1] M. Baccouche, F. Mamalet, C. Wolf, C. Garcia, and A. Baskurt.  Action classification in soccer videos with long short-term memory recurrent neural networks.  In International Conference on Artificial Neural Networks, pages 154–159. Springer, 2010.
  * [2] M. Baccouche, F. Mamalet, C. Wolf, C. Garcia, and A. Baskurt.  Sequential deep learning for human action recognition.  In International Workshop on Human Behavior Understanding, pages 29–39. Springer, 2011.
  * [3] F. Baradel, C. Wolf, J. Mille, and G. W. Taylor.  Glimpse clouds: Human activity recognition from unstructured feature points.  arXiv preprint arXiv:1802.07898, 2018.
  * [4] J. Carreira and A. Zisserman.  Quo vadis, action recognition? a new model and the kinetics dataset.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4724–4733. IEEE, 2017.
  * [5] J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell.  Long-term recurrent convolutional networks for visual recognition and description.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2625–2634, 2015.
  * [6] G. Farnebäck.  Two-frame motion estimation based on polynomial expansion.  In Scandinavian conference on Image analysis, pages 363–370. Springer, 2003.
  * [7] C. Feichtenhofer, A. Pinz, and A. Zisserman.  Convolutional two-stream network fusion for video action recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1933–1941, 2016.
  * [8] A. Haque, A. Alahi, and L. Fei-Fei.  Recurrent attention models for depth-based person identification.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1229–1238, 2016.
  * [9] K. He, X. Zhang, S. Ren, and J. Sun.  Deep residual learning for image recognition.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016.
  * [10] G. E. Hinton.  Training products of experts by minimizing contrastive divergence.  Neural computation, 14(8):1771–1800, 2002.
  * [11] S. Ji, W. Xu, M. Yang, and K. Yu.  3d convolutional neural networks for human action recognition.  IEEE transactions on pattern analysis and machine intelligence, 35(1):221–231, 2013.
  * [12] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei.  Large-scale video classification with convolutional neural networks.  In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 1725–1732, 2014.
  * [13] T. S. Kim and A. Reiter.  Interpretable 3d human action analysis with temporal convolutional networks.  In 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1623–1631. IEEE, 2017.
  * [14] D. P. Kingma and J. Ba.  Adam: A method for stochastic optimization.  arXiv preprint arXiv:1412.6980, 2014.
  * [15] C. Lea, M. D. Flynn, R. Vidal, A. Reiter, and G. D. Hager.  Temporal convolutional networks for action segmentation and detection.  In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 156–165, 2017.
  * [16] J. Li, Y. Wong, Q. Zhao, and M. Kankanhalli.  Unsupervised learning of view-invariant action representations.  In Advances in Neural Information Processing Systems, pages 1262–1272, 2018.
  * [17] J. Liu, A. Shahroudy, D. Xu, and G. Wang.  Spatio-temporal lstm with trust gates for 3d human action recognition.  In European Conference on Computer Vision, pages 816–833. Springer, 2016.
  * [18] M. Liu, H. Liu, and C. Chen.  Enhanced skeleton visualization for view invariant human action recognition.  Pattern Recognition, 68:346–362, 2017.
  * [19] Z. Luo, B. Peng, D.-A. Huang, A. Alahi, and L. Fei-Fei.  Unsupervised learning of long-term motion dynamics for videos.  arXiv preprint arXiv:1701.01821, 2, 2017.
  * [20] J. Y.-H. Ng, M. Hausknecht, S. Vijayanarasimhan, O. Vinyals, R. Monga, and G. Toderici.  Beyond short snippets: Deep networks for video classification.  In Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on, pages 4694–4702. IEEE, 2015.
  * [21] H. Rahmani and M. Bennamoun.  Learning action recognition model from depth and skeleton videos.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5832–5841, 2017.
  * [22] A. Shahroudy, J. Liu, T.-T. Ng, and G. Wang.  Ntu rgb+ d: A large scale dataset for 3d human activity analysis.  arXiv preprint arXiv:1604.02808, 2016.
  * [23] A. Shahroudy, T.-T. Ng, Y. Gong, and G. Wang.  Deep multimodal feature analysis for action recognition in rgb+ d videos.  IEEE transactions on pattern analysis and machine intelligence, 2017\.
  * [24] K. Simonyan and A. Zisserman.  Two-stream convolutional networks for action recognition in videos.  In Advances in neural information processing systems, pages 568–576, 2014.
  * [25] B. Singh, T. K. Marks, M. Jones, O. Tuzel, and M. Shao.  A multi-stream bi-directional recurrent neural network for fine-grained action detection.  In Computer Vision and Pattern Recognition (CVPR), 2016 IEEE Conference on, pages 1961–1970. IEEE, 2016.
  * [26] S. Song and J. Xiao.  Deep sliding shapes for amodal 3d object detection in rgb-d images.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 808–816, 2016.
  * [27] L. Sun, K. Jia, D.-Y. Yeung, and B. E. Shi.  Human action recognition using factorized spatio-temporal convolutional networks.  In Proceedings of the IEEE International Conference on Computer Vision, pages 4597–4605, 2015.
  * [28] D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri.  Learning spatiotemporal features with 3d convolutional networks.  In Computer Vision (ICCV), 2015 IEEE International Conference on, pages 4489–4497. IEEE, 2015.
  * [29] D. Wang, W. Ouyang, W. Li, and D. Xu.  Dividing and aggregating network for multi-view action recognition.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 451–467, 2018.
  * [30] L. Wang, Y. Qiao, and X. Tang.  Action recognition with trajectory-pooled deep-convolutional descriptors.  In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4305–4314, 2015.
  * [31] L. Wang, Y. Xiong, Z. Wang, and Y. Qiao.  Towards good practices for very deep two-stream convnets.  arXiv preprint arXiv:1507.02159, 2015.
  * [32] L. Wang, Y. Xiong, Z. Wang, Y. Qiao, D. Lin, X. Tang, and L. Van Gool.  Temporal segment networks: Towards good practices for deep action recognition.  In European Conference on Computer Vision, pages 20–36. Springer, 2016.
  * [33] P. Wang, W. Li, Z. Gao, Y. Zhang, C. Tang, and P. Ogunbona.  Scene flow to action map: A new representation for rgb-d based action recognition with convolutional neural networks.  In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 595–604, 2017.
  * [34] P. Wang, W. Li, J. Wan, P. Ogunbona, and X. Liu.  Cooperative training of deep aggregation networks for rgb-d action recognition.  arXiv preprint arXiv:1801.01080, 2017.
  * [35] J. Weng, M. Liu, X. Jiang, and J. Yuan.  Deformable pose traversal convolution for 3d action and gesture recognition.  In Proceedings of the European Conference on Computer Vision (ECCV), pages 136–152, 2018.
  * [36] R. Zhao, H. Ali, and P. van der Smagt.  Two-stream rnn/cnn for action recognition in 3d videos.  arXiv preprint arXiv:1703.09783, 2017.
  * [37] M. Zolfaghari, G. L. Oliveira, N. Sedaghat, and T. Brox.  Chained multi-stream networks exploiting pose, motion, and appearance for action classification and detection.  In Computer Vision (ICCV), 2017 IEEE International Conference on, pages 2923–2932. IEEE, 2017.