for every $i$, $R^{Q}_{i+1}$ is a predecessor of $R^{Q}_{i}$

  * •

for every $i$, $R^{Q}_{i}$ contains a neighbourhood of $x$ in $Q$

Indeed, by the definition of a Markovian family there exists one rectangle
$R^{Q}_{0}$ containing a neighbourhood of $x$ in the quadrant $Q$. By Lemma
3.12, there exists a unique predecessor and successor of $R^{Q}_{0}$
containing a neighbourhood of $x$ in $Q$ and by Lemma 3.15 any other rectangle
with this property is a predecessor or a successor of some generation of
$R^{Q}_{0}$.

By Lemmas 3.9 and 3.10, we have that
$\\{x\\}=\overset{+\infty}{\underset{k=0}{\cap}}R^{Q}_{k}$. We therefore
define
$h(x):=\overset{+\infty}{\underset{k=0}{\cap}}h(R^{Q}_{k})\in\overline{\mathcal{P}_{2}}$.
Notice that for every $k$, $h(R^{Q}_{k})$ is a predecessor of
$h(R^{Q}_{k-1})$, therefore once again
$\overset{+\infty}{\underset{k=0}{\cap}}h(R^{Q}_{k})$ corresponds to a unique
point in $\overline{\mathcal{P}_{2}}$. We would now like to show that this
point does not depend on the initial choice of quadrant for $x$.

Independence from the choice of quadrant

Assume first that $x$ is not a boundary periodic point and that the previous
quadrant $Q$ was the $(\epsilon,\epsilon^{\prime})$ quadrant of $x$. Consider
the quadrant $(\epsilon,-\epsilon^{\prime})$ of $x$ and

$i:=\inf\\{k\in\mathbb{Z}|\forall j\geq k\leavevmode\nobreak\
R^{(\epsilon,\epsilon^{\prime})}_{j}\text{ contains a germ of the
$(\epsilon,\epsilon^{\prime})$ and $(\epsilon,-\epsilon^{\prime})$ quadrants
of $x$ }\\}$

Let us first notice that such an $i$ belongs in $\mathbb{Z}\cup\\{-\infty\\}$.
Indeed, suppose that $R^{(\epsilon,\epsilon^{\prime})}_{0}$ contains a germ of
the $(\epsilon,\epsilon^{\prime})$ quadrant of $x$, but not of the
$(\epsilon,-\epsilon^{\prime})$ quadrant of $x$. This implies that $x$ belongs
to a stable boundary component of $R^{(\epsilon,\epsilon^{\prime})}_{0}$, say
$s$. By Lemmas 4.4, 4.5 and 3.15 there exists $k_{0}$ sufficiently big such
that $R^{(\epsilon,\epsilon^{\prime})}_{k_{0}}$ is a $s$-crossing predecessor
of $R^{(\epsilon,\epsilon^{\prime})}_{0}$, that contains a germ of the
$(\epsilon,\epsilon^{\prime})$ and $(\epsilon,-\epsilon^{\prime})$ quadrants
of $x$. By the Markovian intersection property, for every $k\geq k_{0}$,
$R^{(\epsilon,\epsilon^{\prime})}_{k}$ also contains a germ of the
$(\epsilon,\epsilon^{\prime})$ and $(\epsilon,-\epsilon^{\prime})$ quadrants
of $x$. We thus get that $i\in\mathbb{Z}\cup\\{-\infty\\}$.

If $i=-\infty$, then the sequence associated to the quadrant
$(\epsilon,-\epsilon^{\prime})$ is the exact same sequence as for
$(\epsilon,\epsilon^{\prime})$. This can happen for instance when $x$ does not
belong to the stable or unstable leaf of some boundary periodic point.
Therefore, in this case we get the desired result.

If $i\in\mathbb{Z}$, then for every $j\geq i$ the rectangle
$R_{j}^{(\epsilon,\epsilon^{\prime})}$ is contained in the sequence associated
to the quadrant $(\epsilon,-\epsilon^{\prime})$. Let us denote this sequence
$...,R^{(\epsilon,-\epsilon^{\prime})}_{-1},R^{(\epsilon,-\epsilon^{\prime})}_{0},...,R^{(\epsilon,-\epsilon^{\prime})}_{i-1},R^{(\epsilon,-\epsilon^{\prime})}_{i}=R^{(\epsilon,\epsilon^{\prime})}_{i},R^{(\epsilon,-\epsilon^{\prime})}_{i+1}=R^{(\epsilon,\epsilon^{\prime})}_{i+1},...,R^{(\epsilon,-\epsilon^{\prime})}_{n}=R^{(\epsilon,\epsilon^{\prime})}_{n},...$.
The rectangle $R^{(\epsilon,-\epsilon^{\prime})}_{i-1}$ is the successor of
$R^{(\epsilon,-\epsilon^{\prime})}_{i}=R^{(\epsilon,\epsilon^{\prime})}_{i}$
that intersects $\overline{\mathcal{F}^{s}}(x)$ and that is not
$R^{(\epsilon,\epsilon^{\prime})}_{i-1}$. Assume without any loss of
generality that this is the successor of
$R^{(\epsilon,\epsilon^{\prime})}_{i}$ that is right above
$R^{(\epsilon,\epsilon^{\prime})}_{i-1}$ (see Figure 22). In this case, $h$
will send $R^{(\epsilon,-\epsilon^{\prime})}_{i-1}$ to the successor of
$h(R^{(\epsilon,-\epsilon^{\prime})}_{i})$ that is right above
$h(R^{(\epsilon,\epsilon^{\prime})}_{i-1})$. Furthermore, since
$R^{(\epsilon,-\epsilon^{\prime})}_{i-2}$ contains $x$ and a germ of its
$(\epsilon,-\epsilon^{\prime})$ quadrant,
$R^{(\epsilon,-\epsilon^{\prime})}_{i-2}$ is the unique successor of
$R^{(\epsilon,-\epsilon^{\prime})}_{i-1}$ that intersects
$\overline{\mathcal{F}^{s}}(x)$, which corresponds to the lowermost successor
of $R^{(\epsilon,-\epsilon^{\prime})}_{i-1}$. Therefore,
$h(R^{(\epsilon,-\epsilon^{\prime})}_{i-2})$ will correspond to the lowermost
successor of $h(R^{(\epsilon,-\epsilon^{\prime})}_{i-1})$, the successor of
$h(R^{(\epsilon,-\epsilon^{\prime})}_{i-1})$ that intersects
$\overline{\mathcal{F}^{s}}(h(x))$. We can thus show thanks to Lemma 3.10 that
$\overset{i-1}{\underset{k=-\infty}{\cap}}h(R^{(\epsilon,-\epsilon^{\prime})}_{k})$
corresponds to a stable segment in $\overline{\mathcal{F}^{s}}(h(x))$ crossing
$h(R^{(\epsilon,-\epsilon^{\prime})}_{i-1})$.

Figure 22.

We also know thanks to Lemma 3.9 that
$\overset{+\infty}{\underset{k=i}{\cap}}h(R^{(\epsilon,-\epsilon^{\prime})}_{k})=\overset{+\infty}{\underset{k=i}{\cap}}h(R^{(\epsilon,\epsilon^{\prime})}_{k})$
corresponds to an unstable segment in $\overline{\mathcal{F}^{u}}(h(x))$
crossing $h(R^{(\epsilon,-\epsilon^{\prime})}_{i})$. We conclude that
$\overset{\infty}{\underset{k=-\infty}{\cap}}h(R^{(\epsilon,-\epsilon^{\prime})}_{k})=\\{h(x)\\}$.
By a similar argument we can show that
$\overset{\infty}{\underset{k=-\infty}{\cap}}h(R^{(-\epsilon,-\epsilon^{\prime})}_{k})=\overset{\infty}{\underset{k=-\infty}{\cap}}h(R^{(-\epsilon,\epsilon^{\prime})}_{k})=\\{h(x)\\}$,
which gives us the desired result.

Assume now that $x$ is a boundary periodic point. In this case, we can define
$h(x)$ by the same exact argument for some choice of quadrant $Q$ of $x$ ($x$
has infinitely many quadrants in $\overline{\mathcal{P}_{1}}$). Consider now
$Q^{\prime}\neq Q$ the quadrant of $x$ that intersects $Q$ along a stable leaf
of $x$. By a similar argument, we can show that for every $N$,
$\overset{N}{\underset{k=-\infty}{\cap}}h(R^{Q^{\prime}}_{k})$ corresponds to
a stable segment in $\overline{\mathcal{F}^{s}}(h(x))$. Furthermore, if $t$ is
the generator of the stabilizer of a stable or unstable leaf of $x$, then for
any quadrant $q$ of $x$ we have that $(t^{k}(R^{q}_{0}))_{k\in\mathbb{Z}}$ is
a subsequence of $(R^{q}_{k})_{k\in\mathbb{Z}}$. Therefore,
$\big{(}\alpha(t)^{k}(h(R^{q}_{0}))\big{)}_{k\in\mathbb{Z}}$ is also a
subsequence of $(h(R^{q}_{k}))_{k\in\mathbb{Z}}$. Our previous arguments imply
that both $\overset{+\infty}{\underset{k=-\infty}{\cap}}h(R^{Q^{\prime}}_{k})$
and $\overset{+\infty}{\underset{k=-\infty}{\cap}}h(R^{Q}_{k})$ are fixed by
$\alpha(t)$ and belong to the same stable leaf in
$\overline{\mathcal{F}^{s}}$. They thus correspond to the same point $h(x)$,
which gives us the desired result.

$h(\overline{\Gamma_{1}})\subset\overline{\Gamma_{2}}$ and
$h(\overline{\mathcal{P}_{1}}-\overline{\Gamma_{1}})\subset\overline{\mathcal{P}_{2}}-\overline{\Gamma_{2}}$

Indeed, assume that $x\in\overline{\Gamma_{1}}$ and
$h(x)\in\overline{\mathcal{P}_{2}}-\overline{\Gamma_{2}}$. There are
infinitely many quadrants around $x$, therefore there exists an infinite
family of rectangles $(R_{i})_{i\in\mathbb{N}}$ of
$\overline{\mathcal{R}_{1}}$ intersecting $x$ and having 2 by 2 disjoint
interiors. For every $i$ the rectangle $h(R_{i})$ intersects $h(x)$ that has
only finitely many quadrants. Therefore, there exist $i\neq j$ such that
$\overset{\circ}{h(R_{i})}\cap\overset{\circ}{h(R_{j})}\neq\emptyset$.
Therefore, $h(R_{i})$ is a predecessor or successor of some generation of
$h(R_{j})$. This contradicts that the fact that $h$ defines a bijection from
$\overline{\mathcal{R}_{1}}$ to $\overline{\mathcal{R}_{2}}$ that respects the
relations of predecessor/successor.

By a similar argument we can show that
$h(\overline{\mathcal{P}_{1}}-\overline{\Gamma_{1}})\subset\overline{\mathcal{P}_{2}}-\overline{\Gamma_{2}}$

$h$ is a homeomorphism

By applying all the above arguments for $\overline{\mathcal{P}_{2}}$, we can
show that $h$ admits a right and left inverse therefore $h$ is a bijection.

Consider now $x\in\overline{\mathcal{P}_{1}}-\overline{\Gamma_{1}}$ and
$y:=h(x)\in\overline{\mathcal{P}_{2}}-\overline{\Gamma_{2}}$. Denote by
$(R^{(\epsilon,\epsilon^{\prime})}_{k}(x))_{k\in\mathbb{Z}}$ the sequence of
rectangles defined in the previous paragraphs for $x$ and the quadrant
$(\epsilon,\epsilon^{\prime})$. The continuity of $h$ on $x$ follows for the
fact that
$(\underset{(\epsilon,\epsilon^{\prime})\in\\{+,-\\}^{2}}{\cup}\overset{N}{\underset{k=-N}{\cap}}R^{(\epsilon,\epsilon^{\prime})}_{k}(x))_{N\in\mathbb{N}}$
and
$(\underset{(\epsilon,\epsilon^{\prime})\in\\{+,-\\}^{2}}{\cup}\overset{N}{\underset{k=-N}{\cap}}h(R^{(\epsilon,\epsilon^{\prime})}_{k}(x)))_{N\in\mathbb{N}}$
form respectively a basis of neighbourhoods of $x$ in
$\overline{\mathcal{P}_{1}}$ and of $y$ in $\overline{\mathcal{P}_{2}}$.

Consider now $x\in\overline{\Gamma_{1}}$, $y:=h(x)\in\overline{\Gamma_{2}}$,
$\\{Q_{k}(y)|k\in\mathbb{Z}\\}$ (resp. $\\{Q_{k}(x)|k\in\mathbb{Z}\\}$) the
set of all quadrants of $y$ (resp. $x$) and $\mathcal{F}$ the set of all
functions from $\mathbb{Z}$ to $\mathbb{N}^{*}$. Without any loss of
generality we will assume that $h(Q_{k}(x))=Q_{k}(y)$. Denote by
$(R^{Q_{k}(x)}_{l}(x))_{l\in\mathbb{Z}}$ the sequence of rectangles defined in
the previous paragraphs for $x$ and the quadrant $Q_{k}(x)$. Once again the
continuity of $h$ on $x$ follows from the fact that
$(\underset{k\in\mathbb{Z}}{\cup}\overset{f(k)}{\underset{l=-f(k)}{\cap}}R^{Q_{k}(x)}_{l}(x))_{f\in\mathcal{F}}$
and
$(\underset{k\in\mathbb{Z}}{\cup}\overset{f(k)}{\underset{l=-f(k)}{\cap}}h(R^{Q_{k}(x)}_{l}(x)))_{f\in\mathcal{F}}$
form respectively a basis of neighbourhoods of $x$ in
$\overline{\mathcal{P}_{1}}$ and $y$ in $\overline{\mathcal{P}_{2}}$.

The above prove that $h$ is continuous. By applying the same arguments to
$h^{-1}$, we get that $h$ is a homeomorphism.

$h$ sends oriented stable/unstable leaves to oriented stable/unstable leaves

Consider $x\in\overline{\mathcal{P}_{1}}$. Notice that for any choice of
quadrant of $x$, say $q$, we have that
$\overline{\mathcal{F}^{u}}(x)=\overset{\infty}{\underset{k=0}{\cup}}\overset{\infty}{\underset{l=k}{\cap}}R^{q}_{l}$
Therefore,

$h(\overline{\mathcal{F}^{u}}(x))=\overset{\infty}{\underset{k=0}{\cup}}\overset{\infty}{\underset{l=k}{\cap}}h(R^{q}_{l})=\overline{\mathcal{F}^{u}}(h(x))$

In the same way, we can show that $h$ sends stable leaves in
$\overline{\mathcal{P}_{1}}$ to stable leaves in $\overline{\mathcal{P}_{2}}$.

Next, recall that $\overline{\mathcal{F}_{1}^{s,u}}$ and
$\overline{\mathcal{F}_{2}^{s,u}}$ have been endowed with orientations such
that the map $h$ sends the predecessors of any rectangle
$R\in\overline{\mathcal{R}_{1}}$ ordered from left to right, to the
predecessors of $h(R)$ ordered from left to right. Same for the successors of
any rectangle in $\overline{\mathcal{R}_{1}}$. We deduce that $h$ respects the
orientations of $\overline{\mathcal{F}_{1}^{s,u}}$ and
$\overline{\mathcal{F}_{2}^{s,u}}$.

$h$ is equivariant with respect to the actions of the fundamental groups

Consider $x\in\overline{\mathcal{P}_{1}}$ and $g\in\pi_{1}(M_{1}-\Gamma_{1})$.
For any quadrant $q$ of $x$, we have that (see Proposition 8.1):

$h(g(x))=h(g(\overset{+\infty}{\underset{l=-\infty}{\cap}}R^{q}_{l}))=\overset{+\infty}{\underset{l=-\infty}{\cap}}h(g(R^{q}_{l}))=\overset{+\infty}{\underset{l=-\infty}{\cap}}\alpha(g)(h(R^{q}_{l}))=\alpha(g)(h(x))$

∎

##  9\. Cycles around boundary periodic points

Fix $(M,\Phi)$ a transitive Anosov flow, $\mathcal{P}$ its bifoliated plane
endowed with an orientation, $\mathcal{F}^{s,u}$ the stable/unstable
foliations in $\mathcal{P}$ endowed with an orientation, $\mathcal{R}$ a
Markovian family on $\mathcal{P}$ and $\Gamma\subset\mathcal{P}$ the set of
boundary points of $\mathcal{R}$.

In Section 7 we introduced the notion of cycle around a boundary arc point of
$\overline{\mathcal{P}}$. We would like in this section to adapt this notion
of cycle for boundary periodic points, in order to obtain information about
the neighbourhood of these orbits in $M$. This will allow us to prove Theorems
D and E.

######  Lemma 9.1.

Let $p$ be a boundary periodic point in $\mathcal{P}$ and $R\in\mathcal{R}$ a
rectangle containing $p$. Suppose that $p$ is contained in the stable boundary
component $s$ of $R$. There exists no $s$-crossing predecessor of $R$
containing $p$.

###### Proof.

Suppose that such a rectangle $L$ exists. Consider $g\in\text{Stab}(p)$ such
that the stable boundaries of $g(R)$ become very thin and the unstable ones
very long. The intersection of the rectangles $g(R)$ and $L$ is not Markovian.
Absurd. ∎

According to the previous lemma, we can not extend Definition 7.13 for
boundary periodic points.

Take $p\in\mathcal{P}$ a boundary periodic point and $L_{0}\in\mathcal{R}$
containing $p$. Assume without any loss of generality that $L_{0}$ contains a
germ of the $(+,+)$ quadrant of $p$ (there are only a finite number of
rectangles with this property up to the action of $\text{Stab}(p)$). Consider
now the set $Q_{(-,+)}$ of rectangles $r$ intersecting a germ of the $(-,+)$
quadrant of $p$ and such that $L_{0}\cap\mathcal{F}^{u}_{+}(p)\subseteq
r\cap\mathcal{F}^{u}_{+}(p)$. The set $Q_{(-,+)}$ is not stable under the
action of $\text{Stab}(p)$, but it is contained in the union of a finite
number of $\text{Stab}(p)$-orbits of rectangles. We can therefore define
$L_{1}$ as the unique rectangle in $Q_{(-,+)}$ for which for all $r\in
Q_{(-,+)}$

$L_{1}\cap\mathcal{F}^{u}_{+}(p)\subseteq r\cap\mathcal{F}^{u}_{+}(p)$

Notice that $L_{1}$ may be equal to $L_{0}$, if $L_{0}$ intersects a germ of
the $(+,+)$ and $(-,+)$ quadrants of $p$.

Next, we define $Q_{(-,-)}$, the set of rectangles $r$ intersecting a germ of
the $(-,-)$ quadrant of $p$ and such that
$L_{1}\cap\mathcal{F}^{s}_{-}(p)\subset r\cap\mathcal{F}^{s}_{-}(p)$. Again,
we define $L_{2}$ as the unique rectangle in $Q_{(-,-)}$ for which all $r\in
Q_{(-,-)}$

$L_{2}\cap\mathcal{F}^{s}_{-}(p)\subseteq r\cap\mathcal{F}^{s}_{-}(p)$

Finally, if the rectangle $L_{0}$ intersects a germ of the $(+,-)$ quadrant of
$p$, we take $L_{3}=L_{0}$. Otherwise, we define analogously $Q_{(+,-)}$ and
we define $L_{3}$ as the unique rectangle in $Q_{(+,-)}$ for which all $r\in
Q_{(+,-)}$

$L_{3}\cap\mathcal{F}^{u}_{-}(p)\subseteq r\cap\mathcal{F}^{u}_{-}(p)$

Figure 23. Some examples of positive pre-cycles starting from $L_{0}$

By Lemma 9.1 and our previous discussion, we have that any two rectangles
among $L_{0},L_{1},L_{2},L_{3}$ have either disjoint interiors or are equal
(see Figure 23). By eventually removing a rectangle appearing twice, we can
assume that the rectangles $L_{0},L_{1},L_{2},L_{3}$ have two by two disjoint
interiors. We will call $(L_{0},L_{1},L_{2},L_{3})$ _the positive pre-cycle
around $p$ starting from $L_{0}$_. By changing the cyclic order in which we
visit the quadrants around $p$, we can define similarly _the negative pre-
cycle around $p$ starting from $L_{0}$_.

######  Remark 9.2.

To every rectangle intersecting $p$ we associate canonically a positive and a
negative pre-cycle around $p$. The image of a pre-cycle around $p$ by an
element of $\text{Stab}(p)$ is a pre-cycle around $p$. Therefore, there is a
finite number of pre-cycles around $p$ up to the action of $\text{Stab}(p)$.

######  Proposition 9.3.

To any pre-cycle around $p$ we can associate canonically a closed rectangle
path of $\mathcal{P}$.

###### Proof.

Since a generalized rectangle path can canonically be associated to a
rectangle path (see Definition 7.18), it suffices to show that we can
canonically associate any pre-cycle to a generalized rectangle path in
$\mathcal{P}$ beginning and ending by the same rectangle. Let us consider
$L_{0}$ intersecting $p$ and $L_{0},...,L_{k}$ its associated positive pre-
cycle. The case of negative pre-cycles follows from the same arguments. Assume
without any loss of generality that the previous cycle is of the from
$L_{0},L_{1},L_{2},L_{3}$, where the $L_{i}$ have two by two disjoint
interiors and that $L_{0}$ contains a germ of the $(+,+)$ quadrant of $p$. All
the other cases, follow by the same arguments.

We begin our generalized rectangle path by $L_{0}$. Let
$s_{0}=L_{0}\cap\mathcal{F}^{u}_{+}(p)$ and $r_{0}$ be the $s_{0}$-crossing
successor of $L_{0}$ containing the endpoint of $s_{0}$ that is not $p$ (this
rectangle exists by Lemma 4.5). By construction of $L_{1}$, $r_{0}$ is a
successor of $L_{1}$ of some generation. Similarly, we define
$s_{1}:=L_{1}\cap\mathcal{F}^{s}_{-}(p)$,
$s_{2}:=L_{2}\cap\mathcal{F}^{u}_{-}(p)$, $r_{1}$ as the $s_{1}$-crossing
predecessor of $L_{1}$ containing the endpoint of $s_{1}$ that is not $p$ and
$r_{2}$ as the $s_{2}$-crossing successor of $L_{2}$ containing the endpoint
of $s_{2}$ that is not $p$. Again, $r_{1}$ (resp. $r_{2}$) is a predecessor
(resp. successor) of $L_{2}$ (resp. $L_{3}$) of some generation.

Finally, if $L_{0}\cap\mathcal{F}^{s}_{+}(p)\subset
L_{3}\cap\mathcal{F}^{s}_{+}(p)$ we define
$s_{3}:=L_{0}\cap\mathcal{F}^{s}_{+}(p)$ and $r_{3}$ as the $s_{3}$-crossing
predecessor of $L_{0}$ intersecting the endpoint of $s_{3}$ that is not $p$.
If instead, $L_{3}\cap\mathcal{F}^{s}_{+}(p)\subset
L_{0}\cap\mathcal{F}^{s}_{+}(p)$ we define
$s_{3}:=L_{3}\cap\mathcal{F}^{s}_{+}(p)$ and $r_{3}$ as the $s_{3}$-crossing
predecessor of $L_{3}$ intersecting the endpoint of $s_{3}$ that is not $p$.
In any case, $r_{3}$ is a predecessor of some generation of $L_{0}$ and also
$L_{3}$.

We deduce that $(L_{0},r_{0},L_{1},r_{1},L_{2},r_{2},L_{3},r_{3},L_{0})$ is a
generalized rectangle path in $\mathcal{P}$ that is canonically associated to
the pre-cycle $(L_{0},L_{1},L_{2},L_{3})$, which gives us the desired result.
∎

######  Definition 9.4.

Take $p$ a boundary periodic point. The set of rectangle paths associated to
all pre-cycles around $p$ will be called the set of _cycles_ of $p$ and will
be denoted by $\text{Cycl}(p)$.

######  Remark 9.5.

If $c$ is a pre-cycle around $p$ and $g\in\text{Stab}(p)$. The image by $g$ of
the cycle associated to $c$, is the cycle associated to the pre-cycle $g(c)$.
Therefore, by Remark 9.2, the set of cycles around any boundary periodic orbit
$p$ consists of a finite number of orbits by the action of $\text{Stab}(p)$.

More generally,

######  Remark 9.6.

For any $g\in\pi_{1}(M)$ and any boundary periodic point $p$,
$g(\text{Cycl}(p))=\text{Cycl}(g(p))$. Therefore, the set of all cycles around
any boundary periodic point in the $\pi_{1}(M)$-orbit of $p$, consists of a
finite number of rectangle paths up to the action of $\pi_{1}(M)$.

We will now prove that each of the previous orbits of rectangle paths can be
described as a closed rectangle path in any geometric type associated to
$\mathcal{R}$ by Theorem 3.17.

######  Definition 9.7.

Consider a geometric type $G=(n,(h_{i})_{i\in\llbracket
1,n\rrbracket},(v_{i})_{i\in\llbracket
1,n\rrbracket},\mathcal{H},\mathcal{V},\phi,u)$, where
$\mathcal{H}=\\{H_{i}^{j},i\in\llbracket 1,n\rrbracket,j\in\llbracket
1,h_{i}\rrbracket\\}$ and $\mathcal{V}=\\{V_{i}^{j},i\in\llbracket
1,n\rrbracket,j\in\llbracket 1,v_{i}\rrbracket\\}$. We define a _rectangle
path_ in $G$ as a finite sequence of the form
$i_{0},s_{0},i_{1},s_{1},i_{2},...,s_{m},i_{m+1}$ where

  * •

for all $k\in\llbracket 0,m+1\rrbracket$, $i_{k}\in\llbracket 1,n\rrbracket$

  * •

for all $k\in\llbracket 0,m\rrbracket$, $s_{k}$ is of the form
$V_{i_{k}}^{\bullet}$ or $H_{i_{k}}^{\bullet}$.

  * •

for all $k\in\llbracket 0,m\rrbracket$, if $s_{k}\in\mathcal{V}$ (resp.
$s_{k}\in\mathcal{H}$) then it is the image by $\phi$ (resp. $\phi^{-1}$) of
an element of $\mathcal{H}$ (resp. $\mathcal{V}$) of the form
$H_{i_{k+1}}^{\bullet}$ (resp. $V_{i_{k+1}}^{\bullet}$).

Furthermore, the rectangle path
$i_{0},s_{0},i_{1},s_{1},i_{2},...,s_{m},i_{m+1}$ will be called _closed_ when
$i_{0}=i_{m+1}$

Recall that we may think of a geometric type $G$ as a set of rectangles
$R_{1},...,R_{n}$, endowed each with a collection of horizontal and vertical
subrectangles identified by the map $(\phi,u)$. In these terms, the reader may
think of a rectangle path in $G$ as a sequence of the form
$R_{0},s_{0},...,R_{m},s_{m},R_{m+1}$, where for every $k\in\llbracket
0,m\rrbracket$, $s_{k}$ corresponds to a vertical (resp. or horizontal)
subrectangle of $R_{k}$ and a horizontal (resp. or vertical) subrectangle of
$R_{k+1}$.

######  Proposition 9.8.

Let $G=(n,(h_{i})_{i\in\llbracket 1,n\rrbracket},(v_{i})_{i\in\llbracket
1,n\rrbracket},\mathcal{H},\mathcal{V},\phi,u)$ be one of the geometric types
associated to $\mathcal{R}$ by Theorem 3.17. Endow $\mathcal{F}^{s,u}$ with
orientations and choose $r_{1},...,r_{n}$ representatives of every rectangle
orbit in $\mathcal{R}$, so that $\mathcal{R}$ is canonically associated to
$G$, thanks to Lemma 3.19. We have that:

  * •

any rectangle path in $\mathcal{P}$ corresponds to a unique rectangle path in
$G$

  * •

conversely, any rectangle path in $G$ corresponds to a unique
$\pi_{1}(M)$-orbit of rectangle paths in $\mathcal{P}$

###### Proof.

Consider $l_{0},l_{1},...,l_{k}$ a rectangle path in $\mathcal{P}$. Once again
we will think of $G$ as a set of rectangles $R_{1},...,R_{n}$. By our proof of
Theorem 3.17, every $l_{i}$ corresponds to a unique rectangle $R_{l_{i}}$ of
the geometric type $G$. Furthermore, for every $i\in\llbracket 1,k\rrbracket$
$l_{i}$ is a successor or predecessor of $l_{i-1}$, therefore $l_{i}$
corresponds to a unique horizontal or vertical subrectangle $s_{l_{i}}$ of
$R_{l_{i-1}}$. We therefore associate the rectangle path $l_{0},...,l_{k}$ to
the sequence $R_{l_{0}},s_{l_{1}},....,R_{l_{k-1}},s_{l_{k}},R_{l_{k}}$. Using
our proof of Theorem 3.17, is easy to check that the above sequence is a
rectangle path in $G$.

Consider now $R_{0},s_{1},....,R_{k-1},s_{k},R_{k}$ a rectangle path in $G$.
By our proof of Theorem 3.17, the rectangle $R_{0}$ corresponds to a unique
$\pi_{1}(M)$ orbit of rectangles in $\mathcal{P}$. Let $l_{0}$ be a
representative of this orbit. Next, $s_{1}$ corresponds to a unique successor
or predecessor of $l_{0}$, say $l_{1}$ and $R_{2}$ to a unique predecessor or
successor of $l_{1}$, say $l_{2}$. By repeating the previous procedure, we can
associate $R_{0},s_{1},....,R_{k-1},s_{k},R_{k}$ to the rectangle path
$l_{0},...,l_{k}$ in $\mathcal{P}$. The previous rectangle path depends on our
choice of $l_{0}$. If we replace $l_{0}$ by $g(l_{0})$, where
$g\in\pi_{1}(M)$, it is easy to check that the resulting rectangle path will
be the image by $g$ of $l_{0},...,l_{k}$. We conclude that the rectangle path
$R_{0},s_{1},....,R_{k-1},s_{k},R_{k}$ is associated to a unique $\pi_{1}(M)$
orbit of rectangle paths in $\mathcal{P}$. ∎

According to Theorem B, an equivalence class of geometric types describes an
Anosov flow up to surgeries along boundary periodic orbits. We would like at
this point to define a combinatorial object describing an Anosov flow up to
just orbital equivalence.

######  Definition 9.9.

Let $G$ be a geometric type and $\mathcal{A}$ a finite set of closed rectangle
paths in $G$. We will call $(G,\mathcal{A})$ a _geometric type with cycles_.

Two geometric types with cycles $(G,\mathcal{A})$ and
$(G^{\prime},\mathcal{A}^{\prime})$ will be called _equivalent_ if there
exists an equivalence $h$ between $G$ and $G^{\prime}$ (see Definition 2.7)
that bijectively associates closed rectangle paths in $\mathcal{A}$ to closed
rectangle paths in $\mathcal{A}^{\prime}$.

######  Remark 9.10.

Using Remark 2.8, we can show that any equivalence class of geometric types
with cycles contains only a finite number of geometric types with cycles.

By Remark 3.18, for each choice of orientations on $\mathcal{F}^{s,u}$ and
each choice of representatives for every rectangle orbit in $\mathcal{R}$, we
can associate $\mathcal{R}$ to a unique geometric type $G$. By Remark 2.8, the
set of geometric types obtained in this way is finite.

Fix one of the previous geometric types $G$ (together with the choice of
representatives and orientations associated to it). Associate now every
boundary periodic point $p$ of $\mathcal{R}$ to the set of closed rectangle
paths $\text{Cycl}(p)$ (this association is canonical). By Proposition 4.2 and
Remark 9.6, the union of all the above cycles corresponds to finitely many
closed rectangle paths up to the action of $\pi_{1}(M)$. Hence, by Proposition
9.8, we get that we can canonically associate to $\mathcal{R}$ a finite set
$\mathcal{A}$ of closed rectangle paths in $G$. We will call $(G,\mathcal{A})$
the geometric type with cycles associated to $\mathcal{R}$ (for this choice of
orientations and representatives).

By repeating our proof of Theorem 3.17, it is now not hard to show

Theorem D Let $M$ be an orientable, closed and connected 3-manifold and $\Phi$
a transitive Anosov flow on $M$. To any Markovian family $\mathcal{R}$ of
$\Phi$ we can associate canonically a unique and finite class of equivalent
geometric types with cycles.

We would like now to prove Theorem E according to which an equivalence clas of
geometric types with cycles describes the flow up to orbital equivalence. More
precisely,

Theorem E Let $(M_{1},\Phi_{1}),(M_{2},\Phi_{2})$ be two transitive Anosov
flows, $\mathcal{R}_{1},\mathcal{R}_{2}$ two Markovian families in
$\mathcal{P}_{1},\mathcal{P}_{2}$ associated to the same equivalence class of
geometric types with cycles. Then $\Phi_{1}$ is orbitally equivalent to
$\Phi_{2}$.

###### Proof.

Let $\Gamma_{1},\Gamma_{2}$ be the boundary periodic points of
$\mathcal{R}_{1},\mathcal{R}_{2}$ and $\overline{\mathcal{P}_{1}}$,
$\overline{\mathcal{P}_{2}}$ the bifoliated planes of $\Phi_{1}$, $\Phi_{2}$
up to surgeries along $\Gamma_{1}$,$\Gamma_{2}$. Using Lemma 3.19, it is not
hard to show that by choosing properly orientations on
$\mathcal{F}^{s,u}_{1},\mathcal{F}^{s,u}_{2}$ and representatives for every
rectangle orbit in $\mathcal{R}_{1}$ and $\mathcal{R}_{2}$, we may assume that
the Markovian families $\mathcal{R}_{1}$ and $\mathcal{R}_{2}$ are associated
to the same geometric type with cycles. By Theorem B and Theorem 6.10, there
exist an isomorphism
$\alpha:\pi_{1}(M_{1}-\Gamma_{1})\rightarrow\pi_{1}(M_{2}-\Gamma_{2})$ and a
homeomorphism
$h:\overline{\mathcal{P}_{1}}\rightarrow\overline{\mathcal{P}_{2}}$ such that:

  * •

the image by $h$ of any oriented stable/unstable leaf in
$\overline{\mathcal{F}_{1}^{s,u}}$ is an oriented stable/unstable leaf in
$\overline{\mathcal{F}_{2}^{s,u}}$

  * •

for every $g\in\pi_{1}(M_{1}-\Gamma_{1})$ and every
$x\in\overline{\mathcal{P}_{1}}$ we have

$h(g(x))=\alpha(g)(h(x))$

Let us now go back to our proof of Theorem 6.10 and show that under the
additional hypothesis on the cycles of $\mathcal{R}_{1}$ and
$\mathcal{R}_{2}$, the flows $\Phi_{1}$ and $\Phi_{2}$ are orbitally
equivalent.

Indeed, in the proof of Theorem 6.10, for every periodic point
$\overline{\gamma}$ in $\overline{\Gamma_{1}}$ (resp. $\overline{\Gamma_{2}}$)
we defined a canonical basis
$s^{M_{1}}_{\overline{\gamma}},t^{M_{1}}_{\overline{\gamma}}$ (resp.
$s^{M_{2}}_{\overline{\gamma}},t^{M_{2}}_{\overline{\gamma}}$) of its
stabilizer group, where $s^{M_{1}}_{\overline{\gamma}}$
(resp.$s^{M_{2}}_{\overline{\gamma}}$) fixes all the stable (or unstable)
leaves intersecting $\overline{\gamma}$ and $t^{M_{1}}_{\overline{\gamma}}$
(resp.$t^{M_{2}}_{\overline{\gamma}}$) acts as a translation by either $+1$ or
$+2$ on the previous set depending on whether $\overline{\gamma}$ has
respectfully negative or positive eigenvalues. By convention, we will assume
that $s^{M_{1}}_{\overline{\gamma}}$ and $s^{M_{2}}_{\overline{\gamma}}$ act
as expansions on every unstable leaf crossing $\overline{\gamma}$. We also
proved that for every boundary periodic orbit
$\overline{\gamma}\in\overline{\mathcal{P}_{1}}$

$\alpha(s^{M_{1}}_{\overline{\gamma}})=s^{M_{2}}_{h(\overline{\gamma})}$
$\alpha(t^{M_{1}}_{\overline{\gamma}})=t^{M_{2}}_{h(\overline{\gamma})}+k\cdot
s^{M_{2}}_{h(\overline{\gamma})}$

where $k$ belongs in $\mathbb{Z}$ and only depends on the projection of the
orbit $\overline{\gamma}$ on $M_{1}$. We thus associated to every
$\gamma\in\Gamma_{1}$ an integer $k(\gamma)$. Finally, we showed that
$\Phi_{2}$ is orbitally equivalent to the flow obtained by performing a
surgery on $\Phi_{1}$ of coefficient $-k(\gamma)$ along every
$\gamma\in\Gamma_{1}$. Therefore, it suffices to show that for every
$\gamma\in\Gamma_{1}$ we have $k(\gamma)=0$.

Fix $\overline{\gamma}\in\overline{\Gamma_{1}}$ and $\gamma$ its projection on
$\mathcal{P}_{1}$. Consider a positive cycle around $\gamma$, say
$r_{0},...,r_{k}=r_{0}$. Lift the previous cycle to a rectangle path
$\overline{r_{0}},...,\overline{r_{k}}$ in $\overline{\mathcal{P}_{1}}$
intersecting $\overline{\gamma}$. By the construction of the canonical
geometric type with cycles associated to $\mathcal{R}_{1}$,$\mathcal{R}_{2}$
(for our choice of orientations and rectangle representatives), every cycle in
$\mathcal{P}_{1}$ corresponds to a cycle of the geometric type associated to
$\mathcal{R}_{1}$ and $\mathcal{R}_{2}$; therefore it also corresponds to a
cycle in $\mathcal{P}_{2}$. Therefore,
$h(\overline{r_{0}}),...,h(\overline{r_{k}})$ projects to a closed rectangle
path in $\mathcal{P}_{2}$.

Since $r_{0},...,r_{k}=r_{0}$ goes once around $\gamma$, if $\gamma$ has
positive (resp. negative) eigenvalues,
$t^{M_{1}}_{\overline{\gamma}}(\overline{r_{0}})=\overline{r_{k}}$ (resp.
$2\cdot t^{M_{1}}_{\overline{\gamma}}(\overline{r_{0}})=\overline{r_{k}}$).
Thus, by applying $h$, if $\gamma$ has positive (resp. negative) eigenvalues,
$[t^{M_{2}}_{h(\overline{\gamma})}+k(\gamma)\cdot
s^{M_{2}}_{h(\overline{\gamma})}](h(\overline{r_{0}}))=h(\overline{r_{k}})$
(resp. $[2\cdot t^{M_{2}}_{h(\overline{\gamma})}+2k(\gamma)\cdot
s^{M_{2}}_{h(\overline{\gamma})}](h(\overline{r_{0}}))=h(\overline{r_{k}})$).
By our previous arguments, $h(\overline{r_{0}})$ and $h(\overline{r_{k}})$
project to the same rectangles in $\mathcal{P}_{2}$, which is only possible
when $k(\gamma)=0$. ∎

## References

  * [A] V.Dmitry Anosov. _Geodesic flows on closed Riemannian manifolds of negative curvature_ , Trudy Mat. Inst. Steklov., 90 (1967), pp.3-210; Proc. Steklov Inst. Math., 90 (1967), pp.1-235
  * [Ba] Thierry Barbot. Géométrie transverse des flots d’Anosov, Thesis, Ecole Normale Supérieure de Lyon (1992)
  * [Ba1] Thierry Barbot. _Caractérisation des flots d’Anosov en dimension 3 par leurs feuilletages faibles_ , Ergodic Theory Dynam. Systems 15 (1995), 247-270.
  * [Ba2] Thierry Barbot. _Generalizations of the Bonatti-Langevin example of Anosov flow and their classification up to topological equivalence_ , COMMUNICATIONS IN ANALYSIS AND GEOMETRY, Volume 6, Number 4, (1998), pp.749-798
  * [BaFe] Thierry Barbot, Sergio Fenley. _Pseudo-Anosov flows in toroidal manifolds_ , Geometry & Topology 17 (2013) 1877–1954
  * [BaFe1] Thierry Barbot, Sergio Fenley. _Classification and rigidity of totally periodic pseudo-Anosov flows in graph manifolds_ , Ergodic Theory and Dynamical Systems, 35(6), (2015), 1681-1722
  * [BaMa] Thomas Barthelmé, Kathrynn Mann. _Orbit equivalences of $\mathbb{R}$-covered Anosov flows and applications_, arXiv:2012.11811
  * [Be] François Béguin. _Smale diffeomorphisms of surfaces: a classification algorithm_ , Discrete and Continuous Dynamical Systems 11, 2-3 (2004), 261–310
  * [BeBoYu] François Béguin, Christian Bonatti, Bin Yu. _Building Anosov flows on 3-manifolds_ , Geometry and Topology, Mathematical Sciences Publishers (2017), 21 (3), pp.1837 - 1930
  * [BeBoYu1] François Béguin, Christian Bonatti, Bin Yu. _A spectral-like decomposition for transitive Anosov flows in dimension three_ , Math. Z. 282, (2016) 889–912
  * [BeYu] François Béguin, Bin Yu. _A uniqueness theorem for transitive Anosov flows obtained by gluing hyperbolic plugs_ , arXiv:1905.08989
  * [BoIa] Christian Bonatti, Ioannis Iakovoglou. Anosov flows on 3-manifolds: the surgeries and the foliations, arXiv:2007.11518
  * [BoIa1] Christian Bonatti, Ioannis Iakovoglou. _Geometric types associated to Anosov flows in dimension 3_ (in progress)
  * [BoLa] Christian Bonatti, Rémi Langevin, avec la collaboration de Emmanuelle Jeandenans Difféomorphismes de Smale des surfaces, Astérisque, tome 250 (1998)
  * [BoMa] Jonathan Bowden, Kathryn Mann. _$C^{0}$ stability of Anosov boundary action and inequivalent Anosov flows_, Ann. Sci. Éc. Norm. Supér. (4) 55 (2022), no. 4, 1003-1046
  * [ClPi] Adam Clay, Tali Pinsky, _Graph manifolds that admit arbitrarily many Anosov flows_ , arXiv:2006.09101
  * [DeSh] Pierre Dehornoy, Mario Shannon. _Almost equivalence of algebraic Anosov flows_ , arXiv:1910.08457
  * [Fe] Sergio Fenley. _The structure of branching in Anosov flows of 3-manifolds_ , Comment. Math. Helv. 73 (2) (1998) 259-297.
  * [Fe1] Sergio Fenley. _Fenley, Sergio. Anosov flows in 3-manifolds_ , Ann. of Math. (2), 139 (1) (1994), pp.79-115.
  * [Fri] David Fried. _Transitive Anosov flows and pseudo-Anosov maps._ Topology 22 (1983), no. 3, pp.299-303.
  * [Ia] Ioannis Iakovoglou. _Markovian families for generalized Bonatti-Langevin flows_ , (in progress)
  * [Pa] Carlos Frederico Borges Palmeira. _Open manifolds foliated by planes_ , Annals of Mathematics Second Series, Vol. 107, No. 1 (1978), pp. 109-131
  * [Ra] M. Ratner, Markov partitions for C-flows on 3-dimensional manifolds, Mat. Zametki 6 (6) (1969), 693-704.
  * [Si1] Yakov Grigorevich Sinai Markov partitions and C-diffeomorphisms, Functional Anal. Appl. 2 (1968), 64-89.
  * [Si2] Yakov Grigorevich Sinai, Construction of Markov partitions, Functional Anal. Appl. 2 (1968), 70-80.
  * [Sh] Mario Shannon, _Hyperbolic models for transitive topological Anosov flows in dimension three_ , Preprint arXiv:2108.12000
  * [YaYu] Jiagang Yang, Bin Yu. _Classifying expanding attractors on figure eight knot complement space and non-transitive Anosov flows on Franks-Williams manifold_ , arXiv:2004.08921


# KOLOMVERSE: KRISO open large-scale image dataset for object detection in the
maritime universe

Abhilasha Nanda, Sung Won Cho, Hyeopwoo Lee, Jin Hyoung Park
Maritime Safety and Environmental Research Division
Korea Research Institute of Ships and Ocean Engineering
Daejeon, South Korea
$\\{$ananda, gungnir, leehyeopwoo<EMAIL_ADDRESS>Corresponding
author

###### Abstract

Over the years, datasets have been developed for various object detection
tasks. Object detection in the maritime domain is essential for the safety and
navigation of ships. However, there is still a lack of publicly available
large-scale datasets in the maritime domain. To overcome this challenge, we
present KOLOMVERSE, an open large-scale image dataset for object detection in
the maritime domain by KRISO (Korea Research Institute of Ships and Ocean
Engineering). We collected 5,845 hours of video data captured from 21
territorial waters of South Korea. Through an elaborate data quality
assessment process, we gathered around 2,151,470 4K resolution images from the
video data. This dataset considers various environments: weather, time,
illumination, occlusion, viewpoint, background, wind speed, and visibility.
The KOLOMVERSE consists of five classes (ship, buoy, fishnet buoy, lighthouse
and wind farm) for maritime object detection. The dataset has images of
3840$\times$2160 pixels and to our knowledge, it is by far the largest
publicly available dataset for object detection in the maritime domain. We
performed object detection experiments and evaluated our dataset on several
pre-trained state-of-the-art architectures to show the effectiveness and
usefulness of our dataset. The dataset is available at:
https://github.com/MaritimeDataset/KOLOMVERSE.

##  1 Introduction

In the field of computer vision and image processing, object detection is
immensely popular because of its ability to accurately identify and classify
objects from images. Object detection is used to localize and track objects in
an image or a scene. As it is used extensively, there are many publicly
available datasets for implementation, such as MS COCO [1], Pascal VOC [2],
Imagenet [3] and Google Open Images [4] to name a few. MS COCO is the most
frequently used public dataset and contains 328K images with 80 object
categories. Object detection methods have been extensively used for face
identification [5], pedestrian detection [6], vehicle position and velocity
estimation [7] and autonomous vehicles [8]. These deep learning methods have
become popular in the maritime domain as well with implementation of maritime
object detection, segmentation, and vessel re-identification among others.
However, due to the complexity and uncertainty of the sea environment, it is a
challenging task to collect and annotate data [9]. Therefore, there are few
maritime datasets and they are mostly not publicly available.

With the development of the shipping industry across the world, maritime
datasets play a pivotal role in the development of maritime autonomous surface
ships (MASS) as well as navigation safety [10, 11]. The maritime dataset
contains information about vessels and surrounding water objects that help
detect, monitor, and manage sea traffic, maritime safety and navigation, and
military surveillance to prevent illegal activities like smuggling,
infiltration, and dumping of pollutants.

To prevent accidents and ensure safety both during the day and night, maritime
image datasets help obtain a ship’s location, size, and direction, among other
information, to determine whether there exist other objects nearby [12]. There
are a few existing applications of object detection such as harbor
surveillance [13] or collision avoidance [14] for autonomously operating
vessels but a lack of publicly available data and the complexity of the sea
environment makes object detection in the maritime domain a complex and
expensive field of research [9]. Moreover, in contrast to other domains of
object detection such as street sign detection [15], pedestrian detection [6],
or even face detection [5], in the maritime domain, there are comparatively
few public benchmarks available. Hence, recent studies of object detection in
the maritime domain contain only a few comparable results.

Figure 1: Left: Twenty-one territorial waters of Korea for data collection.
Right: Top view of the ship installed with four-channel 4K NVR system.

In this paper, we present a maritime object image dataset, KOLOMVERSE, for
object detection. For data collection, we installed a four-channel 4K
(3840$\times$2160 pixels) network video recorder (NVR) system on the ship’s
front, rear, left, and right sides to collect one fps of video data and built
the KOLOMVERSE based on the images captured from the video data. Seventeen
ships with the NVR system collected the video data from 21 territorial waters
of Korea, which are overlaid on a Google Earth satellite imagery as shown in
Figure 1. We collected 5,845 hours of video data from November 2020 to
February 2021. Through a data quality assessment process, we acquired around
2,151,470 4K resolution images that cover various environments considering
weather, time (illumination and occlusion), viewpoint, background, wind speed,
visibility, and season. The KOLOMVERSE contains five categories (ship, buoy,
fishnet buoy, lighthouse, and wind farm) essential for the safety and
navigation of maritime vessels.

Several object detection models are publicly available such as the YOLO series
[16, 17, 18, 19], SSD [20], fast and faster RCNN (FRCNN) [21, 22], R-FCN [23],
Mask R-CNN [24] and CenterNet [25]. We trained and evaluated our dataset on
pre-existing state-of-the-art models to show that our dataset performs
effectively in deep learning applications and can be expanded to other
meaningful applications. We trained our dataset on benchmarking models, so the
results obtained from our dataset are reproducible. Our dataset and the
trained models are publicly available at
https://github.com/MaritimeDataset/KOLOMVERSE.

##  2 Related Work

There are a handful of maritime datasets available for ship detection and
classification. Prasad et al. [26] introduced the Singapore Maritime Dataset
(SMD). It is one of the few publicly available datasets specifically dedicated
to object detection in maritime environments, but it lacks representative
benchmarking results. Zhang et al. [27] introduced the Vais, a large dataset
that contains more than 1,000 paired RGB and infrared images among six ship
categories - merchant, sailing, passenger, medium, tug, and small. The dataset
consists of 2,865 images with 1,623 visible, 1,242 IR, and 154 nighttime IR
images. The dataset includes 264 uniquely named ships in 6 coarse-grained
categories: merchant ships, sailing ships, medium passenger ships, medium
"other" ships, tugboats, and small boats. Shao et al. [12] developed Seaships
for training and evaluating ship object detection algorithms. The dataset
consists of 31,455 images with six ship classes (ore carrier, bulk cargo
carrier, general cargo ship, container ship, fishing boat, and passenger
ship). Soloviev et al. [28] selected 135 videos from a sightseeing watercraft
operating between the cities of Turku and Ruissalo in South-West Finland,
along the Aura river and into the Finnish Archipelago. The total number of
annotated vessels in this dataset is 850. They also built a second real
maritime dataset in the Finnish Archipelago, consisting largely of maritime
vessels in open sea landscapes. This dataset consists of 1,750 images captured
using visible cameras. The number of vessel objects in the dataset is 9,137.

Other maritime datasets solve challenges like piracy detection, boat
monitoring, segmentation, and vehicle classification.The IPATCH dataset
(piracy detection) developed by Pationo et al. [29], uses multi-sensor
surveillance to protect a vessel at sea from piracy. The dataset solves
challenges from the PETS 2016 workshop, does Low-Level Video Analysis like
object detection and tracking, Mid-Level Video Analysis like ’simple’ event
detection: the behavior recognition of a single actor and High-Level Video
Analysis that is ’complex’ event detection: the behavior and interaction
recognition of several actors. Maritime Detection, Classification, and
Tracking Database (MarDCT) [30] is a database of videos and images containing
data coming from multiple sources (fixed, moving, and Pan-Tilt-Zoom cameras)
and from different scenarios. MarDCT aims to provide visual data that can help
develop intelligent surveillance systems for the maritime environment. The
data sets are divided according to the type of Ground Truth in: Detection,
Classification, and Tracking. Also known as the Argos Boat Classification
dataset, it is used for boat classification and contains images of boats
automatically extracted by the ARGOS system, which operates 24/7 in Venice
(Italy). The dataset comes from an incomparable environment like Venice, but
they present very interesting challenges to vehicle classification. The Marine
Obstacle Detection Dataset (MODD), introduced by Kristan et al. [31] contains
marine videos captured by unmanned surface vehicles (USVs). This dataset is
used to segment images into the sky, the shore, and the sea and detect
obstacles in the sea area. The contributions of our study compared with those
of the aforementioned studies are summarized as follows:

  1. 1.

Autonomous navigation systems need detailed information about the objects in
their environment for safe navigation. Training datasets containing
information not just about the maritime vessels but also about the important
surroundings produce more reliable object detection and navigation systems.
Unlike other maritime datasets that contain only ship or boat categories, our
dataset contains important sea environment categories such as a buoy, fishnet
buoy, lighthouse, and wind farm, in addition to ships.

  2. 2.

A large dataset is crucial for training object detection models efficiently.
Mostly, datasets with fewer samples contain less information and cannot
predict new data effectively. In addition, there is a higher possibility of
overfitting when training is done with smaller datasets [32]. The models built
with small datasets are biased, overfitting, and unreliable. In maritime
applications, building reliable models that ensure maximum safety is extremely
important. The KOLOMVERSE is comparatively much larger than other maritime
datasets, as shown in Table 1.

  3. 3.

It is important to recognize even the smallest objects visible from far away
in a real-time sea environment for effective tracking and safe navigation.
Networks trained on images with high resolution can locate and classify very
small objects visible from far as these images contain detailed information.
As shown in Appendix A, the KOLOMVERSE contains images with objects of various
shapes and sizes, even small ones visible from far away. As shown in Table 1,
the KOLOMVERSE contains images with a high resolution of 3840$\times$2160
pixels. It is much higher compared to other datasets.

  4. 4.

Our dataset has diverse samples because the images were collected from 21
different water bodies. The dataset also has variations in background, scale,
viewpoint, occlusion, and illumination. Therefore, models trained with our
dataset are robust. In addition, unlike other object detection datasets, our
dataset contains images of various shapes, sizes, and backgrounds from the
real-world sea environment that produces reliable real-time detection models.

Table 1: Comparison of the KOLOMVERSE with publicly available maritime object detection datasets. Datasets |  $\\#$ of images |  $\\#$ of classes | Image size
---|---|---|---
Seaships [12] | 31,455 |  6 (Ore carrier, bulk cargo carrier, general cargo ship, container ship, fishing boat, and passenger ship) | 1920$\times$1080
VAIS [27] | 2,865 |  6 (Merchant ships, sailing ships, medium passenger ships, medium “other” ships, tugboats, and small boats) | 1024$\times$768
SMD [26] | 17,450 |  10 (Ferry, buoy, vessel/ship, speed boat, boat, kayak, sail boat, swimming person, flying bird/plane, and other) | 1920$\times$1080
KOLOMVERSE | 2,151,470 |  5 (Ship, buoy, fishnet buoy, lighthouse, and wind farm) | 3840$\times$2160

##  3 Dataset Collection, Annotation and Quality Assessment

###  3.1 Data Collection

To obtain the video data, we installed a four-channel 4K NVR system as shown
in Figure 1. We installed four cameras per ship to record the front, rear,
left, and right directions along with GPS equipment to store location
information simultaneously in the recorded image. To reflect the
characteristics of the sea in Korea, we divided it into nine zones in the
South Sea, 10 zones in the West Sea, and two zones in the East Sea to collect
video data from various environments. The video data included variations in
contrast and color of objects according to various weather conditions and
times.

Data were collected according to specific criteria as follows:

  * •

Weather: Videos were collected by classifying the weather into clear, cloudy,
rainy, snowy, and strong windy. At wind speeds of 14 m/s or more per second,
sailing is prohibited, so such data were excluded from the collection.

  * •

Time: The color, clarity, and contrast of the objects are different depending
on the time, which varies the illuminance and the direction of light in the
image. For this reason, we classified the video as morning, afternoon,
sunrise, sunset, and night.

  * •

Wave height: Since the shaking and inclination of the images and the shapes of
the objects in the video vary depending on the wave height, we collected the
video by classifying it into 0, 1 to 2 meters, and 3 to 4 meters. Sailing is
prohibited for the wave height of 4 meters or more, so those data were
excluded from the collection.

  * •

Wind speed: As the movement of objects in the image and the shape of waves
differ depending on the wind speed, the wind speed information was classified
as 0, 1 m/s to 5 m/s, 5 m/s to 10 m/s, and 10 m/s or more. Sailing is
prohibited at wind speeds of 14 m/s or higher, so such data were excluded from
the collection.

  * •

Visibility: Since the color, resolution, and clarity of objects in the video
vary depending on the visibility, we collected the video by classifying 0.1 to
0.2 km, 0.2 to 0.5 km, 0.5 to 1 km, 1 to 2 km, and 2 km or more.

We employed 17 ships to collect the video data. The collected video data were
stored in mp4 format, and a total of 5,845 hours of data were collected with
4K resolution quality.

###  3.2 Data Annotation

Figure 2: Left: The distribution of the amount of time taken to collect the
video data from 21 territories. Right: Image sample with partially visible and
annotated objects.

First of all, the collected video data of 5,845 hours were converted to image
data in a JPG format. Figure 2 shows the distribution of the amount of time
taken to collect the video data from all the 21 territories. The image
sequences obtained from the video data were captured at one fps and the data
had some redundancy. Therefore, we removed the images with more than 80$\%$
similarity to reduce redundancy. Moreover, monotonous images consisting of
only the sea and islands without any of the five class objects were removed.
When there was a glare of sunlight over an object, possibly occluding the
object, or the image was of poor quality or had weak lighting that made it
difficult to identify the objects, such images were excluded from annotation.

In addition, the annotated bounding box must contain all parts of the object,
and the bounding box and the gap between the object should be less than 20
pixels. In cases of ship classes, sometimes persons or other objects were
captured within a ship. These objects were collectively labeled as the ship.
Moreover, if only a part of an object is visible, such as being obscured by
another object or truncated in the image, such objects were labeled as shown
in Figure 2. Afterward, as with other dataset studies, images acquired using
these annotation rules were labeled manually with a bounding box. Finally,
each bounding box was generated in a CSV file. We also formed a meta data file
for each image of the dataset.

###  3.3 Data Quality Assessment

For post-annotation, we investigated our dataset to identify errors and
correct them to form an accurate and reliable dataset. Firstly, we checked if
the annotation criteria as described above were clearly followed. Moreover, we
checked if any objects were incorrectly labeled and any objects that did not
belong to any of our defined five classes were labeled. We checked for such
errors and if found, we correctly re-labeled the images. In addition, if the
parts of the own ships used to collect our data are visible in the dataset,
such ships were excluded from annotation as they have completely different
shapes from the rest of the ships in the dataset. We also checked if the
bounding box is not exactly positioned on the object.

Figure 3: Left: Framework of data quality assessment procedure. Right:
Distribution of the train-validation-test split in the 21 territories.

Data quality assessment is carried out through a two-step procedure to improve
the data quality and reduce human errors during annotation processing. Based
on our data annotation and quality assessment criteria, novice annotators
first checked every anomalies in the dataset. Then, based on the report
provided by the novice annotators, a set of expert annotators verified the
anomalies and re-labeled the objects. As a result, from the original video
data, we acquired around 2,151,470 4K resolution images in JPG format to
generate our dataset. The total 2,151,470 annotated images are divided into
three parts, train (1,729,601 images), validation (211,038 images) and test
(210,831 images) sets. Figure 3 shows the complete pipeline of our data
quality assessment process and the data split in each of the 21 environments
in our final dataset.

##  4 Dataset Description

The KOLOMVERSE has a total of 2,151,470 4K images and contains images
categorized into 5 distinct classes, namely ship (3,791,372 instances), buoy
(67,048 instances), fishnet buoy (273,051 instances), lighthouse (103,358
instances) and wind farm (338,673 instances). It is important to note that
each image can have multiple object instances and more than one class object
often co-occur in the same image, therefore, total number of object instances
are far more than the total number of images in the dataset.

Figure 4: Left: Pie chart of class instances in the KOLOMVERSE. Right: Image
samples containing objects from the five classes (ship, buoy, fishnet buoy,
lighthouse and wind farm).

Class-wise representation of the number of objects in the dataset is shown in
Figure 4. It also shows image samples containing objects from the five
classes. We present a dataset that can be used to develop robust and reliable
deep learning models for real-time use. However, real-time sea environment
contains wide variations in illumination, background and viewpoint. These
large variations in test data compared to that of training data can be
challenging for a deep learning model as the model becomes biased towards the
learned training data. Therefore, we collected data during different times of
the day like sunrise, daytime, sunset and night time from various locations
and camera angles to gather images with illumination, background and viewpoint
variation.

In terms of practical utilization for a deep learning model, it is important
to collect data containing various situations. The KOLOMVERSE has images with
varied background information as the dataset is collected from 21 different
territorial waters of Korea. In addition, even objects seen from far away are
dangerous at the sea, so very small objects should be recognized effectively.
However, most object detection datasets are not suitable for use in real-world
environments because they have only medium to large scale objects. Therefore,
the KOLOMVERSE is built with high resolution images and contains objects seen
from far away. In the KOLOMVERSE, many objects have occlusions, ranging from
sparse occlusion to heavy occlusion. These variations in occclusions make the
KOLOMVERSE a robust dataset for discriminative learning. In addition,
proportion variation in our dataset is also an important factor to note. The
annotations in our dataset do not only contain fully visible parts of the
objects but also meaningful semi or partially visible objects. Inclusion of
such objects create occlusion and proportion varied samples in our dataset.
These variations in illumination, viewpoint, occlusion, background, scale and
proportion make object detectors trained with the KOLOMVERSE more suitable for
real-time applications. Complete proofs of the variations in the KOLOMVERSE is
provided in Appendix A.

##  5 Experiments on the KOLOMVERSE

Implementation details. The YOLO object detection series [16, 17, 18, 19] is a
one-stage object detection model series that has versions ranging from YOLOv1
upto v5 and a few other variations with high accuracy and speed in the
benchmark datasets. Using GPU, YOLO can be trained as a super fast and
accurate object detector. Therefore, in this paper, we show our data
efficiency in the newer versions of YOLO namely YOLOv3 and v4. We trained our
dataset KOLOMVERSE with the YOLOv3 and YOLOv4 models using this repository
111https://github.com/AlexeyAB/darknet. For fast implementation of object
detectors, Huang et al. [33] released the TensorFlow Object Detection API that
supports TenforFlow2(TF2). The API contains codes for training models such as,
SSD [20] with MobileNet [34], RetinaNet [35], FRCNN [22], Mask R-CNN [24] as
well as SSD with EfficientDet [36] and CenterNet [25]. We trained SSD,
CenterNet and FRCNN on the KOLOMVERSE using this repository
222https://github.com/tensorflow/models/tree/master/research/object$\\_$detection.
SSD is a single shot multibox detector which completely eliminates proposal
generation and computates everything in a single network [20] and EfficientDet
is a new family of object detectors, which consistently achieve much better
efficiency than prior architectures [36]. We performed experiments on SSD with
EfficientDet architecture as backbone. CenterNet is a simple and effective
anchor-free architecture that considers an object as the center point of its
bounding box. It is differentiable, faster, and better than other similar
detectors [25]. We performed experiments on CenterNet with Resnet101
architecture [37] as backbone. The FRCNN [22] is a two-stage architecture and
is a state-of-the-art object detection tool based on a region proposal along
with a fast R-CNN [21] network. Two-stage architectures slide a window through
a feature map to localize objects, classify them and regress bounding boxes.
In this study, we performed experiments on our dataset on the FRCNN network
with Resnet50 as backbone [37] .

Experimental settings. To train YOLO models with our dataset, we first
converted our dataset into the YOLO format. We did not change much of the pre-
trained architectures except that we modified the last fully connected layers
from MS COCO’s 80 classes [1] to 5 classes for our 5-class dataset as the
pretrained models were trained on the MS COCO dataset. Also, we converted the
configuration file of YOLO to be trained on a five classes dataset. In the
configuartion file of YOLO models, we set the maximum batches to 10000 (number
of classes*2000) and therefore, don’t use any other early stopping methods. To
train the YOLO architectures, we resized the input size to 416$\times$416\. We
fine-tuned the YOLO architectures with a stochastic gradient optimizer and
learning rate of 0.0001. Similar to the YOLO preprocessing, we firstly
converted our dataset to train it on the FRCNN model. We resized the input
size to 640$\times$640 and we finetuned the FRCNN with a momentum optimizer
and a cosine decay learning rate of base 0.0004. For the CenterNet model, we
resized the images to 512$\times$512 and finetuned the model with ADAM
optimizer and a cosine decay learning rate of base 1e-6. Moreover, for
training SSD with EfficientDet-D0, images were resized to input size
512$\times$512 and for training with backbone EfficientDet-D1, images were
resized to 640$\times$640\. The SSD models were finetuned with a momentum
optimizer and a cosine decay learning rate of base 8e-4 for SSD with backbone
EfficientDet-D0 and 8e-5 for SSD with backbone EfficientDet-D1. We apply early
stopping for the models trained with TF2 to avoid overfitting. After training,
we performed evaluations on both our validation and our test set. We performed
our experiments using Tensorflow (YOLOv3,YOLOv4) and TF2 (SSD, CenterNet and
FRCNN) and we deployed an NVIDIA RTX 6000 GPU with 40GB memory for the
experiments.

Data sampling for training and evaluation. As KOLOMVERSE is huge, training the
complete dataset is time-consuming and wastes a lot of resources such as
electricity and memory. Therefore, we sampled our dataset to efficiently
perform the experiments in this paper. As ship objects account for about
80$\%$ of our dataset, it is difficult to randomly sample the dataset as the
generated set may contain only ship objects and completely miss samples from
other classes. We first considered only the ship objects that co-occur with
other objects in each image rather than the image that only contain ship
objects. In addition, we sampled data from all the 21 environments and
considered the balance among these environments. We performed our experiments
on the sampled dataset with 149,175 number of training images, 18,643 number
of validation images and 18,601 number of test images. Moreover, the object
instances per class for the experiments are ship (393,936 instances), buoy
(34,080 instances), fishnet buoy (95,815 instances), lighthouse (60,362
instances) and wind farm (147,846 instances). The sampled dataset is also
available to download following the Github link in Section 1.

Detection results. Table 2 shows the evaluation results of the state-of-the-
art detectors when trained and evaluated on our dataset. The results showed
mean average precision (mAP) at Intersection over union (IOU) at 0.50 of both
the validation (val) and the test set. IOU defines the overlap over union
degree of two bounding boxes. From the Table 2, we observed that the CenterNet
with Resnet101 performed better than any other detectors. Moreover, in Table 3
we provide a class-wise performance on both the validation and the test set
using the various object detectors. We observe that the ship class has the
best performance among all the five classes as even after sampling it has
relatively more instances. We also observe that FRCNN performs best for ship
class with mAP of 82.83$\%$ in the validation set and mAP of 82.59$\%$ in the
test set. As the samples of fishnet buoy are very small, it is difficult to
accurately detect these objects. Therefore, fishnet buoy class has the least
mAP among all the 5 classes. CenterNet performs best for fishnet buoy class
with mAP of 47.44$\%$ on the validation set and mAP of 48.04$\%$ on the test
set.

Table 2: State-of-the-art detector performances mean average precision at IOU 0.5 (mAP50) in the validation as well as test set. Model |  Backbone |  mAP50($\%$) val |  mAP50($\%$) test
---|---|---|---
SSD |  EfficientDet-D0 [36] |  46.61 |  46.64
SSD |  EfficientDet-D1 [36] |  52.11 |  52.09
YOLOv3 |  Darknet53 [18] |  59.87 |  59.57
YOLOv4 |  CSPDarknet53 [19] |  60.51 |  59.99
FRCNN |  Resnet50 [37] |  60.85 |  61.28
CenterNet |  Resnet101 [37] |  61.40 |  61.86

Figure 5: Left: Loss function graph of FRCNN and Right: Loss function graph of CenterNet (right). Table 3: Class-wise experimental results of the different object detectors used in our dataset. Model | Evaluation Set |  |  |  mAP50($\%$) |  |
---|---|---|---|---|---|---
|  | Ship | Buoy |  Fishnet Buoy |  Lighthouse | Wind Farm
SSD(EfficientDet-D0) | val | 69.03 | 49.48 |  25.76 |  47.57 | 41.21
| test | 68.60 | 49.25 |  25.67 |  47.87 | 41.83
SSD(EfficientDet-D1) | val | 74.15 | 56.27 |  28.82 |  51.60 | 49.73
| test | 73.95 | 55.13 |  28.80 |  52.36 | 50.22
YOLOv3(Darknet53) | val | 78.08 | 64.70 |  35.04 |  64.14 | 57.40
| test | 77.35 | 63.59 |  34.61 |  64.71 | 57.55
YOLOv4(CSPDarknet53) | val | 78.74 | 64.93 |  38.08 |  65.48 | 55.33
| test | 77.99 | 63.84 |  36.13 |  66.16 | 55.84
FRCNN(Resnet50) | val | 82.83 | 59.21 |  46.41 |  59.08 | 56.71
| test | 82.59 | 58.16 |  46.76 |  61.04 | 57.85
CenterNet(Resnet101) | val | 77.67 | 65.92 |  47.44 |  64.69 | 51.26
| test | 77.34 | 65.77 |  48.04 |  65.50 | 52.66

We also present the loss graphs of the two best performing models trained on
the KOLOMVERSE. Figure 5 shows the loss graphs of FRCNN (Resnet50) and
CenterNet (Resnet101) models trained on the KOLOMVERSE.

##  6 Discussions and Future Work

Ethics and social impact. Currently, we are carrying out an R$\&$D project
with a goal to build a large-scale publicly available maritime dataset in
South Korea. The dataset is collected directly by us, and therefore, there is
no copyright issue. In addition, the name of the ships and human faces if
visible in the data cannot be distinguished because we have blurred such
objects in the dataset. Moreover, we recruited annotators using crowdsourcing
and established an environment to work on a web-based annotation platform
through training courses for platform use and annotation work. Finally, we
presented the KOLOMVERSE, a large-scale dataset annotated in the maritime
domain. The results of this research will be valuable in the development of
various intelligent marine systems such as autonomous navigation systems,
e-Navigation systems, search and rescue systems, and unmanned maritime
monitoring systems and there is no negative societal impact in our dataset.

Data availability. KOLOMVERSE can be downloaded after filling the google form
provided in the Github page (see Section 1). For ethical use of the
KOLOMVERSE, we go through the google form answers and make an informed
decision before sharing the dataset. Moreover, we aim to update the KOLOMVERSE
by gathering suggestions from users and release the updated versions.

Limitations and future work. There still is some room for improvement in this
research. The KOLOMVERSE was built with images of objects detected through
voyages of ships and contains most of the objects recognized at sea. However,
since data were collected through ship navigation, there are more ship data
than other categories. Therefore, it is necessary to resolve the data
imbalance by collecting data for other categories in the future. Moreover,
there are mostly fishing boats in the KOLOMVERSE. We aim to resolve this in
the future by adding different types of ship objects.

##  7 Conclusion

We presented a publicly available large-scale object detection dataset in the
maritime domain. Through this dataset, we aim to overcome the lack of large-
scale datasets for object detection in the maritime domain. Our dataset
contains 2,151,470 4K images with five maritime object classes for detection.
Moreover, compared to other datasets, we have more diverse and difficult
categories crucial to maritime object detection. We trained and evaluated our
data on pre-trained state-of-the-art object detectors and achieved high
accuracies. Our experiments demonstrated that our dataset performs better in
two-stage detectors. Moreover, we show that our dataset can be used as a
benchmark in the maritime domain for object detection. We aim to further
expand our research to broader areas such as object detection, classification
and image segmentation. Finally, we hope that this work will help researchers
build more effective detectors in the maritime domain.

## Acknowledgments and Disclosure of Funding

This research was supported by the project titled “Development of Open
Platform Technologies for Smart Maritime Safety and Industries” funded by the
Korea Research Institute of Ships and Ocean Engineering (PES4450).

## References

  * [1] Lin, T.-Y., M. Maire, S. Belongie, et al.  Microsoft coco: Common objects in context.  In _European conference on computer vision_ , pages 740–755. Springer, 2014.
  * [2] Everingham, M., L. Van Gool, C. K. Williams, et al.  The pascal visual object classes (voc) challenge.  _International journal of computer vision_ , 88(2):303–338, 2010\.
  * [3] Deng, J., W. Dong, R. Socher, et al.  Imagenet: A large-scale hierarchical image database.  In _2009 IEEE conference on computer vision and pattern recognition_ , pages 248–255. IEEE, 2009.
  * [4] Krasin, I., T. Duerig, N. Alldrin, et al.  Openimages: A public dataset for large-scale multi-label and multi-class image classification.  _Dataset available from https://github. com/openimages_ , 2(3):18, 2017.
  * [5] Hjelmås, E., B. K. Low.  Face detection: A survey.  _Computer vision and image understanding_ , 83(3):236–274, 2001.
  * [6] Dollár, P., C. Wojek, B. Schiele, et al.  Pedestrian detection: A benchmark.  In _2009 IEEE Conference on Computer Vision and Pattern Recognition_ , pages 304–311. IEEE, 2009.
  * [7] Friedland, B.  Optimum steady-state position and velocity estimation using noisy sampled position data.  _IEEE transactions on Aerospace and Electronic Systems_ , AES-9(6):906–911, 1973.
  * [8] Wen, L.-H., K.-H. Jo.  Fast and accurate 3d object detection for lidar-camera-based autonomous vehicles using one shared voxel-based backbone.  _IEEE Access_ , 9:22080–22089, 2021.
  * [9] Wang, W., Y. Li, X. Luo, et al.  Ocean image data augmentation in the usv virtual training scene.  _Big Earth Data_ , 4(4):451–463, 2020.
  * [10] Elkins, L., D. Sellers, W. R. Monach.  The autonomous maritime navigation (amn) project: Field tests, autonomous and cooperative behaviors, data fusion, sensors, and vehicles.  _Journal of Field Robotics_ , 27(6):790–818, 2010.
  * [11] Liu, R. W., J. Nie, S. Garg, et al.  Data-driven trajectory quality improvement for promoting intelligent vessel traffic services in 6g-enabled maritime iot systems.  _IEEE Internet of Things Journal_ , 8(7):5374–5385, 2020.
  * [12] Shao, Z., W. Wu, Z. Wang, et al.  Seaships: A large-scale precisely annotated dataset for ship detection.  _IEEE transactions on multimedia_ , 20(10):2593–2604, 2018.
  * [13] Ray, C., R. Dréo, E. Camossi, et al.  Heterogeneous integrated dataset for maritime intelligence, surveillance, and reconnaissance.  _Data in brief_ , 25:104141, 2019.
  * [14] Ramos, M. A., I. B. Utne, A. Mosleh.  Collision avoidance on maritime autonomous surface ships: Operators’ tasks and human failure events.  _Safety science_ , 116:33–44, 2019.
  * [15] Piccioli, G., E. De Micheli, P. Parodi, et al.  Robust method for road sign detection and recognition.  _Image and Vision Computing_ , 14(3):209–223, 1996.
  * [16] Redmon, J., S. Divvala, R. Girshick, et al.  You only look once: Unified, real-time object detection.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 779–788. 2016.
  * [17] Redmon, J., A. Farhadi.  Yolo9000: better, faster, stronger.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 7263–7271. 2017.
  * [18] —.  Yolov3: An incremental improvement.  _arXiv preprint arXiv:1804.02767_ , 2018.
  * [19] Bochkovskiy, A., C.-Y. Wang, H.-Y. M. Liao.  Yolov4: Optimal speed and accuracy of object detection.  _arXiv preprint arXiv:2004.10934_ , 2020.
  * [20] Liu, W., D. Anguelov, D. Erhan, et al.  Ssd: Single shot multibox detector.  In _European conference on computer vision_ , pages 21–37. Springer, 2016.
  * [21] Girshick, R.  Fast r-cnn.  In _Proceedings of the IEEE international conference on computer vision_ , pages 1440–1448. 2015.
  * [22] Ren, S., K. He, R. Girshick, et al.  Faster r-cnn: Towards real-time object detection with region proposal networks.  _Advances in neural information processing systems_ , 28:91–99, 2015\.
  * [23] Dai, J., Y. Li, K. He, et al.  R-fcn: Object detection via region-based fully convolutional networks.  _Advances in neural information processing systems_ , 29, 2016.
  * [24] He, K., G. Gkioxari, P. Dollár, et al.  Mask r-cnn.  In _Proceedings of the IEEE international conference on computer vision_ , pages 2961–2969. 2017.
  * [25] Zhou, X., D. Wang, P. Krähenbühl.  Objects as points.  _arXiv preprint arXiv:1904.07850_ , 2019.
  * [26] Prasad, D. K., D. Rajan, L. Rachmawati, et al.  Video processing from electro-optical sensors for object detection and tracking in a maritime environment: A survey.  _IEEE Transactions on Intelligent Transportation Systems_ , 18(8):1993–2016, 2017.
  * [27] Zhang, M. M., J. Choi, K. Daniilidis, et al.  Vais: A dataset for recognizing maritime imagery in the visible and infrared spectrums.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , pages 10–16. 2015.
  * [28] Soloviev, V., F. Farahnakian, L. Zelioli, et al.  Comparing cnn-based object detectors on two novel maritime datasets.  In _2020 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)_, pages 1–6. IEEE, 2020.
  * [29] Patino, L., T. Cane, A. Vallee, et al.  Pets 2016: Dataset and challenge.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops_ , pages 1–8. 2016.
  * [30] Bloisi, D. D., L. Iocchi, A. Pennisi, et al.  Argos-venice boat classification.  In _2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)_ , pages 1–6. IEEE, 2015.
  * [31] Kristan, M., V. S. Kenk, S. Kovačič, et al.  Fast image-based obstacle detection from unmanned surface vehicles.  _IEEE transactions on cybernetics_ , 46(3):641–654, 2015.
  * [32] Shorten, C., T. M. Khoshgoftaar.  A survey on image data augmentation for deep learning.  _Journal of big data_ , 6(1):1–48, 2019.
  * [33] Huang, J., V. Rathod, C. Sun, et al.  Speed/accuracy trade-offs for modern convolutional object detectors.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_. 2017.
  * [34] Howard, A. G., M. Zhu, B. Chen, et al.  Mobilenets: Efficient convolutional neural networks for mobile vision applications.  _arXiv preprint arXiv:1704.04861_ , 2017.
  * [35] Lin, T.-Y., P. Goyal, R. Girshick, et al.  Focal loss for dense object detection.  In _Proceedings of the IEEE International Conference on Computer Vision (ICCV)_. 2017.
  * [36] Tan, M., R. Pang, Q. V. Le.  Efficientdet: Scalable and efficient object detection.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_. 2020.
  * [37] He, K., X. Zhang, S. Ren, et al.  Deep residual learning for image recognition.  In _Proceedings of the IEEE conference on computer vision and pattern recognition_ , pages 770–778. 2016.

##  Appendix A Appendix

KOLOMVERSE has variations in illumination, viewpoint, occlusion, background,
scale and proportion that makes object detectors trained with the KOLOMVERSE
more suitable for real-time applications. Figure 6 shows image samples with
illumination variation. Figure 7 shows image samples with background
variation. Figure 8 shows image samples with scale variation in ship objects.
Figure 9 shows image samples with scale variation in buoy objects. Figure 10
shows image samples with viewpoint variation. Figure 11 shows image samples
with occlusion variation. Figure 12 shows image samples with proportion
variation.

Figure 6: Image samples showing variation in illumination Figure 7: Ship image
samples showing variations in background Figure 8: Image samples showing
variations in scale within same ship objects Figure 9: Image samples showing
variations in scale within same buoy objects Figure 10: Image samples showing
variations in viewpoint in the same ship object. The viewpoint transitions
from right to left. Figure 11: Image samples showing varying levels of
occlusion among the same ship objects. Figure 12: Image samples showing
variations in proportions in the same ship objects.