# Domains of discontinuity of Lorentzian affine group actions

Michael Kapovich and Bernhard Leeb

###### Abstract

We prove nonemptyness of domains of proper discontinuity of Anosov groups of
affine Lorentzian transformations of ${\mathbb{R}}^{n}$.

There is a substantial body of literature, going back to the pioneering work
of Margulis [Ma], on properly discontinuous non-amenable groups of affine
transformations, see e.g. [A, AMS02, AMS11, Dr, DGK, GLM, Me], and numerous
other papers. In this paper we address a somewhat related question of
nonemptyness of domains of proper discontinuity of discrete groups acting on
affine spaces:

###### Question 1.

Which discrete subgroups $\Gamma<Aff({\mathbb{R}}^{n})$ have nonempty
discontinuity domain in the affine space ${\mathbb{R}}^{n}$?

In this paper we limit ourselves to the following setting: Suppose that
$\Gamma<{\mathbb{R}}^{n}\rtimes O(n-1,1)<Aff({\mathbb{R}}^{n})$ is a discrete
subgroup such that the linear projection $\ell:\Gamma\to O(n-1,1)$ is a
faithful representation with convex-cocompact image, see e.g. [Bo] for the
precise definition. Given a representation $\ell:\Gamma\to O(n-1,1)$, the
affine action of $\Gamma$ is determined by a cocycle $c\in
Z^{1}(\Gamma,{\mathbb{R}}^{n-1,1}_{\ell})$. Even in the case $n=3$ and
$\ell(\Gamma)$ a Schottky subgroup of $O(2,1)$ (which is the setting of
Margulis’ original examples), while some actions are properly discontinuous on
the entire ${\mathbb{R}}^{3}$ (as proven by Margulis, see also [GLM] for a
general description of such actions), nonemptyness of domains of discontinuity
for arbitrary $c$ does not appear to be obvious111The reaction to the question
that we observed included: “clearly true”, “clearly false”, “unclear”..

The main result of this note is:

###### Theorem 2.

Every subgroup $\Gamma<{\mathbb{R}}^{n}\rtimes O(n-1,1)$ with faithful convex-
cocompact linear representation $\ell:\Gamma\to O(n-1,1)$, acts properly
discontinuously on a nonempty open subset of the Lorentzian space
${\mathbb{R}}^{n-1,1}$.

We will prove this theorem by applying results on domains of discontinuity for
discrete group actions on flag manifolds proven in [KLP3]. To this end, we
will begin by identifying the Lorentzian space ${\mathbb{R}}^{n-1,1}$ with an
open Schubert cell in a partial flag manifold of the group $G=O(n,2)$.

Consider the group $G=O(n,2)$ and its symmetric space $X=G/K$, $K=O(n)\times
O(2)$. The group $G$ has two partial flag manifolds: the Grassmannian
$\mathrm{F}_{1}$ of isotropic lines and another partial flag manifold
$\mathrm{F}_{2}$ of isotropic planes in $V={\mathbb{R}}^{n,2}$, where the
quadratic form on $V$ is

$q=x_{1}y_{1}+x_{2}y_{2}+z_{1}^{2}+....+z_{n}^{2}.$

We will use the notation $\langle\cdot,\cdot\rangle$ for the associated
bilinear form on $V$.

In the paper we will be using the Tits boundary $\partial_{Tits}X$ of the
symmetric space $X$ and the incidence geometry interpretation of
$\partial_{Tits}X$. The Tits boundary $\partial_{Tits}X$ is a metric bipartite
graph whose vertices are labelled lines and planes, these are the elements of
$\mathrm{F}_{1}$ and $\mathrm{F}_{2}$ respectively. Two vertices
$L\in\mathrm{F}_{1}$ and $p\in\mathrm{F}_{2}$ are connected by an edge iff the
line $L$ is contained in the plane $p$. The edges of this bipartite graph have
length $\pi/4$. We refer the reader to [Br], [G] and [T].

The group $G$ acts transitively on the set of edges of $\partial_{Tits}X$ and
we can identify the quotient $\partial_{Tits}X/G$ with $\sigma_{mod}$, the
model spherical chamber of $\partial_{Tits}X$. Thus $\sigma_{mod}$ is a
circular segment of the length $\pi/4$. This segment has two vertices, one of
which we denote $\tau_{mod}$, this is the one which is the projection of
$\mathrm{F}_{1}$. The flag manifold $\mathrm{F}_{1}$ is the quotient
$G/P_{L}$, where $P_{L}$ is the stabilizer of an isotropic line $L$ in $G$;
this flag manifold is $n$-dimensional.

Recall that two vertices of $\partial_{Tits}X$ are opposite iff they are
within Tits distance $\pi$ from each other. In terms of the incidence geometry
of the vector space $(V,q)$, two lines $L,\hat{L}\in\mathrm{F}_{1}$ are
opposite iff that they span the plane $\operatorname{span}(L,\hat{L})$ in $V$
such that the restriction of $q$ to $\operatorname{span}(L,\hat{L})$ is
nondegenerate, necessarily of the type $(1,1)$. Two lines
$L,L^{\prime}\in\mathrm{F}_{1}$ are within Tits distance $\pi/2$ iff they span
an isotropic plane in $V$.

Consider a subgroup $P_{L}<G$; it is a maximal parabolic subgroup of $G$; let
$U<P_{L}$ be the unipotent radical of $P_{L}$. Choosing a line $\hat{L}$
opposite to $L$, defines a semidirect product decomposition $P_{L}=U\rtimes
G_{L,\hat{L}}$, where $G_{L,\hat{L}}$ is the stabilizer in $P_{L}$ of the line
$\hat{L}$; equivalently, it is the stabilizer of the parallel set222The
parallel set $P(L,\hat{L})$ is a certain symmetric subspace in $X$, which is
the union of all geodesics $l$ in $X$ which are forward-asymptotic to
$L\in\partial_{Tits}X$ and backward-asymptotic to
$\hat{L}\in\partial_{Tits}X$. The parallel set splits isometrically as the
product $l\times{\mathbb{H}}^{n-1}$, where ${\mathbb{H}}^{n-1}$ is the cross-
section of $P(L,\hat{L})$. $P(L,\hat{L})$. This subgroup is the intersection

$G_{L,\hat{L}}=P_{L}\cap P_{\hat{L}}.$

The orthogonal complement $V_{L,\hat{L}}\subset V$ of the anisotropic plane
$\operatorname{span}(L,\hat{L})$ is invariant under $G_{L,\hat{L}}$, hence,

$G_{L,\hat{L}}\cong{\mathbb{R}}_{+}\times
O(V_{L,\hat{L}},q|_{V_{L,\hat{L}}})\cong{\mathbb{R}}_{+}\times O(n-1,1).$

Here the group ${\mathbb{R}}_{+}$ acts via transvections along geodesics in
the symmetric space $X$ connecting $L$ and $\hat{L}$. The group
$G_{L,\hat{L}}$ acts on both
$(V^{\prime},q^{\prime})=(V_{L,\hat{L}},q|_{V_{L,\hat{L}}})$ and on $U$, where
the action of ${\mathbb{R}}_{+}$ on $V^{\prime}=V_{L,\hat{L}}$ is trivial. In
order to simplify the notation, we set

$O(q^{\prime})=O(V^{\prime},q^{\prime}).$

In terms of linear algebra, ${\mathbb{R}}_{+}$ is the identity component of
the orthogonal group

$O(\operatorname{span}(L,\hat{L}),q|_{\operatorname{span}(L,\hat{L})})\cong
O(1,1).$

We will use the notation

$G^{\prime}_{L}:=U\rtimes O(q^{\prime})<P_{L}.$

This subgroup is the stabilizer in $P_{L}$ of horoballs in $X$ centered at
$L$.

Our next goal is to describe Schubert cells in the Grassmannian
$\mathrm{F}_{1}$. We fix $L\in\mathrm{F}_{1}$ and define the subvariety
$Q_{L}\subset\mathrm{F}_{1}$ consisting of all (isotropic) lines
$L^{\prime}\subset V$ such that $\operatorname{span}(L,L^{\prime})$ is
isotropic (the line $L$ or an isotropic plane). In terms of the Tits’
distance, $Q_{L}-\\{L\\}$ consists of lines $L^{\prime}\in\mathrm{F}_{1}$
within distance $\frac{\pi}{2}$ from $L$. The complement

$L^{opp}=\mathrm{F}_{1}-Q_{L}$

consists of lines opposite to $L$. The group $P_{L}$ acts transitively on
$\\{L\\}$, $Q_{L}-\\{L\\}$ and $L^{opp}$ and each of these subsets is an open
Schubert cell of $\mathrm{F}_{1}$ with respect to $P_{L}$ and we obtain the
$P_{L}$-invariant Schubert cell decomposition

$\mathrm{F}_{1}=\\{L\\}\sqcup(Q_{L}-\\{L\\})\sqcup L^{opp}.$

We next describe $Q_{L}$ more geometrically. A vector $v\in V$ spans an
isotropic subspace with $L$ iff $v\in L^{\perp}$ and satisfies the quadratic
equation $q(v)=0$. Since we are only interested in nonzero vectors $v\neq 0$
and their spans $\operatorname{span}(v)$, we obtain the natural identification

$Q_{L}\cong{\mathbb{P}}(q^{-1}(0)\cap L^{\perp}),$

the right hand-side is the projectivization a conic in $L^{\perp}$. Thus,
$Q_{L}$ is a (projective) conic and $L\in Q_{L}$ is the unique singular point
of the $Q_{L}$.

###### Lemma 3.

Given two opposite isotropic lines $L,\hat{L}$, the intersection of the conics

$E=E_{L,\hat{L}}:=Q_{L}\cap Q_{\hat{L}}$

is an ellipsoid in $Q_{L}$.

###### Proof.

As before, let $V^{\prime}\subset V$ denote the codimension two subspace
orthogonal to both $L,\hat{L}$. Then each $L^{\prime}\in E$ is spanned by a
vector $v\in V^{\prime}$ satisfying the condition $q(v)=0$. In other words,
$E$ is the projectivization of the conic

$\\{v\in V^{\prime}:q(v)=0\\},$

i.e. is an ellipsoid. ∎

Our next goal is to (equivariantly) identify the open cell $L^{opp}$ with the
$n$-dimensional Lorentzian affine space ${\mathbb{R}}^{n-1,1}$ (where a chosen
$\hat{L}\in L^{opp}$ will serve as the origin), so that the group $P_{L}$ is
identified with the group of Lorentzian similarities, where the simply-
transitive action $U\curvearrowright L^{opp}$ is identified with the action of
the full group of translations of ${\mathbb{R}}^{n-1,1}$.

We fix nonzero vectors $e\in L$, $f\in\hat{L}$ such that $\langle
e,f\rangle=1$. Then

$V=\operatorname{span}(e)\oplus\operatorname{span}(f)\oplus V^{\prime}.$

We obtain an epimorphism $\eta:P_{L}\to O(q^{\prime})$ by sending $g\in P_{L}$
first to the restriction $g|L^{\perp}$ and then to the projection of the
latter to the quotient space $V^{\prime}\cong L^{\perp}/L$ (the quotient of
$L^{\perp}$ by the null-subspace of $q|L^{\perp}$). Hence, the kernel of this
epimorphism is precisely the solvable radical $U\rtimes{\mathbb{R}}_{+}$ of
$P_{L}$.

For each $v^{\prime}\in V^{\prime}$ we define the linear transformation (a
shear) $s=s_{v^{\prime}}\in GL(V)$ by its action on $e,f$ and $V^{\prime}$:

  1. 1.

$s(e)=e$.

  2. 2.

$s(f)=-\frac{1}{2}q(v^{\prime})e+f+v^{\prime}$.

  3. 3.

For $w\in V^{\prime}$, $s(w)=w-\langle v^{\prime},w\rangle e$.

The next two lemmata are proven by straightforward calculations which we omit:

###### Lemma 4.

For each $s=s_{v^{\prime}}$ the following hold:

1\. $s\in P_{L}$.

2\. $s$ lies in the kernel of the homomorphism $\eta:P_{L}\to GL(V^{\prime})$
and is unipotent. In particular, $s\in U$ for each $v^{\prime}\in V$.

###### Lemma 5.

The map $\phi:v^{\prime}\mapsto s_{v^{\prime}}$ is a continuous monomorphism
$V^{\prime}\to U$, where we equip the vector space $V^{\prime}$ with the
additive group structure.

Since $U$ acts simply transitively on $L^{opp}$, it is connected and has
dimension $n$. Therefore, the monomorphism $\phi$ is surjective and, hence, a
continuous isomorphism. Thus, $\phi$ determines a homeomorphism
$h:V^{\prime}\to L^{opp}$

$h:v^{\prime}\mapsto
s_{v^{\prime}}(\hat{L})=span\left(-\frac{1}{2}q(v^{\prime})e+f+v^{\prime}\right),$
$h(0)=\hat{L}.$

The group $G_{L,\hat{L}}\cong{\mathbb{R}}_{+}\times O(V^{\prime},q^{\prime})$
acts on both $L^{opp}$ and on $U$ (via conjugation). The center of
$G_{L,\hat{L}}$ acts on $V^{\prime}$ trivially while its action on $U$ is via
a nontrivial character.

###### Proposition 6.

The map $h$ is equivariant with respect to these two actions of
$O(V^{\prime},q^{\prime})$.

###### Proof.

Consider a linear transformation $A\in O(V^{\prime},q^{\prime})$; as before,
we identify $O(V^{\prime},q^{\prime})$ with a subgroup of $O(V,q)$ fixing $e$
and $f$. For an arbitrary $v^{\prime}\in V^{\prime}$ we will verify that

$s_{Av^{\prime}}=As_{v^{\prime}}A^{-1}.$

It suffices to verify this identity on the vectors $e,f$ and arbitrary $w\in
V^{\prime}$. We have:

1\. For each $u\in V^{\prime}$, $s_{u}(e)=e$, while $A(e)=A^{-1}(e)=e$. It
follows that

$e=s_{Av^{\prime}}(e)=As_{v^{\prime}}A^{-1}(e)=e.$

2.

$s_{Av^{\prime}}(f)=-\frac{1}{2}q(Av^{\prime})e+f+Av^{\prime}=-\frac{1}{2}q(v^{\prime})e+f+Av^{\prime}$

while (since $Ae=e$, $Af=f$)

$As_{v^{\prime}}A^{-1}(f)=As_{v^{\prime}}(f)=A(-\frac{1}{2}q(v^{\prime})e+f+v^{\prime})=-\frac{1}{2}q(v^{\prime})e+f+Av^{\prime}.$

3\. For $w\in V^{\prime}$,

$s_{Av^{\prime}}(w)=w-\langle Av^{\prime},w\rangle e=w-\langle
v^{\prime},A^{-1}w\rangle e,$

while

$As_{v^{\prime}}A^{-1}w=As_{v^{\prime}}(A^{-1}w)=A(A^{-1}w-\langle
v^{\prime},A^{-1}w\rangle e)=w-\langle v^{\prime},A^{-1}w\rangle e.\qed$

In view of this proposition we will identify $V^{\prime}$ with the open
Schubert cell $L^{opp}$, which, in turn, enables us to use Lorentzian geometry
to analyze $L^{opp}$ and, conversely, to study discrete subgroups of $P_{L}$
using results of [KLP3] on domains of discontinuity of discrete group actions
on the flag manifold $\mathrm{F}_{1}$. Under the identification
$V^{\prime}\cong L^{opp}$, for each $\hat{L}\in L^{opp}$, the conic
$Q_{\hat{L}}\cap L^{opp}$ becomes a translate of the null-cone of the form
$q^{\prime}$ on $V^{\prime}$ (see Lemma 7 below) and the flag manifold
$\mathrm{F}_{1}$ becomes a compactification of $V^{\prime}$ obtained by adding
to it the “quadric at infinity” $Q_{L}$.

###### Lemma 7.

For all $v^{\prime}\in V^{\prime}$, $q^{\prime}(v^{\prime})=0$ iff $q$
vanishes on $\operatorname{span}(f,h(v^{\prime}))$, i.e. iff $h(v^{\prime})\in
Q_{\hat{L}}$. In other words, $Q_{\hat{L}}\cap L^{opp}$ is the image under $h$
of the null-cone of $q^{\prime}$ in the vector space $V^{\prime}$.

###### Proof.

Since $f$ and $s_{v^{\prime}}(f)$ (spanning the line $h(v^{\prime})$) are
null-vectors of $q$, the vanishing of $q$ on
$\operatorname{span}(f,h(v^{\prime}))$ is equivalent to the vanishing of

$\langle f,s_{v^{\prime}}(f)\rangle=-\frac{1}{2}q(v^{\prime}).\qed$

###### Lemma 8.

For each neighborhood $N$ of $L$ in $Q_{L}$ there exists $\hat{L}\in L^{opp}$
such that $E_{L,\hat{L}}\subset N$.

###### Proof.

We pick $L_{\infty}\in\mathrm{F}_{1}$ opposite to $L$ and, as above, identify
$L_{\infty}^{opp}$ with $(V^{\prime},q^{\prime})$. Then for a sequence
$\hat{L}_{i}\in L^{opp}_{\infty}$ contained in the, say, future light cone of
$Q_{L}\cap L_{\infty}^{opp}$ and converging radially to $L$, the intersections
of null-cones $E_{L,L_{i}}=Q_{L_{i}}\cap Q_{L}$ converge to $L$. Since
$L_{i}\notin Q_{L}$, they are all opposite to $L$. ∎

For each subset $C\subset\mathrm{F}_{1}$, we define the thickening of $C$:

$\mathop{\hbox{Th}}\nolimits(C)=\bigcup_{L\in C}Q_{L}.$

This notion of thickening is a special case of the one developed in [KLP3]
(see also [KL2]): If we restrict to a single apartment $a$ in the Tits
building of $G$, then for the vertex $L\in a$,
$\mathop{\hbox{Th}}\nolimits(L)\cap a=Q_{L}\cap a$ consists of three vertices
within Tits distance $\frac{\pi}{2}$ from $L$. Thus, in the terminology of
[KLP3], the thickening $\mathop{\hbox{Th}}\nolimits$ is fat.

###### Lemma 9.

For any two opposite lines $L,\hat{L}\in\mathrm{F}_{1}$ and each compact
subset $C\subset Q_{\hat{L}}\cap L^{opp}$, the intersection
$\mathop{\hbox{Th}}\nolimits(C)\cap L^{opp}$ is a proper subset of $L^{opp}$.

###### Proof.

Let $H\subset L^{opp}\cong V^{\prime}$ be an affine hyperplane in $V^{\prime}$
intersecting $Q_{\hat{L}}$ only at $\hat{L}$. Then

$C^{\prime}:=\\{L^{\prime}\in H:Q_{L^{\prime}}\cap C\neq\emptyset\\}$

is compact in $H$. Next, observe that for $L_{1},L_{2}\in\mathrm{F}_{1}$,
$L_{1}\in Q_{L_{2}}\iff L_{2}\in Q_{L_{1}}$. Thus, every $L^{\prime}\in
H-C^{\prime}$ does not belong to $\mathop{\hbox{Th}}\nolimits(C)$. ∎

###### Lemma 10.

For each compact $C\subset Q_{L}-\\{L\\}$ the thickening
$\mathop{\hbox{Th}}\nolimits(C)$ is a proper subset of $\mathrm{F}_{1}$.

###### Proof.

Lemma 8 implies that there exists $L_{\infty}\in L^{opp}$ such that
$E_{L,L_{\infty}}$ is disjoint from $C$. Thus, $C$ is contained in
$L^{opp}_{\infty}$. Now the claim follows from Lemma 9. ∎

We now turn to discrete subgroups $\Gamma<G^{\prime}_{L}<P_{L}<G$. We refer
the reader to [KLP3] for the notion of $\tau_{mod}$-regular discrete subgroups
$\Gamma<G$ and their $\tau_{mod}$-limit sets, which are certain closed
$\Gamma$-invariant subsets of $\mathrm{F}_{1}$.

###### Remark 11.

We must also note that the notion equivalent to $\tau_{mod}$-regularity and
the $\tau_{mod}$-lit set was first introduced by Benoist in his highly
influential work [Ben].

An important class of $\tau_{mod}$-regular discrete subgroups $\Gamma<G$
consists of $\tau_{mod}$-Anosov subgroups. Anosov representations $\Gamma\to
G$ whose images are Anosov subgroups were first introduced in [La] for
fundamental groups of closed manifolds of negative curvature, then in [GW] for
arbitrary hyperbolic groups; we refer the reader to our papers [KLP4, KLP5,
KL1], for a simplification of the original definition as well as for
alternative definitions and to [KL2, KLP2] for surveys of the results.

###### Lemma 12.

The $\tau_{mod}$-limit set $\Lambda_{\tau_{mod}}(\Gamma)$ of every
$\tau_{mod}$-regular discrete subgroup $\Gamma<P_{L}$ is contained in $Q_{L}$.

###### Proof.

Recall that $G^{\prime}_{L}$ and, hence, $\Gamma$, preserves each horoball
$Hbo$ in $X$ centered at $L$, where the latter is regarded as a point of the
visual boundary of the symmetric space $X$. Therefore, for each $x\in Hbo$,
the closure of $\Gamma x$ in $\overline{X}=X\cup\partial_{\infty}X$ is
contained in the ideal boundary of $Hbo$, which is the closed
$\frac{\pi}{2}$-ball $\bar{B}(L,\frac{\pi}{2})$ in $\partial_{\infty}X$
centered at $L$, where the distance is computed in the Tits metric on
$\partial_{\infty}X$. For each vertex $\tau$ of the building
$\partial_{Tits}X$ which belongs to $\bar{B}(L,\frac{\pi}{2})$ the star
$\operatorname{st}(\tau)\subset\partial_{\infty}X$ is contained in the closed
ball in $\partial_{\infty}X$ of the radius $\frac{3\pi}{4}$ centered at $L$.
Therefore, the intersection of $\operatorname{st}(\tau)$ with the Grassmannian
$\mathrm{F}_{1}$ is contained in $\bar{B}(L,\frac{\pi}{2})$. It follows from
the definition of the $\tau_{mod}$-limit set that
$\Lambda_{\tau_{mod}}(\Gamma)$ is contained in
$\mathrm{F}_{1}\cap\bar{B}(L,\frac{\pi}{2})=Q_{L}$. ∎

###### Proposition 13.

Suppose that $\Gamma<G^{\prime}_{L}$ is a $\tau_{mod}$-regular discrete
subgroup whose $\tau_{mod}$-limit set does not contain $L$. Then

$\mathop{\hbox{Th}}\nolimits(\Lambda_{\tau_{mod}}(\Gamma))\neq\mathrm{F}_{1}$

and the action

$\Gamma\curvearrowright\mathrm{F}_{1}-\mathop{\hbox{Th}}\nolimits(\Lambda_{\tau_{mod}}(\Gamma))$

is properly discontinuous.

###### Proof.

Since $\Lambda_{\tau_{mod}}(\Gamma)$ is a compact subset of $Q_{L}$, the first
statement of the proposition is a special case of Lemma 10. The proper
discontinuity statement is a special case of a general theorem [KLP3, Theorem
6.13] since the thickening $\mathop{\hbox{Th}}\nolimits$ is fat. ∎

We now describe certain conditions on $\tau_{mod}$-regular discrete subgroups
$\Gamma<G^{\prime}_{L}$ which will ensure that $\Lambda_{\tau_{mod}}(\Gamma)$
does not contain the point $L$. Each subgroup $\Gamma<G^{\prime}_{L}$ has the
linear part $\Gamma_{0}$, i.e. its projection to $O(q^{\prime})\cong
O(n-1,1)$, which is identified with the semisimple factor of the stabilizer in
$P_{L}$ of some $\hat{L}\in L^{opp}$. We now assume that:

  * •

$\Gamma_{0}$ is a convex-cocompact subgroup of $O(n-1,1)$.

  * •

The projection

$\ell:\Gamma\to\Gamma_{0}$

is an isomorphism.

Since $\Gamma_{0}<O(q^{\prime})$ is convex-cocompact and $O(q^{\prime})<P_{L}$
is the Levi subgroup of the parabolic group $P_{L}$ stabilizing a face of type
$\tau_{mod}$ of $\partial_{Tits}X$, it follows that $\Gamma_{0}<G$ is a
${\tau_{mod}}$-Anosov subgroup of $G$; the $\tau_{mod}$-limit set of
$\Gamma_{0}$ is contained in the visual boundary of the cross-section
(isometric to ${\mathbb{H}}^{n-1}$) of the parallel set $P(L,\hat{L})$; in
particular, $\Lambda_{\tau_{mod}}(\Gamma_{0})$ does not contain $L$.

Given a subgroup $\Gamma_{0}<O(q^{\prime})$, the inverse
$\rho:\Gamma_{0}\to\Gamma$ to $\ell:\Gamma\to\Gamma_{0}$ is determined by a
cocycle $c\in Z^{1}(\Gamma_{0},V^{\prime})$ which describes the translational
parts of the elements of $\Gamma$:

$\rho(\gamma):v\mapsto\gamma v+c(\gamma),v\in
V^{\prime}\cong{\mathbb{R}}^{n-1,1}.$

Pick some $t\in{\mathbb{R}}_{+}$; then $tc$ is again a cocycle corresponding
to the conjugate representation $\rho^{t}$, where we identity
$t\in{\mathbb{R}}_{+}$ with a central element of $G_{L,\hat{L}}$. Sending
$t\to 0$ we obtain:

$\lim_{t\to 0}\rho^{t}=id,$

the identity embedding $\Gamma_{0}\to O(n-1,1)<P_{L}$. In view of stability of
Anosov representations (see [GW] and [KLP1]) we conclude that all
representations $\rho^{t}$ are ${\tau_{mod}}$-Anosov and the
$\tau_{mod}$-limit sets of $\Gamma_{t}=\rho^{t}(\Gamma_{0})$ vary continuously
with $t$; moreover,

$t\Lambda_{\tau_{mod}}(\Gamma_{t_{1}})=\Lambda_{\tau_{mod}}(\Gamma_{t_{2}})$

where $t=t_{2}/t_{1}$. In particular,

$\Lambda_{\tau_{mod}}(\Gamma)\subset Q_{L}-\\{L\\}$

is a compact subset. Proposition 13 now implies:

###### Corollary 14.

For each $\Gamma$ as above,

$Th(\Lambda_{\tau_{mod}}(\Gamma))\neq\mathrm{F}_{1}$

and the action

$\Gamma\curvearrowright\mathrm{F}_{1}-Th(\Lambda_{\tau_{mod}}(\Gamma))$

is properly discontinuous.

Thus, we proved that each discrete subgroup $\Gamma<P_{L}$ as above has
nonempty domain of discontinuity in the vector space $V^{\prime}$. Theorem 2
follows. ∎

Acknowledgements. The first author was partly supported by the NSF grant
DMS-16-04241, by a Simons Foundation Fellowship, grant number 391602, by Max
Plank Institute for Mathematics in Bonn, as well as by KIAS (the Korea
Institute for Advanced Study) through the KIAS scholar program. Much of this
work was done during our stay at KIAS and we are thankful to KIAS for its
hospitality.

## References

  * [A] H. Abels, Properly discontinuous groups of affine transformations: a survey, Geom. Dedicata 87 (2001), no. 1-3, pp. 309–333.
  * [AMS02] H. Abels, G. Margulis, G. Soifer, On the Zariski closure of the linear part of a properly discontinuous group of affine transformations, J. Differential Geom. 60 (2002), no. 2, pp. 315–344.
  * [AMS11] H. Abels, G. Margulis, G. Soifer, The linear part of an affine group acting properly discontinuously and leaving a quadratic form invariant, Geom. Dedicata 153 (2011), pp. 1–46.
  * [Ben] Y. Benoist, Propriétés asymptotiques des groupes linéaires, Geom. Funct. Anal. Vol. 7 (1997), no. 1, pp. 1–47.
  * [Bo] B. Bowditch, Geometrical finiteness for hyperbolic groups, J. Funct. Anal. Vol. 113 (1993), no. 2, pp. 245–317.
  * [Br] K. Brown, “Buildings”, Springer-Verlag, 1989.
  * [DGK] J. Danciger, F. Guéritaud, F. Kassel, Margulis spacetimes via the arc complex, Invent. Math. 204 (2016), no. 1, pp. 133–193.
  * [Dr] T. Drumm, Fundamental polyhedra for Margulis space-times, Topology 31 (1992), no. 4, pp. 677–683.
  * [G] P. Garrett, “Buildings and Classical Groups”, CRC Press, 1997.
  * [GLM] W. Goldman, F. Labourie, G. Margulis, Proper affine actions and geodesic flows of hyperbolic surfaces, Ann. of Math. (2) 170 (2009), no. 3, pp. 1051–1083.
  * [GW] O. Guichard, A. Wienhard, Anosov representations: Domains of discontinuity and applications, Invent. Math. Vol. 190 (2012) no. 2, pp. 357–438.
  * [KL1] M. Kapovich, B. Leeb, Finsler bordifications of symmetric and certain locally symmetric spaces, Geometry and Topology, 22 (2018) pp. 2533–2646.
  * [KL2] M. Kapovich, B. Leeb, Discrete isometry groups of symmetric spaces, MSRI Lecture Notes. “Handbook of Group Actions, IV”. The ALM series, International Press, Eds. L.Ji, A.Papadopoulos, S-T.Yau. Chapter 5, pp. 191–290.
  * [KLP1] M. Kapovich, B. Leeb, J. Porti, Morse actions of discrete groups on symmetric spaces, arXiv e-print arXiv:1411.4176, March 2014.
  * [KLP2] M. Kapovich, B. Leeb, J. Porti, Some recent results on Anosov representations, Transformation Groups, 21 (2016) no. 4, pp. 1105–1121.
  * [KLP3] M. Kapovich, B. Leeb, J. Porti, Dynamics on flag manifolds: domains of proper discontinuity and cocompactness, Geometry and Topology, 22 (2017) pp. 157–234.
  * [KLP4] M. Kapovich, B. Leeb, J. Porti, Anosov subgroups: dynamical and geometric characterizations, European Journal of Mathematics, 3 (2017) pp. 808–898.
  * [KLP5] M. Kapovich, B. Leeb, J. Porti, A Morse lemma for quasigeodesics in symmetric spaces and euclidean buildings, Geometry and Topology, 22 (2018) 3827–3923.
  * [Ma] G. Margulis, Free completely discontinuous groups of affine transformations, Soviet Math. Dokl. 28 (1983), no. 2, pp. 435–439.
  * [Me] G. Mess, Lorentz spacetimes of constant curvature, Geom. Dedicata 126 (2007), pp. 3–45.
  * [La] F. Labourie, Anosov flows, surface groups and curves in projective space, Invent. Math. 165 (2006) no. 1, pp. 51–114.
  * [T] J. Tits, “Buildings of spherical type and finite BN-pairs.” Lecture Notes in Mathematics, Vol. 386. Springer-Verlag, Berlin-New York, 1974.

M.K.: Department of Mathematics, University of California, Davis, CA 95616,
USA
email<EMAIL_ADDRESS>
and

Korea Institute for Advanced Study,
207-43 Cheongnyangri-dong, Dongdaemun-gu,
Seoul, South Korea

B.L.: Mathematisches Institut,Universität München, Theresienstr. 39, D-80333
München, Germany, email<EMAIL_ADDRESS>


# Event-assisted Low-Light Video Object Segmentation

Hebei Li 1 Jin Wang 1 Jiahui Yuan 1 Yue Li 1 Wenming Weng 1 Yansong Peng 1
Yueyi Zhang 1, Corresponding Author Zhiwei Xiong 1 Xiaoyan Sun 1, 2
1 University of Science and Technology of China  2 Institute of Artificial
Intelligence, Hefei Comprehensive National Science Center
{lihebei, jin01wang, yuanjiahui, yueli65, wmweng,
<EMAIL_ADDRESS>
{zhyuey, zwxiong<EMAIL_ADDRESS>
###### Abstract

In the realm of video object segmentation (VOS), the challenge of operating
under low-light conditions persists, resulting in notably degraded image
quality and compromised accuracy when comparing query and memory frames for
similarity computation. Event cameras, characterized by their high dynamic
range and ability to capture motion information of objects, offer promise in
enhancing object visibility and aiding VOS methods under such low-light
conditions. This paper introduces a pioneering framework tailored for low-
light VOS, leveraging event camera data to elevate segmentation accuracy. Our
approach hinges on two pivotal components: the Adaptive Cross-Modal Fusion
(ACMF) module, aimed at extracting pertinent features while fusing image and
event modalities to mitigate noise interference, and the Event-Guided Memory
Matching (EGMM) module, designed to rectify the issue of inaccurate matching
prevalent in low-light settings. Additionally, we present the creation of a
synthetic LLE-DAVIS dataset and the curation of a real-world LLE-VOS dataset,
encompassing frames and events. Experimental evaluations corroborate the
efficacy of our method across both datasets, affirming its effectiveness in
low-light scenarios. The datasets are available at
https://github.com/HebeiFast/EventLowLightVOS.

##  1 Introduction

Video Object Segmentation (VOS) refers to a computer vision domain focusing on
algorithms for segmenting objects within a sequence of video frames. Its
applications span across various fields such as autonomous driving,
surveillance, and interactive video editing [37, 29]. VOS is commonly
classified into two categories, contingent on the availability of annotation
for the initial frame: semi-supervised and unsupervised VOS. Our research
primarily delves into the realm of semi-supervised VOS.

Under low-light conditions, insufficient illumination gives rise to a
diminished level of detail and compromised color accuracy. Conventional VOS
methods [6, 34, 5, 12], predicated upon the availability of high-quality
visual inputs, tend to exhibit a marked decline in performance when confronted
with such challenging lighting circumstances.

Unlike traditional imaging methods, event cameras signify a fundamental change
and offer significant advantages in demanding lighting scenarios [39, 11, 13].
These cameras excel in high dynamic range and provide detailed edge and
movement data for objects [2]. These unique features are crucial in segmenting
video objects under low-light conditions. Despite their impressive
capabilities, event cameras lack the ability of capture texture and color
information, which limits their effectiveness in segmentation tasks. Hence,
our paper aims to explore the integration of event-based and frame-based
modalities to improve VOS, particularly in challenging low-light environments.

In pursuit of event-assisted low-light VOS, three pivotal challenges must be
addressed: (i) The absence of a dedicated dataset tailored to low-light
conditions for video object segmentation, encompassing both frames and events.
Existing datasets are captured in standard lighting, leading to a significant
gap in accurately representing low-light situations. (ii) Effectively
exploiting complementary information from frame and event modalities under
low-light conditions remains a complex task. Traditional approaches to
integrate this data fall short due to inherent noise in low-light settings,
necessitating the development of a more robust integration strategy for these
modalities. (iii) Optimizing the utilization of event assistance for matching
poses a substantial challenge. Current methods [5, 33, 32] focus on enhancing
the matching mechanism, assuming normal lighting conditions and relying solely
on image data. Consequently, exploring effective means of leveraging event
assistance in low-light scenarios becomes essential.

To tackle the above challenges, we build low-light datasets and introduce a
novel end-to-end framework designed for low-light VOS, filling a critical
research gap in VOS. For low-light VOS data, we construct a synthetic Low-
Light Event DAVIS (LLE-DAVIS) dataset and collect a Low-Light Event Video
Object Segmentation (LLE-VOS) dataset. These datasets serve as an important
foundation for improving VOS techniques for low-light conditions. For our
framework, we propose the adaptive cross-modal fusion (ACMF) module to combine
complementary information by learning the adaptive filters to select useful
information between two modalities. Besides, we propose an event-guided memory
matching (EGMM) to solve the inaccurate matching mechanism. Our EGMM utilizes
a joint approach of mask and event to guide the network in matching the target
areas of memory, thereby enhancing the matching accuracy. We evaluate our
method on the synthetic LLE-DAVIS dataset and LLE-VOS dataset. Experiments
show our significant effectiveness, setting new standards for performance on
both the LLE-DAVIS (62.8%) and LLE-VOS (67.8%) datasets.

In brief, our contributions are summarized as follows:

  * •

We propose the first event-based low-light VOS framework by utilizing the
unique properties of event cameras.

  * •

The first real-world event-based low-light VOS dataset is constructed, which
contains event streams and frames captured under low-illumination scenarios,
clear images and accurate annotations.

  * •

We elaborately design two components, i.e., an adaptive cross-modal fusion
module and an event-guided memory matching module, for adaptively fusing the
information of both frame and event modalities and enhancing the motion
features for the matching module.

  * •

Both quantitative and qualitative results over synthetic and real-world
datasets showcase that our proposed method outperforms existing state-of-the-
art methods.

##  2 Related Work

Figure 1: Examples of our LLE-VOS dataset. The dataset contains paired normal/low-light APS images, event stream and annotations. Figure 2:  (a) A hybrid camera system for building real-world dataset. We configure two identical cameras with different exposure time for generating normal-light (b) and low-light (c) pairs. | Synthetic LLE-DAVIS Dataset | Real-world LLE-VOS Dataset
---|---|---
| Train | Validation | Total | Train | Validation | Val-Indoor | Val-Outdoor | Total
# Seq. | 60 | 30 | 90 | 50 | 20 | 10 | 10 | 70
# Frm. | 4149 | 1969 | 6118 | 3777 | 1823 | 993 | 830 | 5600
# Evt. | 1406.19M | 626.70M | 2032.89M | 575.61M | 202.93M | 70.83M | 132.10M | 778.54M
Mean # Frm. per Seq. | 69 | 66 | 68 | 76 | 91 | 99 | 83 | 80
Mean # Evt. per Seq. | 23.44M | 20.89M | 22.59M | 11.51M | 10.15M | 7.08M | 13.21M | 11.12M
Mean # Objs. per Seq. | 2.4 | 2.0 | 2.3 | 1.8 | 1.6 | 1.7 | 1.5 | 1.7
Table 1:  The summary of our synthetic LLE-DAVIS dataset and real-world LLE-
VOS dataset, including the number of sequences (#Seq.), frames (#Frm.), events
(#Evt.), the mean number of frames, events and objects (#Objs.) per sequence.

###  2.1 Video Object Segmentation

In the field of VOS, there are three categories of methods: online fine-
tuning-based methods, propagation-based methods, and matching-based methods.
Online fine-tuning-based methods [3, 31, 21] concentrate on adjusting pre-
existing segmentation networks during the evaluation phase to align them with
the specific object being targeted. Propagation-based methods [9, 4, 32, 18,
35] aim to expedite testing time by employing the mask from the previous frame
to predict the mask for the current frame. Meanwhile, Matching-based methods,
highlighted in [20, 23, 6, 5, 34, 33, 16, 24], ascertain pixel classification
by evaluating its resemblance to the target object across memory frames.

###  2.2 Event Segmentation

There has been a growing interest in tailoring segmentation methods for event
cameras, considering their advantageous features. Specifically, in event-based
motion segmentation and event-based semantic segmentation. Stoffregen _et al._
[26] proposed a distinctive per-event segmentation method aimed at estimating
event-object associations. Mitrokhin _et al._ [17] introduced a graph
convolutional neural network to address the challenge of accurately analyzing
dynamic scene evolution over time. Zhou _et al._ [40] developed an approach to
tackle motion segmentation in event-based camera data by minimizing energy and
fitting multiple motion models. On the other hand, Alonso _et al._ [1]
presented a novel representation of event data and released a new semantic
dataset for event-based semantic segmentation. Additionally, Sun _et al._ [27]
proposed an unsupervised domain adaptation method for transferring semantic
segmentation tasks from labeled image datasets to unlabeled event data.
Moreover, Xia _et al._ [30] introduced an unsupervised cross-domain framework
for effective nighttime semantic segmentation. Despite these advancements, a
prevailing challenge lies in the inability of these methods to consistently
track specific objects throughout entire video sequences. This limitation
signifies a significant area necessitating further enhancement and future
research endeavors.

###  2.3 Low-Light Event Application

Numerous studies [25, 39, 13, 14, 38] have delved into the potential of event
cameras under low-light conditions. Zhang _et al._ [36] proposed an
unsupervised domain adaptation network aimed at reconstructing images captured
by event cameras in low-light conditions to resemble those taken during
daylight. Jiang _et al._ [10] introduced a framework utilizing event cameras’
superior dynamic range to produce clear images in near-darkness by integrating
underexposed frames and event streams. Liu _et al._ [14] proposed a novel
method for enhancing low-light videos using synthetic events from multiple
frames, addressing the artifacts in extreme low light or fast-motion
scenarios. Liang _et al._ [13] suggested a video enhancement approach for low-
light conditions, establishing spatiotemporal coherence from frame-based and
event cameras through a neural network. Zhou _et al._ [39] introduced a two-
stage approach to enhance the deblurring of low-light images, leveraging the
high dynamic range and low latency of event cameras. Nonetheless, prevailing
research on event-based low-light scenarios primarily focuses on foundational
tasks, leaving the domain of VOS largely unexplored.

##  3 Benchmark Dataset

Figure 3:  (a) Overview of the proposed method for event-assisted low-light
video object segmentation. (b) The structure of Adaptive Cross-Modal Fusion
(ACMF) module. (c) The structure of Event-Guided Memory Matching (EGMM)
module.

To the best of our knowledge, there has been no event-based VOS dataset in
low-light scenarios. In this work, we build two low-light event-based VOS
datasets, consisting of a synthetic LLE-DAVIS dataset and a real-world LLE-VOS
dataset. Tab. 1 presents the summary of these two datasets, including the
number of sequences, frames, objects, and events on each of the dataset
splits. Fig. 1 shows some examples in the LLE-VOS dataset.

###  3.1 Synthetic Dataset

The DAVIS 2017 dataset is a widely used benchmark for the VOS task [19, 6, 12,
5]. Thus based on this dataset, we construct a synthetic event dataset, named
LLE-DAVIS, specifically tailored for the Low-Light Event-Based VOS task. Since
the original DAVIS 2017 videos are recorded under normal-light conditions, we
employ a devised technique [15] to generate low-light videos. Specifically,
for a given normal-light frame $I_{t}$, we introduce random adjustments to the
gamma correction factor and linear scaling factor to synthesize the
corresponding low-light frame $L_{t}$. This process is mathematically
expressed as:

$\displaystyle L_{t}=\beta\times(\alpha\times I_{t})^{\gamma}+N_{\sigma},$ (1)

Here, $\alpha$, $\beta$, $\gamma$ are randomly sampled from the uniform
distributions $U(0.9,1)$, $U(0.5,1)$ and $U(7,9)$, respectively. $N_{\sigma}$
represents Gaussian noise $N(0,\sigma)$, with $\sigma$ values drawn from a
uniform distribution $U(0,0.05)$. Subsequently, we employ FILM [22] to
interpolate DAVIS videos to 100fps. Then, these frames are fed into the ESIM
model [7] to generate events. In total, we have assembled 90 low-light video
sequences, each accompanied by temporally-synchronized event streams.

###  3.2 Real-World Dataset

To collect the real-world LLE-VOS dataset, we build a hybrid camera system as
shown in Fig. 2(a). The hybrid camera system is equipped with two DAVIS346
event cameras [28], each adept at capturing temporally synchronized APS frames
and event streams. A beam splitter is integrated into the system to uniformly
distribute incoming light, guaranteeing that both cameras record identical
scenes. Additionally, we conduct accurate geometric calibration and temporal
synchronization between two cameras. More details are provided in the
Supplementary Material. We adjust the exposure times of these two cameras to
collect a pair of normal-light and low-light videos in one shot. We employ a
group of 20 volunteers to accomplish the annotation work.

Our final dataset includes 70 videos, consisting of paired normal and low-
light videos, along with their corresponding segmentation annotations and
event streams. The videos are recorded at a diverse range of locations,
including gyms, playgrounds, classrooms, meeting rooms, and zoos. To enhance
the robustness of the dataset, it includes varying lighting conditions and
contains a rich spectrum of motion information. We randomly select 50 video
clips as the training set and 20 as the testing set. To further evaluate the
robustness of our proposed method across various scenes, we divide the test
set into indoor and outdoor scenarios, with 10 clips each. To our best, this
is the first real-world dataset for low-light event-based VOS. It should be
noted that this dataset can also be applied to other tasks such as event
motion segmentation and low-light video enhancement.

##  4 Method

###  4.1 Overview

We propose a novel event-assisted VOS framework, designed specifically for
low-light conditions, as depicted in Fig. 3(a). The core components of our
architecture comprise an event encoder, an image encoder, an Event-Guided
Memory Matching (EGMM) module, an Adaptive Cross-Modal Fusion (ACMF) module,
and an Identity Assignment (ID) module [34]. The event encoder and image
encoder extract the event feature and image feature, respectively. Then the
multi-scale features of events and images are processed by the ACMF module,
which generates complementary features. At the first time step, the EGMM
module uses these features to produce query $Q_{1}$, key $K_{1}$ and value
$V_{1}$. Subsequently, the ID module processes an initial segmentation
annotation to create a mask feature. This generated mask feature, along with
the predicted mask from the EGMM and the event features, are then
systematically archived in the Memory Bank. From the second time step until
the final moment, the Memory Bank provides stored representations to the EGMM,
along with the representations from image and event encoder. The EGMM then
continually generates the mask at each time step, which is preserved in the
Memory Bank for subsequent iterative refinement.

###  4.2 Event Representation

Our proposed framework transforms an asynchronous event streams from $t-1$ to
$t$ into corresponding voxel grids, denoted by $E_{t-1\rightarrow
t}\in\mathbb{R}^{B\times H\times W}$, where $B$, $H$ and $W$ represent the
number of temporal bins, the height and width of the grids, respectively.

###  4.3 Adaptive Cross-Modal Fusion

We introduce the Adaptive Cross-Modal Fusion (ACMF) to adaptively select the
effect information from event and image in low-light environments, which is
depicted in Fig. 3(b). Under low-light conditions, the image modality provides
inadequate texture and color information of objects while the event modality
offers richer edge and motion cues. These distinct attributes of events are
advantageous for segmentation tasks.

Initially, ACMF merges the image feature $F_{t}^{img}$ and event feature
$F_{t}^{evt}$ to create a more comprehensive feature set. This combined
feature set is then processed to capture coupled information. The processed
features act as a filter that selectively integrates information from two
features through element-wise multiplications, reducing noise and enhancing
relevant details. Diminished image quality under low-light conditions often
results in lost structural details. Therefore, a channel-attention (CA) and a
spatial-attention (SA) extract edge information from event feature to complete
such details. This information helps to restore invisible object contours. Two
convolutional layers further enhance two features. The final output is yielded
by summing two features, acting as a robust representation for VOS.

###  4.4 Event-Guided Memory Matching

Our proposed Event-Guided Memory Matching (EGMM) introduces a novel mechanism
designed to enhance the accuracy of object matching between the current
feature $F_{t}$ and the prior feature $K_{t-1}$, specifically under
circumstances of imprecise mask predictions. The EGMM architecture,
illustrated in Fig. 3(b), integrates event feature with mask feature from
memory to concentrate on motion-specific areas within a scene, consequently
refining feature matching.

Initially, the current feature $F_{t}$ generates the $Q_{t}$, $K_{t}$, $V_{t}$
through linear layer. $K_{t}$ and $V_{t}$ are saved in the Memory Bank for the
next time step. Then, the event feature $M_{t-1}^{\text{evt}}$ and mask
feature $M_{t-1}^{\text{mask}}$ from Memory Bank are sent to the Guided
Module, showing in the right of Fig. 3(c). The purpose of the guided branch is
to improve unreliable mask predictions. It first integrates the multi-scale
information from the concatenated feature through three parallel convolutional
layers with various kernel sizes. Then, pooling layers and convolutional
layers further reform its channel context, resulting in a guided signal for
more accurate matching. Subsequently, the guided signal after passing a
Sigmoid function is multiplicated with the past feature $K_{t-1}$, filtering
misaligned features of the object region. The filtering process is described
by the equation:

$\displaystyle G_{t}$
$\displaystyle=\text{Guide}(M_{t-1}^{\text{evt}},M_{t-1}^{\text{mask}})$ (2)
$\displaystyle K^{\prime}_{t-1}$ $\displaystyle=K_{t-1}\cdot G_{t},$ (3)

where Guide$(.,.)$ represents the Guided module.
$K_{t-1},M_{t-1}^{\text{evt}}$ and $M_{t-1}^{\text{mask}}$ represent the
previous key, event and mask features derived from memory, respectively.
$K^{\prime}_{t-1}$ and $G$ are the filtered key and guided branch output,
respectively.

The filtered key $K^{\prime}_{t-1}$ interacts with $Q_{t}$ derived from
$F_{t}$ through multiplication and softmax, formulated by:

$A_{t}=\text{Softmax}\left(\frac{Q_{t}(K^{\prime}_{t-1})^{T}}{\sqrt{d_{k}}}\right),$
(4)

where $A_{t}$ is the attention map and $d_{k}$ is the scaling factor
corresponding to the dimension of the key vectors. The matching result $R_{t}$
is obtained by multiplication between the attention map $A_{t}$ and the
summation of $G_{t}$ and $V_{t-1}$:

$R_{t}=A_{t}(G_{t}+V_{t-1}).$ (5)

Finally, we concat the current feature and matching result to combine the
current and memory information. Then the concatenation result is sent to the
mask decoder to generate the current mask.

$Mask_{t}=Decoder(Concat(R_{t},F_{t})).$ (6)

This EGMM module thereby provides a robust solution for enhancing the
consistency of VOS, proving particularly beneficial in scenarios with
unreliable mask predictions.

Method | Indoor Scenes | Outdoor Scenes | Overall
---|---|---|---
$\mathcal{J}$ | $\mathcal{F}$ | $\mathcal{J}\&\mathcal{F}$ | $\mathcal{J}$ | $\mathcal{F}$ | $\mathcal{J}\&\mathcal{F}$ | $\mathcal{J}$ | $\mathcal{F}$ | $\mathcal{J}\&\mathcal{F}$
STCN [NIPS2021] [6] | 0.486 | 0.321 | 0.403 | 0.400 | 0.309 | 0.354 | 0.445 | 0.316 | 0.380
XMem [ECCV2022] [5] | 0.664 | 0.528 | 0.596 | 0.507 | 0.456 | 0.481 | 0.590 | 0.494 | 0.542
AOT [NIPS2021] [34] | 0.699 | 0.618 | 0.659 | 0.592 | 0.571 | 0.581 | 0.649 | 0.596 | 0.623
DeAOT [NIPS2022] [33] | 0.716 | 0.643 | 0.680 | 0.580 | 0.580 | 0.580 | 0.653 | 0.614 | 0.633
Zero-DCE [8] \+ STCN [6] | 0.513 | 0.334 | 0.424 | 0.415 | 0.313 | 0.364 | 0.467 | 0.324 | 0.396
Zero-DCE [8] \+ XMem [5] | 0.681 | 0.527 | 0.604 | 0.535 | 0.497 | 0.516 | 0.612 | 0.513 | 0.563
Zero-DCE [8] \+ AOT [34] | 0.694 | 0.601 | 0.647 | 0.568 | 0.555 | 0.562 | 0.635 | 0.579 | 0.607
Zero-DCE [8] \+ DeAOT [33] | 0.648 | 0.595 | 0.621 | 0.594 | 0.586 | 0.590 | 0.622 | 0.590 | 0.606
Ours | 0.789 | 0.710 | 0.749 | 0.604 | 0.588 | 0.596 | 0.702 | 0.653 | 0.678
Table 2: Quantitative comparisons of various VOS methods on the real-world
LLE-VOS dataset. The best results are marked in bold.

###  4.5 Loss function in VOS

To effectively train our VOS framework, we implement a composite loss
function, denoted as $\mathcal{L}$. This function combines the Binary Cross-
Entropy (BCE) loss and the Soft Jaccard (SJ) loss. The overall loss is
formalized as:

$\mathcal{L}=\sum_{t=1}^{T}\left(\alpha\sum_{o=1}^{N}\mathcal{L}_{\text{BCE}}^{(t,o)}+\beta\sum_{o=1}^{N}\mathcal{L}_{\text{SJ}}^{(t,o)}\right),$
(7)

where $\mathcal{L}_{\text{BCE}}^{(t,o)}$ and $\mathcal{L}_{\text{SJ}}^{(t,o)}$
represent the BCE loss and SJ loss for the object $o$ at time step $t$,
respectively. $N$ denotes the total number of segmented objects. The
parameters $\alpha$ and $\beta$ are the weights attributed to the BCE loss and
IoU loss, respectively. $T$, $\alpha$, $\beta$ are set as 5, 0.5 and 0.5,
respectively.

##  5 Experiment

###  5.1 Comparison Methods

We compare our method with state-of-the-art VOS methods, which include: STCN
[6], XMem [5], AOT [34], and DeAOT [33]. In addition to these direct VOS
methods, we also compare our method with two-step approachs, which first
utilize Zero-DCE [8] for low-light video enhancement and then apply above
mentioned methods for VOS.

Following [6, 5], the evaluation of the VOS task employs the $\mathcal{J}$
score for segmentation accuracy, reflecting the average IoU between
predictions and ground truth, and the $\mathcal{F}$ score for boundary
preciseness, comparing the similarity of segmentation edges to actual
contours. The mean of these scores denoted as $\mathcal{J}\&\mathcal{F}$,
offers a comprehensive and balanced measure of overall performance.

###  5.2 Implementation Details

For model optimization, we deploy the AdamW optimizer with an initial learning
rate of $2\times 10^{-4}$ and a weight decay of 0.07. We set our batch size to
8 and train our model over 50,000 iterations. For a fair comparison, we
maintain the original training strategies of the models to obtain the best
models. We apply standard data augmentation techniques: random scaling, random
cropping, random horizontal flipping, and resizing. Besides, we randomly
reverse the video and event sequences. The crop size of random cropping is
(465, 465) for the LLE-DAVIS dataset and (256, 256) for the LLE-VOS dataset.
All models are trained from scratch on both synthetic and real-world datasets
under the same settings to ensure equitable comparisons. All the training
experiments are conducted on four NVIDIA A800 GPUs.

###  5.3 Experimental Results

####  5.3.1 Quantitative Results

Synthetic Dataset. Tab. 3 provides a detailed quantitative analysis of our
framework in comparison with other methods on the LLE-DAVIS dataset. The
comparative results clearly demonstrate that our end-to-end approach combined
with event and image consistently outperforms the existing methods in terms of
standard VOS metrics. Notably, our method exhibits a substantial performance
increase with an improvement of 6.9% over the AOT method in terms of
$\mathcal{J}\&\mathcal{F}$. Additionally, our method significantly improves
upon the two-stage method Zero-DCE+AOT, with increments of 6.7% for
$\mathcal{J}\&\mathcal{F}$. These improvements are due to the robust feature
extraction capabilities of our framework and its effective integration of both
image and event data in our VOS framework.

Method | LLE-DAVIS
---|---
$\mathcal{J}$ | $\mathcal{F}$ | $\mathcal{J}\&\mathcal{F}$
STCN [NIPS2021] [6] | 0.424 | 0.453 | 0.438
XMem [ECCV2022] [5] | 0.465 | 0.477 | 0.471
AOT [NIPS2021] [34] | 0.540 | 0.578 | 0.559
DeAOT [NIPS2022] [33] | 0.541 | 0.571 | 0.556
Zero-DCE [8] \+ STCN [6] | 0.440 | 0.469 | 0.455
Zero-DCE [8] \+ XMem [5] | 0.494 | 0.512 | 0.503
Zero-DCE [8] \+ AOT [34] | 0.544 | 0.577 | 0.561
Zero-DCE [8] \+ DeAOT [33] | 0.541 | 0.577 | 0.559
Ours | 0.602 | 0.654 | 0.628
Table 3:  Quantitative comparisons of various VOS methods on the synthetic
LLE-DAVIS dataset. The best results are marked in bold.  Figure 4:
Qualitative comparisons with other methods on the synthetic LLE-DAVIS dataset.
Figure 5: Qualitative comparisons with other methods on the real-world LLE-VOS
dataset.

Real-world Dataset. Tab. 2 presents the quantitative results of one-stage and
two-stage VOS methods on the LLE-VOS dataset. Given that real-world scenarios
often involve extremely low-light conditions, the Zero-DCE method struggles to
perform effectively. Therefore, the two-stage method Zero-DCE+AOT does not
perform better than the AOT method. However, our method can improve the
$\mathcal{J}\&\mathcal{F}$ metric over the existing one-stage VOS methods,
with an increment when compared to DeAOT. Furthermore, our method outperforms
Zero-DCE+AOT. In particular, it increases the $\mathcal{J}\&\mathcal{F}$ score
from 0.607 to 0.678 in comparison to Zero-DCE+AOT. Our approach is measured
against other one-stage and two-stage methods.

###  5.4 Qualitative Results

Fig. 4 showcases a qualitative comparison of our proposed method against both
one-stage and two-stage methods on the synthetic LLE-DAVIS dataset under low-
light conditions. A key observation is that our method produces precise object
masks that closely match the groundtruth. Instead, AOT either misses details
or overlays redundant masks onto areas without objects. The Zero-DCE+AOT
method offers enhancements over AOT alone, yet it fails to achieve the same
level of clarity and precision provided by our method. These examples
illustrate our method’s superior ability to distinguish and outline objects in
conditions where light is limited, demonstrating the practical advantage of
integrating image and event data. This integration proves especially
beneficial in difficult lighting, ensuring our VOS framework remains effective
and reliable.

Real-world Dataset. Fig. 5 illustrates qualitative results between our method
and other approaches AOT and Zero-DCE+AOT on the real-world LLE-VOS dataset.
The results of AOT method often miss parts of the objects, but our method is
much better at finding the full shape, even when objects are hard to see in
the frame. Our method stands out against Zero-DCE+AOT by being more precise
and not mixing up different objects. This capability demonstrates that our
approach is not only effective in simulated conditions but also maintains its
reliability on real-world data, proving its practical applicability in low-
light conditions.

| Method | $\mathcal{J}$ | $\mathcal{F}$ | $\mathcal{J}\&\mathcal{F}$
---|---|---|---|---
Fwk Input | Image Only | 0.540 | 0.578 | 0.559
Event Only | 0.532 | 0.563 | 0.547
Image + Event | 0.602 | 0.654 | 0.628
Fwk Mod. | Baseline | 0.555 | 0.614 | 0.584
\+ ACMF | 0.578 | 0.623 | 0.601
\+ ACMF + EGMM | 0.602 | 0.654 | 0.628
EGMM Input | None | 0.578 | 0.623 | 0.601
Mask | 0.461 | 0.468 | 0.464
Event | 0.540 | 0.563 | 0.552
Event + Mask | 0.602 | 0.654 | 0.628
Table 4: Ablation results of different configurations on the VOS task. ‘Fwk
Input’, ‘Fwk Module’ and ‘EGMM Input’ represent the input type of framework
input, framwork module and the input type of EGMM module, respectively. The
best results are marked in bold.

###  5.5 Ablation Study

Impact of input modalities. In the first part of Tab. 4, we show the
comparative performance of the VOS network with different inputs. Utilizing
only the image modality as input, the network achieves a
$\mathcal{J}\&\mathcal{F}$ score of 0.559. In contrast, relying solely on the
event modality yields a $\mathcal{J}\&\mathcal{F}$ score of 0.547. These
results are inferior compared to the combined input model, which attains a
$\mathcal{J}\&\mathcal{F}$ score of 0.628. This marked improvement through
fusing both image and event modalities validates the necessity of multi-
modality fusion for VOS under low-light conditions.

$L$ | $\mathcal{J}$ | $\mathcal{F}$ | $\mathcal{J}\&\mathcal{F}$ | FPS
---|---|---|---|---
1 | 0.577 | 0.628 | 0.603 | 22.50
2 | 0.595 | 0.642 | 0.618 | 21.32
3 | 0.602 | 0.654 | 0.628 | 20.26
4 | 0.594 | 0.649 | 0.622 | 19.16
Table 5:  Ablation results of different numbers of EGMM blocks on the LLE-
DAVIS dataset. The best results are marked in bold.

The effectiveness of ACMF and EGMM. The second part of Tab. 4 validates the
effectiveness of the ACMF and EGMM modules within our framework. Initially,
our baseline method employs a straightforward fusion approach by concatenating
event and image features. The integration of the ACMF module enhances
performance, improving the $\mathcal{J}$ score to 0.578, the $\mathcal{F}$
score to 0.623, and the composite $\mathcal{J}\&\mathcal{F}$ score to 0.601.
Incorporating both ACMF and EGMM modules further boosts the
$\mathcal{J}\&\mathcal{F}$ score to 0.628, indicating a significant
contribution to the VOS performance on the synthetic LLE-DAVIS dataset.

Fig. 6 presents a qualitative assessment of the individual and combined
impacts of ACMF and EGMM. The baseline method exhibits inaccuracies in contour
delineation and mask separation. The ACMF module refines the contour accuracy
for the yellow human. However, the network still struggles to differentiate
between the green human and the vehicle. The integration of the EGMM module
effectively enables the network to distinguish between car and human.

The effect of event and mask in EGMM. To evaluate the individual and combined
contributions of event and mask priors within the EGMM module, we present a
quantitative analysis in the third part of Tab. 4. Without EGMM, the
$\mathcal{J}\&\mathcal{F}$ score is 0.601. Utilizing mask information alone
yields a $\mathcal{J}\&\mathcal{F}$ score of 0.464 due to imprecise prediction
to disturb matching. When the model processes only event data, the
$\mathcal{J}\&\mathcal{F}$ is 0.552 due to noise and background motion.
However, the fusion of both event and mask priors significantly enhances model
performance, as evidenced by the improved $\mathcal{J}\&\mathcal{F}$ of 0.628.
These results clearly illustrate the collaborate effect of combining mask and
event information, which leads to a more robust and accurate performance in
the EGMM.

The choice of EGMM block number. Tab. 5 examines the impact of changing the
number of EGMM blocks on our model’s performance. As we increase the number of
EGMM blocks, the performance of our model improves, but it leads to more
redundancy. When the number of EGMM blocks is four, the
$\mathcal{J}\&\mathcal{F}$ score decreases. Therefore, we use three EGMM
blocks in our experimental setup.

Figure 6:  Visual results of ablation on different modules.

##  6 Conclusion

In this paper, we present a new approach to Video Object Segmentation (VOS) in
low-light conditions. Unlike traditional VOS methods that rely heavily on
high-quality video, our innovative framework uses the unique features of event
cameras to improve segmentation accuracy. Our introduction of Adaptive Cross-
Modal Fusion and Event-Guided Memory Matching has notably enhanced VOS
performance. Our thorough testing on specially created Low-Light Event DAVIS
(LLE-DAVIS) and Low-Light Event Video Object Segmentation (LLE-VOS) datasets
proves that our method is superior, setting new standards for low-light VOS.
This study paves the way for further breakthroughs in VOS and related fields.

##  7 Acknowledgments

This work was in part supported by the National Natural Science Foundation of
China (NSFC) under grants 62032006 and 62021001.

## References

  * Alonso and Murillo [2019] Inigo Alonso and Ana C Murillo.  Ev-segnet: Semantic segmentation for event-based cameras.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops_ , 2019.
  * Brandli et al. [2014] Christian Brandli, Raphael Berner, Minhao Yang, Shih-Chii Liu, and Tobi Delbruck.  A 240 $\times$ 180 130 db 3 $\mu$s latency global shutter spatiotemporal vision sensor.  _IEEE Journal of Solid-State Circuits_ , 49(10):2333–2341, 2014.
  * Caelles et al. [2017] Sergi Caelles, Kevis-Kokitsi Maninis, Jordi Pont-Tuset, Laura Leal-Taixé, Daniel Cremers, and Luc Van Gool.  One-shot video object segmentation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 221–230, 2017.
  * Chen et al. [2020] Xi Chen, Zuoxin Li, Ye Yuan, Gang Yu, Jianxin Shen, and Donglian Qi.  State-aware tracker for real-time video object segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 9384–9393, 2020.
  * Cheng and Schwing [2022] Ho Kei Cheng and Alexander G Schwing.  Xmem: Long-term video object segmentation with an atkinson-shiffrin memory model.  In _European Conference on Computer Vision (ECCV)_ , pages 640–658. Springer, 2022.
  * Cheng et al. [2021] Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang.  Rethinking space-time networks with improved memory coverage for efficient video object segmentation.  _Advances in Neural Information Processing Systems_ , 34:11781–11794, 2021.
  * Gehrig et al. [2020] Daniel Gehrig, Mathias Gehrig, Javier Hidalgo-Carrió, and Davide Scaramuzza.  Video to events: Recycling video datasets for event cameras.  In _IEEE Conf. Comput. Vis. Pattern Recog. (CVPR)_ , 2020.
  * Guo et al. [2020] Chunle Guo Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong.  Zero-reference deep curve estimation for low-light image enhancement.  In _Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)_ , pages 1780–1789, 2020.
  * Jampani et al. [2017] Varun Jampani, Raghudeep Gadde, and Peter V Gehler.  Video propagation networks.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 451–461, 2017.
  * Jiang et al. [2023a] Yu Jiang, Yuehang Wang, Siqi Li, Yongji Zhang, Minghao Zhao, and Yue Gao.  Event-based low-illumination image enhancement.  _IEEE Transactions on Multimedia_ , pages 1–12, 2023a.
  * Jiang et al. [2023b] Yu Jiang, Yuehang Wang, Siqi Li, Yongji Zhang, Minghao Zhao, and Yue Gao.  Event-based low-illumination image enhancement.  _IEEE Transactions on Multimedia_ , 26:1920–1931, 2023b.
  * Li et al. [2022] Mingxing Li, Li Hu, Zhiwei Xiong, Bang Zhang, Pan Pan, and Dong Liu.  Recurrent dynamic embedding for video object segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 1332–1341, 2022.
  * Liang et al. [2023] Jinxiu Liang, Yixin Yang, Boyu Li, Peiqi Duan, Yong Xu, and Boxin Shi.  Coherent event guided low-light video enhancement.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 10615–10625, 2023.
  * Liu et al. [2023] Lin Liu, Junfeng An, Jianzhuang Liu, Shanxin Yuan, Xiangyu Chen, Wengang Zhou, Houqiang Li, Yan Feng Wang, and Qi Tian.  Low-light video enhancement with synthetic event guidance.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 1692–1700, 2023.
  * Lv et al. [2021] Feifan Lv, Yu Li, and Feng Lu.  Attention guided low-light image enhancement with a large scale low-light simulation dataset.  _International Journal of Computer Vision_ , 129(7):2175–2193, 2021.
  * Mao et al. [2021] Yunyao Mao, Ning Wang, Wengang Zhou, and Houqiang Li.  Joint inductive and transductive learning for video object segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 9670–9679, 2021.
  * Mitrokhin et al. [2020] Anton Mitrokhin, Zhiyuan Hua, Cornelia Fermuller, and Yiannis Aloimonos.  Learning visual motion segmentation using event surfaces.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 14414–14423, 2020.
  * Oh et al. [2018] Seoung Wug Oh, Joon-Young Lee, Kalyan Sunkavalli, and Seon Joo Kim.  Fast video object segmentation by reference-guided mask propagation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 7376–7385, 2018.
  * Oh et al. [2019a] Seoung Wug Oh, Joon-Young Lee, Ning Xu, and Seon Joo Kim.  Video object segmentation using space-time memory networks.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , 2019a.
  * Oh et al. [2019b] Seoung Wug Oh, Joon-Young Lee, Ning Xu, and Seon Joo Kim.  Video object segmentation using space-time memory networks.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 9226–9235, 2019b.
  * Perazzi et al. [2017] Federico Perazzi, Anna Khoreva, Rodrigo Benenson, Bernt Schiele, and Alexander Sorkine-Hornung.  Learning video object segmentation from static images.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 2663–2672, 2017.
  * Reda et al. [2022] Fitsum Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless.  Film: Frame interpolation for large motion.  In _European Conference on Computer Vision_ , pages 250–266. Springer, 2022.
  * Seong et al. [2020] Hongje Seong, Junhyuk Hyun, and Euntai Kim.  Kernelized memory network for video object segmentation.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16_ , pages 629–645. Springer, 2020.
  * Seong et al. [2021] Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seongwon Lee, Suhyeon Lee, and Euntai Kim.  Hierarchical memory matching network for video object segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 12889–12898, 2021.
  * Shi et al. [2023] Peilun Shi, Jiachuan Peng, Jianing Qiu, Xinwei Ju, Frank Po Wen Lo, and Benny Lo.  Even: An event-based framework for monocular depth estimation at adverse night conditions.  _arXiv preprint arXiv:2302.03860_ , 2023.
  * Stoffregen et al. [2019] Timo Stoffregen, Guillermo Gallego, Tom Drummond, Lindsay Kleeman, and Davide Scaramuzza.  Event-based motion segmentation by motion compensation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 7244–7253, 2019.
  * Sun et al. [2022] Zhaoning Sun, Nico Messikommer, Daniel Gehrig, and Davide Scaramuzza.  Ess: Learning event-based semantic segmentation from still images.  In _European Conference on Computer Vision (ECCV)_ , pages 341–357. Springer, 2022.
  * Taverni et al. [2018] Gemma Taverni, Diederik Paul Moeys, Chenghan Li, Celso Cavaco, Vasyl Motsnyi, David San Segundo Bello, and Tobi Delbruck.  Front and back illuminated dynamic and active pixel vision sensors comparison.  _IEEE Transactions on Circuits and Systems II: Express Briefs_ , 65(5):677–681, 2018.
  * Wang et al. [2017] Wenguan Wang, Jianbing Shen, and Fatih Porikli.  Selective video object cutout.  _IEEE Transactions on Image Processing_ , 26(12):5645–5655, 2017.
  * Xia et al. [2023] Ruihao Xia, Chaoqiang Zhao, Meng Zheng, Ziyan Wu, Qiyu Sun, and Yang Tang.  Cmda: Cross-modality domain adaptation for nighttime semantic segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 21572–21581, 2023.
  * Xiao et al. [2018] Huaxin Xiao, Jiashi Feng, Guosheng Lin, Yu Liu, and Maojun Zhang.  Monet: Deep motion exploitation for video object segmentation.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , pages 1140–1148, 2018.
  * Xu et al. [2022] Xiaohao Xu, Jinglu Wang, Xiao Li, and Yan Lu.  Reliable propagation-correction modulation for video object segmentation.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 2946–2954, 2022.
  * Yang and Yang [2022] Zongxin Yang and Yi Yang.  Decoupling features in hierarchical propagation for video object segmentation.  _Advances in Neural Information Processing Systems_ , 35:36324–36336, 2022.
  * Yang et al. [2021] Zongxin Yang, Yunchao Wei, and Yi Yang.  Associating objects with transformers for video object segmentation.  _Advances in Neural Information Processing Systems_ , 34:2491–2502, 2021.
  * Zhang et al. [2019] Lu Zhang, Zhe Lin, Jianming Zhang, Huchuan Lu, and You He.  Fast video object segmentation via dynamic targeting network.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , pages 5582–5591, 2019.
  * Zhang et al. [2020] Song Zhang, Yu Zhang, Zhe Jiang, Dongqing Zou, Jimmy Ren, and Bin Zhou.  Learning to see in the dark with events.  In _Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XVIII 16_ , pages 666–682. Springer, 2020.
  * Zhang et al. [2016] Ziyu Zhang, Sanja Fidler, and Raquel Urtasun.  Instance-level segmentation for autonomous driving with deep densely connected mrfs.  In _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , pages 669–677, 2016.
  * Zhou et al. [2021a] Chu Zhou, Minggui Teng, Jin Han, Chao Xu, and Boxin Shi.  Delieve-net: Deblurring low-light images with light streaks and local events.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 1155–1164, 2021a.
  * Zhou et al. [2023] Chu Zhou, Minggui Teng, Jin Han, Jinxiu Liang, Chao Xu, Gang Cao, and Boxin Shi.  Deblurring low-light images with events.  _International Journal of Computer Vision_ , 131(5):1284–1298, 2023.
  * Zhou et al. [2021b] Yi Zhou, Guillermo Gallego, Xiuyuan Lu, Siqi Liu, and Shaojie Shen.  Event-based motion segmentation with spatio-temporal graph cuts.  _IEEE Transactions on Neural Networks and Learning Systems_ , 34(8):4868–4880, 2021b.