# On the Gauss-Epple homomorphism of the braid group $B_{n}$, and
generalizations to Artin groups of crystallographic type

Joshua Guo111Newton South High School Kevin Chang222Department of Mathematics,
Columbia University, New York, NY 10027

###### Abstract

In this paper, we introduce a broad family of group homomorphisms that we name
the Gauss-Epple homomorphisms. In the setting of braid groups, the Gauss-Epple
invariant was originally defined by Epple based on a note of Gauss as an
action of the braid group $B_{n}$ on the set
$\\{1,\dots,n\\}\times\mathbb{Z}$; we prove that it is well-defined. We
consider the associated group homomorphism from $B_{n}$ to the symmetric group
$\operatorname{Sym}(\\{1,\dots,n\\}\times\mathbb{Z})$. We prove that this
homomorphism factors through $\mathbb{Z}^{n}\rtimes S_{n}$ (in fact, its image
is an order 2 subgroup of the previous group). We also describe the kernel of
the homomorphism and calculate the asymptotic probability that it contains a
random braid of a given length. Furthermore, we discuss the super-Gauss-Epple
homomorphism, a homomorphism which extends the generalization of the Gauss-
Epple homomorphism and describe a related 1-cocycle of the symmetric group
$S_{n}$ on the set of antisymmetric $n\times n$ matrices over the integers. We
then generalize the super-Gauss-Epple homomorphism and the associated
1-cocycle to Artin groups of finite type. For future work, we suggest studying
possible generalizations to complex reflection groups and computing the vector
spaces of Gauss-Epple analogues.

##  1 Introduction

###  1.1 Braid groups

Figure 1: An example braid

Fix, once and for all, an arbitrary integer $n\geq 2$; we shall use the
notation $[n]:=\\{1,\dots,n\\}$. A braid on $n$ strands is a topological
object consisting of $n$ strands in 3-dimensional space whose endpoints are
fixed to two distinguished parallel planes, such as in Figure 1. (We assume,
as is common, that the endpoints on each plane are collinear. We also choose
one plane to be the “top” and number the strands from $1$ to $n$ in their
order on this plane; we choose the other to be the “bottom”.)

Figure 2: Braid multiplication

By “gluing” two braids on $n$ strands together (i.e., identifying the “bottom”
plane of the first with the “top” plane of the second, as depicted in Figure
2), we can create a third braid on $n$ strands. This operation, which we view
as composition, endows the set of all braids on $n$ strands up to isotopy
(i.e., topological deformation) with the structure of a monoid (i.e., a set
with an associative and identity). In fact, this operation is invertible; an
example braid inverse is shown in Figure 3. As a result, the set of all braids
on $n$ strands up to isotopy is in fact a group, the braid group $B_{n}$.

Figure 3: A braid and its inverse

The braid group $B_{n}$ is a well-studied mathematical object. Emil Artin
first proposed it in 1925 [3] and further discussed it in a 1947 paper [2]. It
is well-known that $B_{n}$ has no torsion and that the full twist generates
its center. Several efficient algorithms to solve $B_{n}$’s word and conjugacy
problems are known [8]. It is also well-known (as the Nielsen-Thurston
classification) that all braids are either periodic, reducible, or pseudo-
Asonov [8]. Furthermore, $B_{n}$ is linear, as there is a faithful (!)
representation (the Lawrence-Krammer representation) $B_{n}\to
GL_{n(n-1)/2}(\mathbb{Z}[q^{\pm 1},t^{\pm 1}])$ [10].

The braid group $B_{n}$ has a canonical generating set of $n-1$ generators
known as the Artin generators and conventionally denoted
$\sigma_{1},\dots,\sigma_{n-1}$. The Artin generator $\sigma_{i}$ consists of
the braid that twists the $i$th leftmost strand over and to the right of the
$(i+1)$th strand. With this generating set, the braid group has the following
presentation:

$\displaystyle
B_{n}=\langle\sigma_{1},\dots,\sigma_{n-1}|\sigma_{i}\sigma_{i+1}\sigma_{i}=\sigma_{i+1}\sigma_{i}\sigma_{i+1}\forall
i,\sigma_{i}\sigma_{j}=\sigma_{j}\sigma_{i}\forall i,j:|i-j|>1\rangle.$

The writhe is a group homomorphism $B_{n}\to\mathbb{Z}$ defined by the
relation $\sigma_{i}\mapsto 1$. The image of any braid $\beta$ under writhe is
called the writhe of the braid, denoted $|\beta|$. The permutation is a group
homomorphism $B_{n}\to S_{n}$ defined by the relation
$\sigma_{i}\mapsto(i,i+1)$, where the right hand side is a transposition. The
permutation of a braid is defined similarly and denoted $\pi_{\beta}$.

Figure 4: The braid $(\sigma_{2}^{-1}\sigma_{1}^{-1})^{2}$, which has
permutation (1,3,2) and writhe $-4$.

Both the writhe and permutation are surjective homomorphisms but not
injective. The writhe-permutation homomorphism $WP:B_{n}\to\mathbb{Z}\times
S_{n}$ is defined by $\beta\mapsto(|\beta|,\pi_{\beta})$. It is neither
surjective nor injective, as we shall prove later.

###  1.2 Artin groups of crystallographic type

An Artin group $\mathcal{A}$ is a group presented by a finite set of
generators and at most one braid relation (i.e., a relation of the form $a=b$,
$ab=ba$, $aba=bab$, $abab=baba$, etc.) between any two generators. The
presentation of an Artin group can be depicted in a Dynkin diagram, as shown
in Figure 5.

Figure 5: Dynkin diagrams of finite Coxeter groups [9] Figure 6: The $G_{2}$
root system with two simple roots highlighted [4]

A Coxeter group $\mathcal{C}$ is a group presented with the generators and
relations of an Artin group and the relations that the square of any generator
is the identity. A Coxeter group’s presentation can also be depicted in a
Dynkin diagram.

Sometimes, a Coxeter group is generated by the reflections upon a root system
$\Phi$, which is a special set of vectors. (In this case, they are known as
Weyl groups.) The associated Artin group $\mathcal{A}$ is then called an Artin
group of crystallographic type. Furthermore, in the cases where the Coxeter
group is indeed associated with a root system, an isomorphism between the
Coxeter group and the group of linear transformations sends the canonical
Coxeter group generators to the reflections associated with some choice of
simple roots $\Delta\subset\Phi$. (We refer to the simple root associated to a
canonical Artin generator $a$ of $\mathcal{A}$ as $\Delta_{a}$.) One such set
of simple roots, corresponding to the case commonly referred to as $G_{2}$, is
shown in Figure 6.

###  1.3 Complex reflection groups

Figure 7: Partial table of complex reflection groups [6]

We adopt the terminology and notation of Broué [5].

Let $V$ be some finite-dimensional $\mathbb{C}$-vector space. A pseudo-
reflection is a linear transformation that acts trivially on some hyperplane
(called its reflecting hyperplane).

A complex reflection group $W$ is a finite subgroup of $GL(V)$ generated by
pseudo-reflections.

We denote the set of pseudo-reflections in $W$ by $\Psi$. (This notation is an
exception; it comes from Dr. Trinh’s notes [11], not from Broué.) The
hyperplane arrangement in $V$ consisting of the reflecting hyperplanes of the
$\Psi$ is denoted by $\mathcal{A}$, and its hyperplane complement
$V-\cup_{H\in\mathcal{A}}H$ is denoted as $M$. We define the analogous braid
group $B:=\pi_{1}(M/W)$; in other words, the fundamental group of $M$ up to
rotation by elements of $W$.

Similar to the previous case, the braid groups $B$ have canonical Artin-like
presentations, and the corresponding Coxeter groups $W$ have canonical
Coxeter-like presentations [5]. More precisely, there exists a subset
$\mathbf{S}=\\{\mathbf{s}_{1},\dots,\mathbf{s}_{n}\\}$ of $B$ consisting of
distinguished braid reflections, and a set $R$ of relations of the form
$w_{1}=w_{2}$, where $w_{1}$ and $w_{2}$ are positive words of equal length in
the elements of $\mathbf{S}$, such that $\langle S|R\rangle$ is a presentation
of $B$. Moreover, their images $s_{1},\dots,s_{n}$ in $W$ generate $W$, and
the group $W$ is presented by

$\langle S|R;(\forall s\in S)(s^{e_{s}}=1)\rangle$

where $e_{s}$ denotes the order of $s$ in $W$.

There is a quotient map $B\to W$; we denote the image of an element $b\in B$
under this quotient map simply as $W(b)$.

###  1.4 Overview of results

We study a large family of mathematical objects that we call the Gauss-Epple
homomorphisms. This research helps us understand the structure of braid groups
$B_{n}$, which describe the structure of braids, a kind of topological object.

The first such homomorphism was implicitly introduced by Epple [7] as an
action of $B_{n}$ based on a note by Gauss. Around the time of his note, Gauss
was primarily interested in topology for its applications to electromagnetism
and celestial dynamics. In this work, we generalize this concept to a broader
family of homomorphisms from Artin groups of finite type, a large family of
groups including the braid groups.

Firstly, in Section 2, we prove that there exists a well-defined and unique
left group action of $B_{n}$ on $\mathbb{Z}\times[n]$, as implicit in [7]. We
refer to this action as the Gauss-Epple action. It is equivalent to a group
homomorphism from $B_{n}$ to $\operatorname{Sym}(\mathbb{Z}\times[n])$, which
we call the Gauss-Epple homomorphism (denoted $GE$). In Subsection 2.1, we
describe the writhe-permutation homomorphism $WP:B_{n}\to\mathbb{Z}\times
S_{n}$ (the homomorphism that maps a braid to its writhe and underlying
permutation) in greater detail, as we treat this homomorphism as a toy model
of $GE$. We show that the image of writhe-permutation homomorphism
$WP:B_{n}\to\mathbb{Z}\times S_{n}$ is a particular order 2 subgroup of
$\mathbb{Z}\times S_{n}$. Then, in Subsection 2.2, we show that the image of
$GE$ is (isomorphic to) an order 2 subgroup of $\mathbb{Z}^{n}\rtimes S_{n}$.
In Subsection 2.3, we discuss the kernel of $GE$. We find that this group is
strictly contained in the kernel of the writhe-permutation homomorphism.

We summarize these results with the following commutative diagram:

${{B_{n}}}$${{\mathbb{Z}^{n}\rtimes
S_{n}}}$${{\operatorname{Sym}([n]\times\mathbb{Z})}}$${{\mathbb{Z}\times
S_{n}}}$$\scriptstyle{WP}$$\scriptstyle{GE}$

Each of the maps in this diagram is a group homomorphism, as explained below:

  * •

The map $GE$, from $B_{n}$ to $\mathrm{Sym}(\mathbb{Z}^{2})$, is the Gauss-
Epple action of $B_{n}$ on $\mathbb{Z}^{2}$.

  * •

The map from $B_{n}$ to $\mathbb{Z}^{n}\rtimes S_{n}$ maps a braid to the
tuple of its vector and its permutation.

  * •

The map from $\mathbb{Z}^{n}\rtimes S_{n}$ to $\mathrm{Sym}(\mathbb{Z}^{2})$
is the map $(\pi,\ell)\mapsto((a,b)\mapsto(\pi(a),b+\ell_{a}))$.

  * •

The map $WP$, from $B_{n}$ to $\mathbb{Z}\times S_{n}$, maps a braid to its
writhe and braid permutation.

  * •

The map from $\mathbb{Z}^{n}\rtimes S_{n}$ to $\mathbb{Z}\times S_{n}$ maps
$(\ell,\pi)$ to $(\sum\ell,\pi)$.

In Section 3, we briefly discuss another action of $B_{n}$ (the symmetric-
Gauss-Epple action) mentioned by Epple. It turns out that this action has very
similar properties to the Gauss-Epple action, including sharing the same
kernel.

In Section 4, we discuss the super-Gauss-Epple homomorphism (denoted $SGE$), a
homomorphism of $B_{n}$ that refines $GE$. To describe the image of $SGE$, we
introduce a 1-cocycle of the symmetric group $S_{n}$ on the set of $n\times n$
antisymmetric matrices, which we prove has a remarkable nonnegativity
property.

Finally, in Section 5, we generalize our results to the contexts of Artin
groups of finite type, a broad family of groups that generalize the braid
groups. More specifically, we introduce a family of novel homomorphisms (with
domain $\mathcal{A}$ and range $\mathbb{Z}^{\Phi}\rtimes\mathcal{C}$) that are
analogous to the super-Gauss-Epple homomorphism in the classical case of
$B_{n}$. We also note corresponding 1-cocycles, which also share an analogous
nonnegativity property.

We close in Section 6 with remarks about complex reflection groups.

####  1.4.1 Acknowledgements

We thank our advisor, Minh-Tâm Trinh, for proposing this project and giving us
helpful directions. We also thank the MIT PRIMES program for supporting our
research.

##  2 The Gauss-Epple homomorphism

Figure 8: Page 283 of Gauss’s Handbuch 7 [7]

We formally introduce the Gauss-Epple action. This action was initially
defined by Epple [7] based on a note of Gauss. Since Gauss’s notation involved
complex numbers (more precisely, the Gaussian integers $\mathbb{Z}[i]$), the
Gauss-Epple action was originally defined in terms of complex numbers. For
notational and theoretical simplicity, however, we will define the Gauss-Epple
action as an action of $B_{n}$ on $[n]\times\mathbb{Z}$ instead of on
$\mathbb{Z}[i]$. Our reasons for doing so should become clear shortly. We also
view the Gauss-Epple action as a group homomorphism from $B_{n}$ to
$\operatorname{Sym}([n]\times\mathbb{Z})$. (This is a specific instance of
.the classical trick to convert between group actions $G\times X\to X$ and
group homomorphisms $G\to(X\to X)$: by currying the input.)

As defined by Epple (based on Gauss’s notation), the Gauss-Epple homomorphism
is given as follows:

######  Definition 2.1 (Gauss-Epple action).

For any $n\in\mathbb{N}$, the Gauss-Epple action
$\alpha:B_{n}\times\mathbb{Z}[i]\to\mathbb{Z}[i]$ is the unique left group
action of $B_{n}$ (with canonical Artin generators
$\sigma_{1},\sigma_{2},\dots$) on the Gaussian integers $\mathbb{Z}[i]$
defined by the following generating relation:

$\alpha(\sigma_{k},z):=\begin{cases}z&\Re(z)\notin\\{k,k+1\\}\\\
z+1&\Re(z)=k\\\ z-1+i&\Re(z)=k+1\end{cases}.$

We simplify this definition by replacing each complex number involved with the
ordered pair of its real and complex parts, and then restricting the first
component to the elements of $[n]$. This notational change yields an action on
the elements of $[n]\times\mathbb{Z}$. Therefore, as we define it, the Gauss-
Epple action is given as follows:

######  Definition 2.2 (Gauss-Epple action).

For any $n\in\mathbb{N}$, the Gauss-Epple action
$GE:B_{n}\times([n]\times\mathbb{Z})\to[n]\times\mathbb{Z}$ is the unique left
group action of $B_{n}$ (with canonical Artin generators
$\sigma_{1},\sigma_{2},\dots$) on $[n]\times\mathbb{Z}$ defined by the
following generating relation:

$GE(\sigma_{k},(a,b)):=\begin{cases}(a,b)&a\notin\\{k,k+1\\}\\\ (k+1,b)&a=k\\\
(k,b+1)&a=k+1\end{cases}.$

######  Remark 2.1.

Note that this definition implies that the inverses of the Artin generators
act as follows:

$GE(\sigma_{k}^{-1},(a,b))=\begin{cases}(a,b)&a\notin\\{k,k+1\\}\\\
(k+1,b-1)&a=k\\\ (k,b)&a=k+1\end{cases}.$

We now prove that the Gauss-Epple action is well-defined, a fact stated
without proof by Epple [7].

######  Lemma 2.2.

The Gauss-Epple action $GE$ is uniquely defined by Definition 2.2 as a left
group action of $B_{n}$ on $\mathbb{Z}^{2}$.

###### Proof.

Since the Artin generators satisfy the generating relations
$\sigma_{k}\sigma_{l}=\sigma_{l}\sigma_{k}$ for all $k,l$ such that $|k-l|\geq
2$ and $\sigma_{k}\sigma_{k+1}\sigma_{k}=\sigma_{k+1}\sigma_{k}\sigma_{k+1}$
for all $k\in[n-1]$, it is enough to verify that that the analogous relations
hold for the Gauss-Epple action to prove Lemma 2.2.

Suppose that $k,l$ are two integers satisfying $|k-l|\geq 2$. We observe that
$\\{k,k+1\\}\cap\\{l,l+1\\}=\varnothing$, which will become important later.
Let $a,b$ be arbitrary members of $\mathbb{Z}$. We prove that, for all
integers $k,l$ satisfying $|k-l|\geq 2$, we have
$GE(\sigma_{k})GE(\sigma_{l})=GE(\sigma_{l})GE(\sigma_{k})$, as follows:

  * •

If $a\notin\\{k,k+1,l,l+1\\}$, then
$\sigma_{k}\sigma_{l}(a,b)=(a,b)=\sigma_{l}\sigma_{k}(a,b)$.

  * •

If $a\in\\{k,k+1\\}$, then
$\sigma_{k}\sigma_{l}(a,b)=\sigma_{k}(a,b)=\sigma_{l}\sigma_{k}(a,b)$. (From
the second to third step, we use the fact that the first component of
$\sigma_{k}(a,b)$ is in $\\{k,k+1\\}$, and so outside of $\\{l,l+1\\}$.)

  * •

If $a\in\\{l,l+1\\}$, then
$\sigma_{k}\sigma_{l}(a,b)=\sigma_{l}(a,b)=\sigma_{l}\sigma_{k}(a,b)$.

Again, suppose that $k,a,b$ are arbitrary integers satisfying
$k\in\\{1,\dots,n-2\\}$. Then we prove that we have
$GE(\sigma_{k})GE(\sigma_{k+1})GE(\sigma_{k})=GE(\sigma_{k+1})GE(\sigma_{k})GE(\sigma_{k+1})$
as follows:

  * •

If $a\notin\\{k,k+1,k+2\\}$, then
$\sigma_{k}\sigma_{k+1}\sigma_{k}(a,b)=(a,b)=\sigma_{k+1}\sigma_{k}\sigma_{k+1}(a,b)$.

  * •

If $a=k$, then
$\sigma_{k}\sigma_{k+1}\sigma_{k}(a,b)=\sigma_{k}\sigma_{k+1}(k+1,b)=\sigma_{k}(k+2,b)=(k+2,b)$,
and
$\sigma_{k+1}\sigma_{k}\sigma_{k+1}(a,b)=\sigma_{k+1}\sigma_{k}(k,b)=\sigma_{k+1}(k+1,b)=(k+2,b)$.

  * •

If $a=k+1$, then
$\sigma_{k}\sigma_{k+1}\sigma_{k}(a,b)=\sigma_{k}\sigma_{k+1}(k,b+1)=\sigma_{k}(k,b+1)=(k+1,b+1)$,
and
$\sigma_{k+1}\sigma_{k}\sigma_{k+1}(a,b)=\sigma_{k+1}\sigma_{k}(k+2,b)=\sigma_{k+1}(k+2,b)=(k+1,b+1)$.

  * •

If $a=k+2$, then
$\sigma_{k}\sigma_{k+1}\sigma_{k}(a,b)=\sigma_{k}\sigma_{k+1}(k+2,b)=\sigma_{k}(k+1,b+1)=(k,b+2)$,
and
$\sigma_{k+1}\sigma_{k}\sigma_{k+1}(a,b)=\sigma_{k+1}\sigma_{k}(k+1,b+1)=\sigma_{k+1}(k,b+2)=(k,b+2)$.

∎

From now on, we consider GE as a group homomorphism
$GE:B_{n}\to\operatorname{Sym}([n]\times\mathbb{Z})$

######  Remark 2.3.

Note that, by letting $n$ vary, we can extend the Gauss-Epple homomorphism
$GE:B_{n}\to\operatorname{Sym}([n]\times\mathbb{Z})$ to a homomorphism
$GE:B_{\infty}\to\operatorname{Sym}(\\{1,2,\dots\\}\times\mathbb{Z})$, where
the group $B_{\infty}$ is defined to be the direct limit of the groups
$B_{n}$. However, we do not pursue this direction.

###  2.1 Image of WP

Firstly, note the following classical fact, which motivates the study of
images of group homomorphisms:

######  Theorem 2.4 (First Isomorphism Theorem).

Let $\phi:G\to H$ be a group homomorphism. Then $\operatorname{im}(\phi)\cong
G/\ker\phi$.

For the group homomorphisms we study, it turns out that the images are much
simpler than the kernels. Hence we will completely determine the images but
only briefly discuss the kernels.

Since the Gauss-Epple action permutes the $a$ part without respect to the $b$
part according to the permutation of the braid, it is evident that braids in
the kernel of the Gauss-Epple action have the identity permutation (i.e., are
pure). Early on, however, we discovered that braids in the kernel of the
Gauss-Epple homomorphism have zero writhe too (we will prove this in Lemma
2.7). Hence we decided to study the invariant $WP$ (which stands for “writhe-
permutation”), a weaker invariant than the Gauss-Epple homomorphism. This
invariant, too, turns out to be a group homomorphism: in this case, with a
range of $\mathbb{Z}\times S_{n}$. In fact, we find that the image of $WP$ is
an order 2 subgroup of the range:

######  Theorem 2.5 (Structure of imWP).

There is a braid on $n$ strands with writhe $w\in\mathbb{Z}$ and permutation
$\pi\in S_{n}$ iff (if and only if) $\pi$ and $w$ have the same parity.

###### Proof.

Necessity: Suppose that there is a braid on $n$ strands with writhe
$w\in\mathbb{Z}$ and permutation $\pi\in S_{n}$. If the number of twists of
the braid is even, both $\pi$ and $w$ will also necessarily be even.
Similarly, if the number of twists of the braid is odd, both $\pi$ and $w$
will necessarily be odd. Thus the claim holds.

Sufficiency: Let $\pi$ and $w$ be arbitrary permutations and integers with the
same parity. Since $S_{n}$ is generated by $(1,2),(2,3),\dots,(n-1,n)$, we can
write $\pi=\prod_{i=1}^{f}t_{a_{i}}$, where $t_{i}=(i,i+1)$ and $a_{i}$ is
some arbitrary sequence of integers.

Consider the braid $\beta^{\prime}=\prod_{i=1}^{f}\sigma_{a_{i}}$, where
$a_{i}$ is as before. Trivially, $\beta^{\prime}$ has permutation $\pi$. By
the argument in the previous part of the proof, the writhe of $\beta^{\prime}$
has the same parity as $\pi$, which has the same parity as $w$, so that
$\sigma_{1}^{w-|\beta^{\prime}|}$ is pure. We conclude that the braid
$\beta=\beta^{\prime}\sigma_{1}^{w-|\beta^{\prime}|}$ has writhe $w$ and
permutation $\pi$, as desired, and we are done. ∎

It is clear that since the image of $WP$ is not $B_{n}$ (nor is it the image
of $GE$), that $WP$ must have a nontrivial kernel, and moreover, an element
lying outside of the kernel of $GE$. We can make this explicit by giving the
example $\sigma_{1}^{-2}\sigma_{2}^{2}$.

###  2.2 Image of GE

The computation of the image of the Gauss-Epple homomorphism (henceforth
abbreviated imGE) is similar, albeit much more tedious. Firstly, we prove that
the Gauss-Epple action factors through $\mathbb{Z}^{n}\rtimes S_{n}$. (The
group rule of $\mathbb{Z}^{n}\rtimes S_{n}$ is
$(\pi,\ell)(\pi^{\prime},\ell^{\prime})=(\pi\pi^{\prime},\ell^{\prime}+\pi^{\prime}\ell)$.)

######  Theorem 2.6 (Structure theorem for imGE).

Define the group inclusion $\iota:\mathbb{Z}^{n}\rtimes
S_{n}\to\operatorname{Sym}([n]\times\mathbb{Z})$ by the equation
$(\pi,\ell)\to((a,b)\to(\pi(a),b+\ell(a))$, and define the group surjection
$\varsigma:B_{n}\to\mathbb{Z}^{n}\rtimes S_{n}$ by

$\varsigma(\sigma_{k}):=((k,k+1),\vec{e_{k}}).$

Then $GE=\varsigma\circ\iota$.

###### Proof.

It is trivial to verify that $GE=\varsigma\circ\iota$ for the Artin
generators. By induction, and by the fact that all three functions are group
homomorphisms, it follows that $GE=\varsigma\circ\iota$ for all braids. ∎

As an aside, we observe the Gauss-Epple action is transitive when restricted
to $[n]\times\mathbb{Z}$. However, as we can deduce from Theorem 2.6, it is
NOT doubly transitive.

We also relate imWP to imGE with the following fact (which motivates the
earlier study of imWP):

######  Lemma 2.7.

Let $\beta$ be a braid, and suppose that $\varsigma(\beta)=(\pi,\ell)$ (where
$\varsigma$ is defined as in Theorem 2.6). Then the sum of the components of
$\ell$ is the writhe of $\beta$.

###### Proof.

Note that $\varkappa:\mathbb{Z}^{n}\rtimes S_{n}\to\mathbb{Z}$ defined by
$\varkappa((\pi,\ell)):=\sum_{i}\ell_{i}$ is a group homomorphism. We know
that $\varkappa$ is a group homomorphism since, for any
$\pi,\ell,\pi^{\prime},\ell^{\prime}$, we have

$\displaystyle\varkappa((\pi,\ell)(\pi^{\prime},\ell^{\prime}))$
$\displaystyle=\varkappa((\pi\pi^{\prime},\ell\pi^{\prime}+\ell^{\prime}))$
$\displaystyle=\sum_{i}(\ell(\pi^{\prime}(i))+\ell^{\prime}(i))$
$\displaystyle=\sum_{i}\ell(\pi^{\prime}(i))+\sum_{i}\ell^{\prime}(i)$
$\displaystyle=\sum_{i}\ell(i)+\sum_{i}\ell^{\prime}(i)$
$\displaystyle=\varkappa((\pi,\ell))+\varkappa((\pi^{\prime},\ell^{\prime})),$

as desired.

Hence $\varkappa\circ\varsigma$ is a group homomorphism from $B_{n}$ to
$\mathbb{Z}$.

We also note that writhe is a group homomorphism from $B_{n}$ to $\mathbb{Z}$.
Furthermore, it is easy to verify that
$\varkappa\circ\varsigma(\sigma_{k})=|\sigma_{k}|=1$. Since
$\varkappa\circ\varsigma$ and writhe agree on the generators of $B_{n}$, they
must agree everywhere by induction, and so be the same. ∎

To fully characterize imGE, we begin with the special case of pure braids.

######  Lemma 2.8.

For any $\ell\in\mathbb{Z}^{n}$, there is a pure braid $\beta$ with
$\ell_{\beta}=\ell$ iff $\ell$ has even sum.

###### Proof.

Necessity: Suppose that there exists such a pure braid $\beta$. Then
$\sum_{a}\ell_{a}=|\beta|$, and the latter is even since $\beta$ is pure.
Hence $\sum_{a}\ell_{a}$ must be pure, and we are done.

Sufficiency: Let $\beta_{1}$ and $\beta_{2}$ be arbitrary pure braids. Then
$\ell_{\beta_{1}\beta_{2}}=\ell_{\beta_{1}}+\ell_{\beta_{2}}$, and
$\ell_{\beta_{1}^{-1}}=-\ell_{\beta_{1}}$. Hence the set of all possible
$\ell_{\beta}$ values for braids $\beta$ forms a lattice in $\mathbb{Z}^{n}$.

We compute that $\ell_{\sigma_{k}^{2}}=(0,\dots,0,1,1,0,\dots,0)$, where the
$1,1$ is in the $k$th and $(k+1)$th places and where we have identified $\ell$
with a vector in $\mathbb{Z}^{n}$. Similarly, we compute
$\ell_{(\sigma_{k}\sigma_{k+1})^{3}}=(0,\dots,0,2,2,2,0,\dots,0)$.

Since $(a,a+b-2c,c)=(a-2c)(1,1,0)+(b-2c)(0,1,1)+c(2,2,2)$, we conclude that
these vectors span the lattice of all elements of $\mathbb{Z}^{n}$ with even
sum, and we are done. ∎

Then, we will combine this result with some further lemmas to characterize the
image in Theorem 2.9. Now we can fully characterize imGE.

######  Theorem 2.9.

Let $\pi\in S_{n}$ and $\ell\in\mathbb{Z}^{n}$. Then there exists a braid
$\beta$ such that $\pi_{\beta}=\pi,\ell_{\beta}=\ell$ iff the sum of $\ell$
has the same parity as $\pi$.

###### Proof.

Necessity: Suppose such a braid exists. By Lemma 2.7, the sum of
$\ell=\ell_{\beta}$ must be the writhe of $\beta$. This must have the same
parity as $\pi_{\beta}=\pi$ by Theorem 2.5, and we are done.

Sufficiency: Suppose that the sum of $\ell$ has the same parity as $\pi$. We
can construct a braid $\beta_{1}$ with permutation $\pi$. By Theorem 2.5 and
Lemma 2.7, $\ell_{\beta_{1}}$ must have a sum with the same parity as $\pi$.
By Lemma 2.8, there is a pure braid $\beta_{2}$ such that
$\ell_{\beta_{2}}=\ell-\ell_{\beta_{1}}$ (since the right hand side has even
sum). Then the braid $\beta_{3}:=\beta_{1}\beta_{2}$ has permutation $\pi$ and
vector $\ell$, as desired, and we are done. ∎

###  2.3 Kernel of GE

It is natural to consider the kernel of the Gauss-Epple homomorphism. We find
that imWP is a quotient group of imGE, as kerWP is a supergroup of kerGE.

Since the image of GE is not $B_{n}$, the kernel of GE must necessarily be
nontrivial. In fact, by computational means, we found several explicit
examples:

  * •

$\sigma_{1}^{2}\sigma_{2}^{2}\sigma_{1}^{-2}\sigma_{2}^{-2}$ (which
corresponds to the Whitehead link [15]);

  * •

$\sigma_{1}^{-1}\sigma_{3}^{-1}\sigma_{2}^{2}\sigma_{3}^{-1}\sigma_{1}^{-1}\sigma_{2}^{2}$;

  * •

$(\sigma_{1}\sigma_{2}^{-1})^{3}$;

  * •

$(\sigma_{2}\sigma_{1}^{-1})^{3}$;

  * •

$\sigma_{1}\sigma_{2}^{-1}\sigma_{1}^{2}(\sigma_{1}\sigma_{2}^{-1})^{2}\sigma_{2}^{-2}$;

  * •

$(\sigma_{1}\sigma_{2}\sigma_{1}^{2}\sigma_{2}^{-1})^{2}\sigma_{1}^{-2}$.

We can make several observations about the structure of kerGE: We know that
the kernel of the Gauss-Epple action must be a subgroup of the group of pure
braids, which in turn is a subgroup of the braid group $B_{n}$. Since $B_{n}$
has no torsion, the Gauss-Epple kernel must also have no torsion and be
infinite.

####  2.3.1 Random braids

We study the probability that a random braid of $n$ generators lies in the
kernel of the Gauss-Epple action.

Let $G$ be an arbitrary group and $H$ an arbitrary normal subgroup. Then, for
all $g\in G$, we know that $g\in H$ iff $\phi(g)=e$, where $\phi$ is the
quotient map $G\to G/H$. (This effectively reduces our problem to studying the
quotient group $K:=G/H$.) Now, defining $V(N)$ to be the number of elements of
$K$ that can be produced from words of length $N$, we have the following deep
theorem:

######  Theorem 2.10.

[1, 12, 13] For any $d$, the probability of a random walk on $K$ returning to
the identity on the $N$th step is on the order of $N^{-d/2}$ iff $V(N)$ is
comparable to $N^{d}$.

To apply this theorem, we take $G=B_{n},H=\ker GE,K=\operatorname{im}GE$.
Since $V(N)$ is comparable to $N^{n}$ thanks to Theorem 2.9, we conclude by
Theorem 2.10 that the probability without filtering of an element being in the
kernel of Gauss-Epple is asymptotically on the order of $N^{-n/2}$, which is
the result we seek.

##  3 The symmetric Gauss-Epple homomorphism

In his paper [7], Epple introduced yet another action of the braid group,
which he called the symmetric Gauss-Epple homomorphism. This object is defined
as follows:

######  Definition 3.1 (Symmetric Gauss-Epple action).

For any $n\in\mathbb{N}$, the symmetric Gauss-Epple action
$symGE:B_{n}\times\mathbb{Z}^{2}\to\mathbb{Z}^{2}$ is the unique left group
action of $B_{n}$ (with canonical Artin generators
$\sigma_{1},\sigma_{2},\dots$) on $\mathbb{Z}^{2}$ defined by the following
relation:

$\forall
k\in[n],a,b\in\mathbb{Z}:\operatorname{symGE}(\sigma_{k},(a,b))=\begin{cases}(a,b)&a\notin\\{k,k+1\\}\\\
(k+1,b+1)&a=k\\\ (k,b+1)&a=k+1\end{cases}.$

For similar reasons as in the proof of Theorem 2.6, the symmetric Gauss-Epple
action (which we abbreviate symGE) also factors through $\mathbb{Z}^{n}\rtimes
S_{n}$ into $\operatorname{Sym}(\mathbb{Z}^{2})$:

######  Theorem 3.1 (Structure theorem for imGE).

Define the group inclusion $\iota:\mathbb{Z}^{n}\rtimes
S_{n}\to\operatorname{Sym}([n]\times\mathbb{Z})$ by the equation
$(\pi,\ell)\to((a,b)\to(\pi(a),b+\ell(a))$, and define the group surjection
$\varpi:B_{n}\to\mathbb{Z}^{n}\rtimes S_{n}$ by

$\varpi(\sigma_{k}):=(\vec{e_{k}}+\vec{e_{k+1}},(k,k+1)).$

Then $symGE=\iota\circ\varpi$.

###### Proof.

Trivial. ∎

We then note that the symmetric Gauss-Epple action and the Gauss-Epple action
have the same kernel:

######  Theorem 3.2.

The kernel of the symmetric Gauss-Epple action is the same as the kernel of
the ordinary Gauss-Epple action.

###### Proof.

Suppose $\beta$ is an arbitrary pure braid. We show that
$GE(\beta)=(\ell,\text{id})\leftrightarrow\operatorname{symGE}(\beta)=(2\ell,\text{id})$.
It is clear that the “permutation parts” are all identity, so we only focus on
the “vector parts”. It suffices to then show this claim for the the generators
of the pure braid group $P_{n}$. Note that $P_{n}$ (the pure braid group on
$n$ strands) is generated by
$A_{i,j}:=\sigma_{j-1}\dots\sigma_{i+1}\sigma_{i}^{2}\sigma_{i+1}^{-1}\dots\sigma_{j-1}^{-1}$
[14]; for example, the generators for $P_{3}$ are
$\sigma_{1}^{2},\sigma_{2}^{2},\sigma_{1}\sigma_{2}^{2}\sigma_{1}^{-1}.$

We compute iteratively that
$GE(A_{i,j})=(\vec{e_{i}}+\vec{e_{j}},\text{id}),\operatorname{symGE}(A_{i,j})=(2\vec{e_{i}}+2\vec{e_{j}},\text{id})$.
Hence the claim holds, and we are done. ∎

######  Remark 3.3.

By the First Isomorphism Theorem, the previous theorem implies that the images
of the symmetric Gauss-Epple action and the images of the regular Gauss-Epple
action must be isomorphic as groups.

##  4 The super-Gauss-Epple homomorphism

We define another homomorphism, this time of type signature
$B_{n}\to\mathbb{Z}^{n(n-1)}\rtimes S_{n}$, which we name the super-Gauss-
Epple homomorphism (abbreviated SGE). More precisely, define $O_{i,j}$ to be
the matrix with a 1 entry at the $(i,j)$th place and 0 entries everywhere
else. Then we define SGE as follows:

######  Definition 4.1 (super-Gauss-Epple homomorphism).

The super-Gauss-Epple homomorphism $SGE:B_{n}\to\mathbb{Z}^{n(n-1)}\rtimes
S_{n}$ is defined by the following equation:

$SGE(\sigma_{i}):=(O_{i,i+1},(i,i+1)).$

(Here, by slight abuse of notation, we consider elements of
$\mathbb{Z}^{n(n-1)}$ to be $n\times n$ matrices with all zeros along the
diagonal.) An example calculation of SGE is depicted in Figure 9.

Figure 9: An example calculation of SGE.

To verify that this is indeed a homomorphism, we verify the braid relations
(as is sufficient and necessary):

$\displaystyle SGE(\sigma_{i}\sigma_{i+1}\sigma_{i})$
$\displaystyle=SGE(\sigma_{i})\cdot SGE(\sigma_{i+1})\cdot SGE(\sigma_{i})$
$\displaystyle=(O_{i,i+1},(i,i+1))\cdot(O_{i+1,i+2},(i+1,i+2))\cdot(O_{i,i+1},(i,i+1))$
$\displaystyle=(O_{i,i+1}+(i,i+1)\cdot O_{i+1,i+2}+(i,i+1)(i+1,i+2)\cdot
O_{i,i+1},$ $\displaystyle\hskip 15.0pt(i,i+1)(i+1,i+2)(i,i+1))$
$\displaystyle=(O_{i,i+1}+O_{i,i+2}+O_{i+1,i+2},(i,i+1)(i+1,i+2)(i,i+1))$
$\displaystyle=(O_{i,i+1}+O_{i,i+2}+O_{i+1,i+2},(i,i+2,i+1)),$

and

$\displaystyle SGE(\sigma_{i+1}\sigma_{i}\sigma_{i+1})$
$\displaystyle=SGE(\sigma_{i+1})SGE(\sigma_{i})SGE(\sigma_{i+1})$
$\displaystyle=(O_{i+1,i+2},(i+1,i+2))\cdot(O_{i,i+1},(i,i+1))\cdot(O_{i+1,i+2},(i+1,i+2))$
$\displaystyle=(O_{i+1,i+2}+(i+1,i+2)\cdot O_{i,i+1}+(i+1,i+2)(i,i+1)\cdot
O_{i+1,i+2},$ $\displaystyle\hskip 15.0pt(i+1,i+2)(i,i+1)(i+1,i+2))$
$\displaystyle=(O_{i,i+1}+O_{i,i+2}+O_{i+1,i+2},$ $\displaystyle\hskip
15.0pt(i,i+1)(i+1,i+2)(i,i+1))$
$\displaystyle=(O_{i,i+1}+O_{i,i+2}+O_{i+1,i+2},(i,i+2,i+1)),$

as desired.

A braid’s image under the super-Gauss-Epple homomorphism stores at least as
much information about it as the ordinary Gauss-Epple homomorphism. Note that
this is strictly more information, as we can give the explicit example
$\sigma_{2}^{2}\sigma_{3}^{-1}\sigma_{2}\sigma_{1}^{2}\sigma_{2}^{-1}\sigma_{3}^{-1}\sigma_{1}^{-2}$.

(By the Third Isomorphism Theorem, this also follows iff the image of the
super-Gauss-Epple homomorphism is not isomorphic to that of the ordinary
Gauss-Epple homomorphism.)

It is to be noted that the kernel of SGE is still nontrivial. For example, it
contains the braid $(\sigma_{1}\sigma_{2}^{-1})^{3}$, which was mentioned
before as being a nontrivial example of the kernel of regular GE.

Hence, we have the following commutative diagram:

${{B_{n}/[P_{n},P_{n}]}}$${{B_{n}}}$${{P_{n}^{ab}\rtimes
S_{n}}}$${{\operatorname{Sym}(\mathbb{Z}\times\binom{n}{2})}}$${{\mathbb{Z}^{n}\rtimes
S_{n}}}$${{\operatorname{Sym}(\mathbb{Z}\times[n])}}$${{\mathbb{Z}\times
S_{n}}}$$\scriptstyle{UGE}$$\scriptstyle{SGE}$$\scriptstyle{GE}$$\scriptstyle{WP}$

###  4.1 A 1-cocycle

Firstly, note that the matrix of (the super-Gauss-Epple homomorphism of) a
pure braid is always symmetric:

######  Lemma 4.1.

Let $\beta\in P_{n}$, and suppose that $SGE(\beta)=(M,\text{id})$. Then $M$ is
a symmetric matrix.

###### Proof.

Recall that $P_{n}$ (the pure braid group on $n$ strands) is generated by

$A_{i,j}:=\sigma_{j-1}\dots\sigma_{i+1}\sigma_{i}^{2}\sigma_{i+1}^{-1}\dots\sigma_{j-1}^{-1}$

[14]; for example, the generators for $P_{3}$ are
$\sigma_{1}^{2},\sigma_{2}^{2},\sigma_{1}\sigma_{2}^{2}\sigma_{1}^{-1}.$

Note that $SGE(A_{i,j})=(O_{i,j}+O_{j,i},id)$, and the right component is
clearly symmetric. Since SGE matrices add for pure braids, the conclusion
holds by induction. ∎

Hence, the difference between the upper half and the (transposed) lower half
is always only determined by the braid permutation.

Now, let $F:B_{n}\to M_{n,n}(\mathbb{Z})$ be defined so that
$SGE(\beta)=(M,s_{\beta})\implies F(\beta)=M-M^{T}$. Then
$F(\beta)=0\forall\beta\in P_{n}$. Furthermore, $F$ satisfies the equality
$F(\beta\gamma)=F(\beta)+M_{s_{\beta}}F(\gamma)M_{s_{\beta}}^{-1}$. Hence
$F(\pi\beta)=F(\beta)+id\cdot F(\beta)=F(\beta)~{}\forall\beta\in B_{n},\pi\in
P_{n}$, so that the value of $F(\beta)$ only depends on the permutation of the
braid $\beta$, and that in general $F$ projects down to a function
$\overline{F}:S_{n}\to M_{n,n}(\mathbb{Z})$. Furthermore, we have
$\overline{F}(\beta\gamma)=F(\beta)+M_{s_{\beta}}F(\gamma)M_{s_{\beta}}^{-1}$
(which we shall abbreviate as
$\overline{F}(\beta\gamma)=\overline{F}(\beta)+\beta\cdot\overline{F}(\gamma)$.
Therefore, $\overline{F}$ is a 1-cocycle of the action of $S_{n}$ on
$M_{n,n}(\mathbb{Z})$, and in fact of the action on the set of antisymmetric
$n\times n$ matrices over the integers.

While computing values of $\overline{F}$ for some random braids, we noticed
that the entries in the upper half were always in $\\{0,1\\}$. We prove this
inductively as follows:

######  Theorem 4.2.

Let $\varrho$ be an arbitrary permutation, and $i,j$ be integers with $i>j$.
Then $\overline{F}(\varrho)[i,j]\in\\{0,1\\}$.

###### Proof.

We assume this for some $\varrho$ and prove it for $\tau\varrho$, where
$\tau:=(\iota,\iota+1)$ is a transposition.

Since
$\overline{F}(\tau\varrho)=\overline{F}(\tau)+\tau\cdot\overline{F}(\varrho)t$,
the conjecture is instantly verified for all $i,j$ except for $\iota,\iota+1$
(since the first term is trivially zero and the second term is in $\\{0,1\\}$
by the inductive hypothesis). In the special case $i=\iota,j=\iota+1$, we
compute

$\displaystyle\overline{F}(\tau\varrho)[\iota,\iota+1]$
$\displaystyle=\overline{F}(\tau)[\iota,\iota+1]+(\tau\cdot\overline{F}(\varrho)[\iota,\iota]$
$\displaystyle=1-\overline{F}(\varrho)[\iota,\iota+1]$
$\displaystyle\in\\{0,1\\}.$

(Here, we use the inductive hypothesis
$\overline{F}(\varrho)[\iota,\iota+1]\in\\{0,1\\}$.) Hence the claim holds.

By induction on the minimum number of simple transpositions required to be
multiplied together to represent an arbitrary permutation, we are then done. ∎

This means that the image of the super-Gauss-Epple homomorphism is a group
with $\mathbb{Z}^{\frac{n(n-1)}{2}}$ as a normal subgroup and $S_{n}$ as the
quotient group. Note that it is not a semidirect product, since no braid with
permutation $(1,2)$ has a square in the kernel of the super-Gauss-Epple
homomorphism. (Indeed, suppose such a braid $\beta=\sigma_{1}\varpi$ existed,
and let $SGE(\beta)=(M,(1,2))$ for some matrix $M$. Then the matrix of
$SGE(\beta^{2})$ is $M+(1,2)\cdot M$. But since $M[1,2]$ is one more than
$((1,2)\cdot M)[1,2]=M[2,1]$ and both are integers, the two cannot sum to
zero, which is a contradiction.)

##  5 Artin groups of crystallographic type

In this section, we explore generalizations of the Gauss-Epple homomorphism to
the setting of Artin groups, which naturally project to corresponding Coxeter
groups (generalizing how the braid group projects to the symmetric group).

###  5.1 Special cases

####  5.1.1 The $I_{2}(4),I_{2}(6)$ cases

This are special cases of the dihedral Artin groups, which are the Artin
groups of type $I_{2}(2n)$ for integers $n$. The abelianization is
$\mathbb{Z}^{2}$ and the Coxeter group is $D_{2n}$. Hence, the generators map
to $D_{2n}\times\mathbb{Z}^{2}$ as follows:

generator | $D_{2n}\times\mathbb{Z}^{2}$
---|---
$a$ | $(s,(1,0))$
$b$ | $(sr,(0,1))$

For GE-like purposes, we may consider $D_{2n}$ as a subset of $S_{n}$.

For the case $n=4$, with $a\to(13),b\to(01)(23)$ in permutation notation, this
gives rise to the following system of equations:

$\begin{cases}\ell_{a,1}+\ell_{b,2}+\ell_{a,2}+\ell_{b,3}&=\ell_{b,1}+\ell_{a,1}+\ell_{b,4}+\ell_{a,4}\\\
\ell_{a,2}+\ell_{b,1}+\ell_{a,3}+\ell_{b,2}&=\ell_{b,2}+\ell_{a,4}+\ell_{b,1}+\ell_{a,3}\\\
\ell_{a,3}+\ell_{b,4}+\ell_{a,4}+\ell_{b,1}&=\ell_{b,3}+\ell_{a,3}+\ell_{b,2}+\ell_{a,2}\\\
\ell_{a,4}+\ell_{b,3}+\ell_{a,1}+\ell_{b,4}&=\ell_{b,4}+\ell_{a,2}+\ell_{b,3}+\ell_{a,1}\\\
\end{cases}$

(We used a computer program to generate this system of equations.)

Solving this system manually reveals a six-dimensional space:

$\ell_{a}=(\ell_{a,1},\ell_{a,2},\ell_{a,3},\ell_{a,2}),\ell_{b}=(\ell_{b,1},\ell_{b,2},\ell_{b,3},\ell_{b,2}+\ell_{b,3}-\ell_{b,1}).$

Similarly, for the case $n=6$ (also denoted $G_{2}$), we make the ansatz
$a\to(26)(35),b\to(12)(36)(45)$, based on an action of the Coxeter group on
the hexagon. With the map
$a\to\ell_{a}:=(\ell_{a,1},\ell_{a,1},\dots,\ell_{a,6}),b\to\ell_{b}:=(\ell_{b,1},\dots,\ell_{b,6})$,
this produces the following system of linear equations:

$\begin{cases}\ell_{a,1}+\ell_{b,1}+\ell_{a,2}+\ell_{b,6}+\ell_{a,3}+\ell_{b,5}&=\ell_{b,1}+\ell_{a,2}+\ell_{b,6}+\ell_{a,3}+\ell_{b,5}+\ell_{a,4}\\\
\ell_{a,2}+\ell_{b,6}+\ell_{a,3}+\ell_{b,5}+\ell_{a,4}+\ell_{b,4}&=\ell_{b,2}+\ell_{a,1}+\ell_{b,1}+\ell_{a,2}+\ell_{b,6}+\ell_{a,3}\\\
\ell_{a,3}+\ell_{b,5}+\ell_{a,4}+\ell_{b,4}+\ell_{a,5}+\ell_{b,3}&=\ell_{b,3}+\ell_{a,6}+\ell_{b,2}+\ell_{a,1}+\ell_{b,1}+\ell_{a,2}\\\
\ell_{a,4}+\ell_{b,4}+\ell_{a,5}+\ell_{b,3}+\ell_{a,6}+\ell_{b,2}&=\ell_{b,4}+\ell_{a,5}+\ell_{b,3}+\ell_{a,6}+\ell_{b,2}+\ell_{a,1}\\\
\ell_{a,5}+\ell_{b,3}+\ell_{a,6}+\ell_{b,2}+\ell_{a,1}+\ell_{b,1}&=\ell_{b,5}+\ell_{a,4}+\ell_{b,4}+\ell_{a,5}+\ell_{b,3}+\ell_{a,6}\\\
\ell_{a,6}+\ell_{b,2}+\ell_{a,1}+\ell_{b,1}+\ell_{a,2}+\ell_{b,6}&=\ell_{b,6}+\ell_{a,3}+\ell_{b,5}+\ell_{a,4}+\ell_{b,4}+\ell_{a,5}\\\
\end{cases}$

This system is highly symmetric and many of the variables cancel. Therefore,
we can solve it exactly, producing the following linear parametrization of the
entire solution space over the integers, which is nine-dimensional:

$\ell_{a}=(a_{1},a_{2},a_{3},a_{1},a_{2}+x_{2},a_{3}+x_{2}),\ell_{b}=(b_{1},b_{2},b_{3},b_{1}+y_{1},b_{2}-y_{1},b_{3}+y_{3}),$

where $a_{1},a_{2},a_{3},b_{1},b_{2},b_{3},x_{2},y_{1},y_{3}$ range
independently over $\mathbb{Z}$.

####  5.1.2 The $B_{n}$ case

The isomorphism between the Coxeter group of this case and $C_{2}\wr
S_{n}=C_{2}^{n}\rtimes S_{n}$ can be tabulated in generators (where we list
the generators of the Coxeter graph as $a_{1},a_{2},\dots,a_{n}$) as follows:

Coxeter | $S_{n}$ | $C_{2}^{n}$
---|---|---
$a_{1}$ | id | $(1,0,\dots,0)$
$a_{2}$ | (1, 2) | $(0,0,\dots,0)$
$a_{3}$ | (2, 3) | $(0,0,\dots,0)$
⋮ | ⋮ | ⋮
$a_{n}$ | ($n-1$, $n$) | $(0,0,\dots,0)$

In the classical case of the braid group $B_{3}$, the space of possible
“linking vector” analogues is four-dimensional, and spanned by “symmetric GE”
($\sigma_{1}\mapsto(1,1,0),\sigma_{2}\mapsto(0,1,1)$), “row GE”
($\sigma_{1}\mapsto(0,0,1),\sigma_{2}\mapsto(1,0,0)$), “zero-ish GE 1”
($\sigma_{1}\mapsto(1,-1,0),\sigma_{2}\mapsto 0$), and “zero-ish GE 2”
($\sigma_{1}\mapsto 0,\sigma_{2}\mapsto(0,1,-1)$). We can derive this by
noting that if we have
$\sigma_{1}\mapsto(a_{1},b_{1},c_{1}),\sigma_{2}\mapsto(a_{2},b_{2},c_{2})$,
then the single braiding relation
$\sigma_{1}\sigma_{2}\sigma_{1}=\sigma_{2}\sigma_{1}\sigma_{2}$ gives us
$c_{1}=a_{2}$ and $a_{1}+b_{1}=b_{2}+c_{2}$ as constraints after
simplification; some elementary algebra then gives us the four basis vectors.

As for the images of the corresponding homomorphisms of $B_{3}$ over the pure
braids only, the image of symmetric GE is the set of vectors of 3 even
integers which sum to 0 modulo 4, the image of row GE is just the set of
vectors of 3 even integers, and the images of the two “zero-ish” analogs are
just all zero.

As for higher braid groups $B_{n}$, the “far commutativity relations”
$\sigma_{i}\sigma_{j}=\sigma_{j}\sigma_{i}$ for $|i-j|\geq 2$ basically
constrain all the entries of the “linking vector” corresponding to
$\sigma_{i}$ (which I’ll call $\ell_{i}$ by analogy) right of the
$(i+1)^{\text{th}}$ to be the same within, and the entries of $\ell_{i}$ left
of the $i^{\text{th}}$ to also be the same within. Combined with the braid
relations
$\sigma_{i}\sigma_{i+1}\sigma_{i}=\sigma_{i+1}\sigma_{i}\sigma_{i+1}$, we get
$\ell_{i,i+2}=\ell_{i+1,i}$ as well as
$\ell_{i,i}+\ell_{i,i+1}=\ell_{i+1,i+1}+\ell_{i+1,i+2}$. Hence the set of
possible $\ell_{i}$ combinations can be spanned by the following:

  * •

Classical symmetric GE ($\ell_{i}=e_{i}+e_{i+1}$)

  * •

Row GE for row $i_{0}$
($\ell_{i_{0}}=\sum_{i:i>i_{0}+1}e_{i},\ell_{i_{0}+1}=\sum_{i:i<i_{0}}e_{i},\ell_{i}=0$
elsewise)

  * •

Zero-ish GE for row $i_{0}$ ($\ell_{i_{0}}=e_{i_{0}}-e_{i_{0}+1},\ell_{i}=0$
elsewise)

There are also corresponding 1-cocycles. We can take linear combinations
thereof to get even more 1-cocycles since $\mathbb{Z}^{3}$ is abelian.

###  5.2 Super-Gauss-Epple analogues

Since $\mathcal{C}$ acts on $\Phi$, this induces an action of $\mathcal{C}$ on
$\mathbb{Z}^{\Phi}$. Hence, there is a homomorphism
$\mathcal{A}\to\mathbb{Z}^{\Phi}\rtimes\mathcal{C}$, defined by
$a\mapsto(\alpha_{a},\mathcal{C}(a))$, where $\mathcal{C}(a)$ is the image of
$a$ under the quotient map $\mathcal{A}\to\mathcal{C}$; by analogy, we shall
also denote it $SGE$, or $SGE_{\mathcal{A}}$ when necessary.

This is a generalization of the super-Gauss-Epple homomorphism, when
$\mathcal{A}=B_{n},\mathcal{C}=S_{n}$. Here, the root system $\Phi$ is
$\\{\vec{e_{i}}-\vec{e_{j}}|1\leq i\neq j\leq n\\}$, which has size $n(n-1)$,
and we identify the $\vec{e_{i}}-\vec{e_{j}}$ component of $\mathbb{Z}^{\Phi}$
with the $(i,j)$th component of a matrix whose diagonal elements are all zero.

We can show that this is indeed a homomorphism with geometric arguments.

######  Theorem 5.1.

There is a unique homomorphism
$\mathcal{A}\to\mathbb{Z}^{\Phi}\rtimes\mathcal{C}$, defined by
$a\mapsto(\alpha_{a},\mathcal{C}(a))$, where $\mathcal{C}(a)$ is the image of
$a$ under the quotient map $\mathcal{A}\to\mathcal{C}$.

###### Proof.

We show that the braid relations of $\mathcal{A}$ are satisfied. To do this,
we show that the $\mathbb{Z}^{\Phi}$ sides are identical. As these values are
formal linear combinations of roots (elements of $\Phi$), we also consider
them as multisets, and prove them equal accordingly. (We do not consider the
$\mathcal{C}$ sides because these are trivially equal due to the fact that
there is a bona-fide homomorphism $\mathcal{A}\to\mathcal{C}$.)

Let $a_{1}$ and $a_{2}$ be two generators of $\mathcal{A}$ for which there is
a braid relation. For notational simplicity, we will refer to the
corresponding vectors as
$\vec{v}_{1}:=\mathcal{C}(a_{1}),\vec{v}_{2}:=\mathcal{C}(a_{2})$. We shall
also introduce the notation $*$. For two arbitrary vectors $\vec{v}$ and
$\vec{w}$, we use $\vec{v}*\vec{w}$ to denote the vector that results from
reflecting $\vec{w}$ over the orthogonal hyperplane of $\vec{v}$. We will
compose it right-associatively, so that $\vec{v}*\vec{w}*\vec{x}$ shall refer
to $\vec{v}*(\vec{w}*\vec{x})$. We now perform casework depending on the
length of this braid relation:

Figure 10: Two vectors for Coxeter group generators with a braid relation of
length 2.

Length 2: We have $a_{1}a_{2}=a_{2}a_{1}$, so $\vec{v}_{1}$ and $\vec{v}_{2}$
are orthogonal, as shown in Figure 10. We find that
$\vec{v}_{1}*\vec{v}_{2}=\vec{v}_{2}$ and that
$\vec{v}_{2}*\vec{v}_{1}=\vec{v}_{1}$. Therefore, we have
$\\{\vec{v}_{1},\vec{v}_{1}*\vec{v}_{2}\\}=\\{\vec{v}_{2},\vec{v}_{2}*\vec{v}_{1}\\}$,
as desired.

Figure 11: Two vectors for Coxeter group generators with a braid relation of
length 3.

Length 3: We have $a_{1}a_{2}a_{1}=a_{2}a_{1}a_{2}$, so $\vec{v}_{1}$ and
$\vec{v}_{2}$ have an angle of $120^{\circ}$ between them, as shown in Figure
11. We find that $\vec{v}_{1}*\vec{v}_{2}=\vec{v}_{2}*\vec{v}_{1}$ (both being
the vector that bisects the angle between $\vec{v}_{1}$ and $\vec{v}_{2}$).
Furthermore, we find that $\vec{v}_{1}*(\vec{v}_{2}*\vec{v}_{1})=\vec{v}_{2}$
and $\vec{v}_{2}*(\vec{v}_{1}*\vec{v}_{2})=\vec{v}_{1}$. Therefore, we have
$\\{\vec{v}_{1},\vec{v}_{1}*\vec{v}_{2},\vec{v}_{1}*(\vec{v}_{2}*\vec{v}_{1})\\}=\\{\vec{v}_{2},\vec{v}_{2}*\vec{v}_{1},\vec{v}_{2}*(\vec{v}_{1}*\vec{v}_{2})\\}$,
as desired.

Figure 12: Two vectors for Coxeter group generators with a braid relation of
length 4.

Length 4: We have $(a_{1}a_{2})^{2}=(a_{2}a_{1})^{2}$, so $\vec{v}_{1}$ and
$\vec{v}_{2}$ have an angle of $135^{\circ}$ between them, as shown in Figure
12. We find that $\vec{v}_{1}*\vec{v}_{2}$ has an angle of $45^{\circ}$ from
$\vec{v}_{1}$ towards $\vec{v}_{2}$, and is so orthogonal to $\vec{v}_{2}$.
Similarly, $\vec{v}_{2}*\vec{v}_{1}$ has an angle of $45^{\circ}$ from
$\vec{v}_{2}$ towards $\vec{v}_{1}$, and is so orthogonal to $\vec{v}_{1}$.
Therefore, we have
$\vec{v}_{1}*(\vec{v}_{2}*\vec{v}_{1})=\vec{v}_{2}*\vec{v}_{1}$ and
$\vec{v}_{2}*(\vec{v}_{1}*\vec{v}_{2})=\vec{v}_{1}*\vec{v}_{2}$. Furthermore,
we have $\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}=\vec{v}_{2}$ and
$\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}=\vec{v}_{1}$. Therefore, we
conclude that
$\\{\vec{v}_{1},\vec{v}_{1}*\vec{v}_{2},\vec{v}_{1}*(\vec{v}_{2}*\vec{v}_{1}),\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}\\}=\\{\vec{v}_{2},\vec{v}_{2}*\vec{v}_{1},\vec{v}_{2}*(\vec{v}_{1}*\vec{v}_{2},\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}\\}$,
as desired.

Figure 13: Two vectors for Coxeter group generators with a braid relation of
length 6.

Length 6: We have $(a_{1}a_{2})^{3}=(a_{2}a_{1})^{3}$, so $\vec{v}_{1}$ and
$\vec{v}_{2}$ have an angle of $150^{\circ}$ between them, as shown in Figure
13. Divide this angle by four evenly spaced vectors, named
$\vec{w}_{1},\vec{w}_{2},\vec{w}_{3},\vec{w}_{4}$, so that the order of the
vectors is
$\vec{v}_{1},\vec{w}_{1},\vec{w}_{2},\vec{w}_{3},\vec{w}_{4},\vec{v}_{2}$
(with $30$-degree gaps between each consecutive pair). As before, we can
calculate that
$\vec{v}_{1},\vec{v}_{1}*\vec{v}_{2},\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1},\dots,\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}$
is precisely
$\vec{v}_{1},\vec{w}_{1},\vec{w}_{2},\vec{w}_{3},\vec{w}_{4},\vec{v}_{2}$, in
that order. Similarly, we can compute that
$\vec{v}_{2},\vec{v}_{2}*\vec{v}_{1},\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2},\dots,\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}$
is precisely
$\vec{v}_{2},\vec{w}_{4},\vec{w}_{3},\vec{w}_{2},\vec{w}_{1},\vec{v}_{1}$, in
that order. Hence, we have
$\\{\vec{v}_{1},\vec{v}_{1}*\vec{v}_{2},\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1},\dots,\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}\\}=\\{\vec{v}_{2},\vec{v}_{2}*\vec{v}_{1},\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2},\dots,\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}*\vec{v}_{2}*\vec{v}_{1}\\}$
as desired, and we are done. ∎

###  5.3 Another commutative diagram

We can summarize this in the following commutative diagram:

${{\mathcal{P}}}$${{\mathcal{A}}}$${{\mathcal{C}}}$${{\mathcal{P}/[\mathcal{P},\mathcal{P}]}}$${{\mathcal{A}/[\mathcal{P},\mathcal{P}]}}$$\scriptstyle{SGE}$

Here, every arrow is either an inclusion map (indicated as being injective) or
a quotient map (indicated as being surjective).

###  5.4 More 1-cocycles

Again, suppose that $\mathcal{A}$ is an Artin group of finite type,
$\mathcal{C}$ is its associated Coxeter group, $\Phi$ is an associated root
system, and $\Delta\subset\Phi$ is an associated subset of simple roots (and
let $\Delta_{a}$ be the associated simple root for any Artin generator $a$ of
$\mathcal{A}$). Define the function $F:\mathcal{A}\to\mathbb{Z}^{\Phi}$ by

$\forall a\in\mathcal{A}:SGE(a)=(\ell,\mathcal{C}(a))\implies
F(a)=\ell-\overline{\ell},$

where $\ell$ is a vector of $\mathbb{Z}^{\Phi}$ and $\overline{\ell}$ is the
image of $\ell$ under the rotation/reflection of $\mathbb{Z}^{\Phi}$ given by
$e_{\alpha}\to e_{-\alpha}\forall\alpha\in\Phi$. It is clear that $F$
satisfies the relation $F(ab)=F(a)+\mathcal{C}(a)\cdot F(b)$.

Then, we have the following lemmas (generalizing those of Section 4.1):

######  Lemma 5.2.

Let $F:\mathcal{A}\to\mathbb{Z}^{\Phi}$ be as defined for Artin groups of
finite type, and let $p\in\mathcal{P}$ (where $\mathcal{P}$ is the kernel of
the quotient map $\mathcal{A}\to\mathcal{C}$). Then $F(p)=\vec{0}$, and hence
the map $F$ descends to a map $\overline{F}:\mathcal{C}\to\mathbb{Z}^{\Phi}$

###### Proof.

Let $a$ be an Artin generator. Then $F(a^{2})=F(a)+\mathcal{C}(a)\cdot
F(a)=(\phi_{\Delta_{a}}-\phi_{-\Delta_{a}})+(\phi_{-\Delta_{a}}-\phi_{\Delta_{a}})=\vec{0}$.

Again, let $p\in\mathcal{P}$, $\upsilon$ be an Artin generator of
$\mathcal{A}$, and suppose that $F(p)=0$. Then $F(\upsilon
p\upsilon^{-1})=F(\upsilon)+\mathcal{C}(\upsilon)\cdot
F(p)+\mathcal{C}(\upsilon p)\cdot
F(\upsilon^{-1})=(\phi_{\Delta_{\upsilon}}-\phi_{-\Delta_{\upsilon}})+\mathcal{C}(\upsilon)\cdot\vec{0}+(\phi_{-\Delta_{\upsilon}}-\phi_{\Delta_{\upsilon}})=\vec{0}$.

By induction, we thus show that $p\in\mathcal{P}$, $F(p)=0$. Hence, $F$ maps
cosets of $\mathcal{P}$ in $\mathcal{A}$ into single elements of
$\mathbb{Z}^{\phi}$, so we conclude $\overline{F}$ exists and we are done. ∎

######  Lemma 5.3.

Let $\overline{F}:\mathcal{C}\to\mathbb{Z}^{\Phi}$ be as defined in Lemma 5.2,
and let $\Phi^{+}$ be the set of positive roots associated to the choice
$\Delta$ of simple roots. Then for all $\phi\in\Phi^{+},c\in\mathcal{C}$, we
have $\overline{F}(c)[\phi]\in\\{0,1\\}$ (where $\overline{F}(c)[\phi]$ is the
$\phi$ component of $\overline{F}(c)$).

###### Proof.

If $c=e$, then the statement is trivial. Hence, suppose it holds for $c$; we
shall prove it for $\vartheta c$, where $\vartheta$ is a Coxeter generator of
$\mathcal{C}$ (that is, the image of an Artin generator of $\mathcal{A}$). To
this end, we find, for any $\phi\in\Phi$, that $\overline{F}(\vartheta
c)[\phi]=\overline{F}(\vartheta)[\phi]+(\vartheta\cdot\overline{F}(c))[\phi]=\overline{F}(\vartheta)[\phi]+\overline{F}(c)[\vartheta\star\phi]$,
where $\vartheta\star\phi$ is the image of $\phi$ under the reflection across
the hyperplane orthogonal to $\alpha_{\vartheta}$. If
$\alpha_{\vartheta}\neq\phi$, then the first term vanishes, and since
$\vartheta\star\phi$ also belongs to $\Phi^{+}$, the conclusion follows. If
$\alpha_{\vartheta}=\phi$, then we have $\overline{F}(\vartheta
c)[\phi]=1+\overline{F}(c)[-\phi]=1-\overline{F}(c)[\phi]\in\\{0,1\\}$, and
the conclusion again follows. By induction on the length of $c$, we are done.
∎

Since this cocycle exists, the image of the super-Gauss-Epple homomorphism is
not a semidirect product (as there is no way to embed the Coxeter group as a
subgroup). However, we can show that the image of $\mathcal{P}$ under the
super-Gauss-Epple homomorphism is abelian. Note that this is not a semidirect
product, for reasons analogous to those given at the end of Section 4.1
(namely, that no element of $\mathcal{A}$ whose image in $\mathcal{C}$ is a
canonical generator can have a square whose image under super-Gauss-Epple is
that of the identity).

We can also characterize the image of the super-Gauss-Epple homomorphism the
following way: Note that, if $c\in\mathcal{A}$ and $a$ is a canonical
generator of $\mathcal{A}$, then
$SGE(ca^{2}c^{-1})=(v_{c},\mathcal{C}(c))(\Phi_{\Delta_{a}}+\Phi_{-\Delta_{a}},e)(-\mathcal{C}(c)^{-1}v_{c},W(c))=(\Phi_{\mathcal{C}(c)\cdot\Delta_{a}}+\Phi_{\mathcal{C}(c)\cdot\Delta_{a}},e)$.
Hence, the set of all pairs $(v,c)\in\mathbb{Z}^{\Phi}\rtimes C$ that belong
to the image of SGE are the pairs such that $v-\overline{v}=\overline{F}(c)$.

##  6 Complex reflection groups

We considered a candidate analogue of a “symmetric super-Gauss-Epple
homomorphism”, which would have type signature
$B\to\mathbb{Z}^{\mathcal{A}}\rtimes W$: namely, a homomorphism given by
generating relations $s\to(e_{H_{s}},W(s))$, where $H_{s}$ is a distinguished
generator of $B$ whose corresponding element in $W$ is a member of $\Psi$. For
example, in the case of $\mathcal{C}=G(n,1,1)=\mathbb{Z}/n\mathbb{Z}$, we have
$\mathcal{A}=\mathbb{Z}$, and $\mathcal{HA}$ consists of a single element
$\mathcal{H}$. Therefore, the associated homomorphism is an injective
homomorphism of type $\mathbb{Z}\to\mathbb{Z}/n\mathbb{Z}\times\mathbb{Z}$
given by the generating relation $1\to(1,1)$. It is trivial to verify that
this is indeed a homomorphism.

However, using a geometric argument, we show that no such homomorphism exists,
unless $B$ is an Artin group of finite type:

######  Theorem 6.1 (No SGE for complex reflection groups).

Let $B$ be some braid group associated to a complex reflection group, $W$ be
the associated complex reflection group, and $\mathcal{HA}$ the associated
hyperplane arrangement. Suppose that, in the canonical Artin-like presentation
of $B$, there is a relation of the form $aba=bab$, where $a$ and $b$ are two
distinct generators. Then there is no homomorphism
$B\to\mathbb{Z}^{\mathcal{A}}\rtimes W$ given by generating relations
$s\to(e_{H_{s}},W(s))$, where $H_{s}$ is a distinguished generator of $B$
whose corresponding element in $W$ is a member of $\Psi$.

###### Proof.

We prove
$(e_{H_{a}},W(a))(e_{H_{b}},W(b))(e_{H_{a}},W(a))\neq(e_{H_{b}},W(b))(e_{H_{a}},W(a))(e_{H_{b}},W(b))$.

We can easily compute that the former part of the left hand side will be
$e_{H_{a}}+e_{W(a)H_{b}}+e_{W(a)W(b)e_{H_{a}}}$, and the former part of the
right hand side would be $e_{H_{b}}+e_{W(b)H_{a}}+e_{W(b)W(a)e_{H_{b}}}$. We
claim that $H_{a}=W(b)W(a)H_{b},H_{b}=W(a)W(b)H_{a}$, but that $W(a)H_{b}\neq
W(b)H_{a}$.

Firstly, however, we shall need to use the determinant trick. We have
$W(a)W(b)W(a)=W(b)W(a)W(b)$. Since both sides are linear transformations, we
can take determinants:
$\det(W(a))\det(W(b))\det(W(a))=\det(W(b))\det(W(a))\det(W(b))$. Since these
determinants are nonzero real numbers, we can cancel to obtain
$\det(W(a))=\det(W(b))$. This equality is important because letting
$\zeta_{W(a)}$ be the multiplier of $W(a)$ and similarly for $\zeta_{W(b)}$,
we can conclude that $\zeta_{W(a)}=\zeta_{W(b)}$, and will refer to both
constants as merely $\zeta$.

Now, to show $H_{a}=W(b)W(a)H_{b}$, let $s_{\zeta,H}$ be the pseudo-reflection
that multiplies by $\zeta$ and preserves $H$. We compute

$\displaystyle s_{W(b)W(a)H_{b},\zeta}$
$\displaystyle=W(b)W(a)s_{H_{b},\zeta}(W(b)W(a))^{-1}$
$\displaystyle=W(b)W(a)W(b)(W(b)W(a))^{-1}$
$\displaystyle=W(a)W(b)W(a)(W(b)W(a))^{-1}$ $\displaystyle=W(a)$
$\displaystyle=s_{H_{a},\zeta},$

which gives the claim. By symmetry, we can also obtain $H_{b}=W(a)W(b)H_{a}$.

Now to show that $W(a)H_{b}\neq W(b)H_{a}$, we compute

$\displaystyle s_{W(a)H_{b},\zeta}$
$\displaystyle=W(a)s_{H_{b},\zeta}W(a)^{-1}$ $\displaystyle=W(a)W(b)W(a)^{-1}$
$\displaystyle=(W(a)W(b)W(a))W(a)^{-2},$

and similarly
$s_{W(b)H_{a},\zeta}=(W(b)W(a)W(b))W(b)^{-2}=(W(a)W(b)W(a))W(b)^{-2}$. Observe
now that $W(a)^{2}=s_{H_{a},\zeta^{2}}$ and $W(b)^{2}=s_{H_{b},\zeta^{2}}$.
Since $a$ and $b$ are distinct, this means that $H_{a}$ and $H_{b}$ are
distinct. This in turn implies that $s_{H_{a},\zeta^{2}}$ and
$s_{H_{b},\zeta^{2}}$ are distinct unless $\zeta^{2}=1$. However, if
$\zeta^{2}=1$, then $\zeta=-1$, so that $a$ and $b$ are merely real
reflections, as desired. ∎

## References

  * [1] Georgios Alexopoulos.  Convolution powers on discrete groups of polynomial volume growth.  In Canad. Math. Soc. Conf. Proc. 21, pages 31–57, 1997.
  * [2] E. Artin.  Theory of Braids.  Annals of Mathematics, 48(1):101–126, 1947.
  * [3] Emil Artin.  Theorie der Zöpfe.  Abhandlungen aus dem Mathematischen Seminar der Universität Hamburg, 4(1):47–72, 1925.
  * [4] Ben Nasmith.  All About Root Systems.  https://www.cantorsparadise.com/all-about-root-systems-20c651b19d25, 2020\.
  * [5] Michel Broué.  Reflection Groups, Braid Groups, Hecke Algebras, Finite Reductive Groups.  In Current Developments in Mathematics, pages 1–107, 2000.
  * [6] Michel Broué.  Introduction to Complex Reflection Groups and Their Braid Groups.  Springer, 2010.
  * [7] Moritz Epple.  Orbits of Asteroids, a Braid, and the First Link Invariant.  Math. Intell., 1998.
  * [8] Juan González-Meneses.  Basic results on braid groups.  In Annales Mathématiques Blaise Pascal, volume 18, pages 15–59, 2011.
  * [9] Kilom691.  Finite_Dynkin_diagrams.svg.  https://commons.wikimedia.org/wiki/File:DynkinDiagrams.svg, 2007\.
  * [10] Daan Krammer.  Braid groups are linear.  Annals of Mathematics, pages 131–156, 2002.
  * [11] Minh-Tâm Trinh.  https://math.mit.edu/~mqt/math/research/2002_07.pdf.
  * [12] L. Saloff-Coste.  Random Walks On Infinite Graphs And Groups (Cambridge Tracts in Mathematics 138). By Wolfgang Woess.  https://www.math.tugraz.at/~woess/BLMSrev.pdf.
  * [13] Laurent Saloff-Coste.  Probability on Groups: Random Walks and Invariant Diffusions.  Not. Am. Math. Soc., 2001.
  * [14] Alex Suciu.  The Pure Braid Groups and Their Relatives.  https://web.northeastern.edu/suciu/slides/Caen-Jan2016.pdf.
  * [15] Eric W. Weisstein.  Whitehead Link.  https://mathworld.wolfram.com/WhiteheadLink.html.  From MathWorld–A Wolfram Web Resource.


# LapSeg3D: Weakly Supervised Semantic Segmentation of Point Clouds
Representing Laparoscopic Scenes

Benjamin Alt1, Christian Kunz2, Darko Katic1, Rayan Younis3, Rainer Jäkel1,
Beat Peter Müller-Stich3, Martin Wagner3, and Franziska Mathis-Ullrich2 *This
work was supported by the German Federal Ministry of Education and Research
under the grant 13GW0471C.1B. Alt, D. Katic and R. Jäkel are with Artiminds
Robotics GmbH, 76131 Karlsruhe, Germany2 C. Kunz and F. Mathis-Ullrich are
with the Institute for Anthropomatics and Robotics, Karlsruhe Institute of
Technology, 76131 Karlsruhe, Germany, corresponding author:
<EMAIL_ADDRESS>R. Younis, B.P. Müller-Stich and M. Wagner are with
the Department for General, Visceral and Transplantation Surgery, Heidelberg
University Hospital, 69120 Heidelberg, Germany.

###### Abstract

The semantic segmentation of surgical scenes is a prerequisite for task
automation in robot assisted interventions. We propose LapSeg3D, a novel DNN-
based approach for the voxel-wise annotation of point clouds representing
surgical scenes. As the manual annotation of training data is highly time
consuming, we introduce a semi-autonomous clustering-based pipeline for the
annotation of the gallbladder, which is used to generate segmented labels for
the DNN. When evaluated against manually annotated data, LapSeg3D achieves an
F1 score of 0.94 for gallbladder segmentation on various datasets of ex-vivo
porcine livers. We show LapSeg3D to generalize accurately across different
gallbladders and datasets recorded with different RGB-D camera systems.

©2022 IEEE. Personal use of this material is permitted. Permission from IEEE
must be obtained for all other uses, in any current or future media, including
reprinting/republishing this material for advertising or promotional purposes,
creating new collective works, for resale or redistribution to servers or
lists, or reuse of any copyrighted component of this work in other works.

##  I INTRODUCTION

The understanding of the surgical scene is a crucial requirement of active
robotic assistance and task automation in robot assisted surgery (RAS). An
important step in this pipeline is the semantic segmentation of the
laparoscopic image frame, where every pixel or voxel is assigned a class label
of the structure it belongs to, e.g. the gallbladder. In the standard of care,
mono laparoscopes are used and the surgical scene is visualized to the surgeon
on a monitor. In recent years, the use of stereo laparoscopes has increased,
as they have become the state of the art in robotic telemanipulators, such as
the da Vinci (Intuitive Surgical Inc., USA), improving the surgeon’s spatial
understanding of the surgical scene. However, their potential is not fully
utilized. In particular and towards autonomous robotic assistance in surgery,
stereo laparoscopes offer several advantages to automated systems and
navigation systems (with augmented reality or otherwise), as it is possible to
reconstruct the three-dimensional surgical scene from the left and right image
streams. This provides a better basis for surgical planning and the automated
robotic execution of a task, e.g. grasping and tissue manipulation. To make
specific use of the 3D-reconstructed surgical scene for surgical navigation,
the individual points need to be semantically segmented, i.e. assigned to
their corresponding tissue type (e.g. “liver” or “gallbladder”). This is
especially challenging due to the circumstances in RAS, such as the necessity
for small sensors (which fit in the laparoscope), wet surfaces, blood and
smoke in the scene, and oftentimes similar textures and colors of organs.

In current research, semantic segmentation of the surgical scene has primarily
been focused on the recognition and pose estimation of surgical instruments in
two-dimensional laparoscopic image frames [1, 2], which is an essential step
for skill automation [3]. Presented approaches mainly utilize random forests
[4, 5] or convolutional neural networks (CNNs) [6, 7, 8, 9].

Figure 1: Setup of the ex-vivo experiments and the resulting segmented point
cloud of the surgical scene.

Few presented methods address the semantic segmentation of the full surgical
scene. Scheikl et al. [10] investigated the performance of different deep
learning architectures (e.g. U-Net, TernausNet, FCN, LinkNet, SegNet) for the
pixel-wise semantic segmentation of the complete surgical scene. Five classes
were segmented (instruments, liver, gallbladder, fat and other). The
structures were segmented with an Intersection over Union (IoU) of 0.79.
Maqbool et al. [11] propose a method for pixel-wise semantic segmentation of
the surgical scene by using a CNN Encoder-Decoder architecture. Additional
works have been introduced as results of the MICCAI 2021 Endoscopic Vision
‘HeiSurf-Subchallenge’ for surgical workflow analysis and full scene
segmentation [12].

Figure 2: Overview of the proposed pipeline for the segmentation of point
clouds representing laparoscopic scenes.

While these methods address the semantic segmentation of two-dimensional image
frames of mono laparoscopes, few works exist that present approaches to
perform a voxel-wise classification on three-dimensional point clouds.
Haouchine and Cotin [13] segment point clouds of laparoscopic scenes based on
curvatures and normals to approximate connected surfaces of organs, e.g. the
liver. However, the organs’ textures are not taken into account.

In other non-surgical domains several works for 3D image segmentation have
been proposed. Qi et al. introduce a deep neural network (DNN) to process
point clouds for various 3D recognition tasks, such as image segmentation [14,
15]. Other works deal with object recognition and shape completion in point
clouds, e.g. 3D Shape Nets [16] and voxel-based methods [16, 17]. Tchapmi et
al. propose SegCloud, an end-to-end 3D segmentation framework for point clouds
[18]. The LapSeg3D architecture proposed in this work is comparable to 3D-UNet
proposed by Cicek et al. [19], where it is utilized for kidney segmentation in
volumetric confocal microscopic data.

In this work, we focus on cholecystectomy, i.e. the removal of the gallbladder
from the liver. We introduce LapSeg3D, a novel approach for semantic
segmentation of point clouds to identify the 3D-surface of a gallbladder. Our
core contribution is a weakly supervised clustering-based pipeline capable of
bootstrapping a large, diverse dataset of labelled training examples (cf. Fig.
2). This dataset is then used to train a DNN to perform the segmentation task.
By generating its training data using weak supervision, LapSeg3D benefits from
the fast inference and good generalization capabilities of DNNs while avoiding
the need for manually labelled training data. We evaluate our approach on a
diverse set of ex-vivo gallbladders. LapSeg3D is highly accurate, achieving F1
scores of 0.94, and generalizes well across different gallbladders. During
run-time, the complete segmentation from raw point cloud to upsampled
segmentation results is performed in 162 ms ($\sigma$=3 ms) on a NVIDIA
GeForce 3080 Ti GPU, with the DNN itself performing inference in 17 ms
($\sigma$=1 ms). This enables intraoperative online usage, such as for
navigation or visual servoing of robotic manipulators.

##  II METHODS

###  II-A DNN for RGB-D gallbladder segmentation

To segment the gallbladder in RGB-D images of surgical scenes, we propose a
DNN based on 3D-UNet [19]. 3D-UNet is a 3D extension of UNet, a state-of-the-
art architecture for 2D biomedical image segmentation [20]. Our 3D-UNet
implementation takes a 80x80x80 voxel grid $X$ of the surgical scene as input,
where each voxel has three channels corresponding to the RGB color of the
scene at this voxel. The network outputs a 80x80x80 voxel grid $\hat{Y}$ of
the surgical scene, where each voxel has one binary channel indicating whether
it is part of the gallbladder. The network architecture is shown in Fig. 3.
There are several deviations from the original 3D-UNet architecture. Most
notably, our network has much fewer parameters (746,365 compared to
19,069,955), due to the smaller dimensions of the input voxel grid as well as
smaller feature maps at each layer. The smaller number of parameters reduces
the hardware requirements at runtime: LapSeg3D has a memory footprint of 2700
MB, enabling our network to perform inference on CPUs and consumer-grade GPUs.
We use instance normalization before each Rectified Linear Unit (ReLU)
activation, as we empirically found it to outperform batch normalization for
our application. The weighted Softmax output activation function in [19] was
replaced by a Sigmoid activation, as the DNN learns a voxel-wise binary
classification task and labels are available for every voxel.

Figure 3: Neural network architecture of LapSeg3D.

Given a training dataset
$\mathcal{D}_{train}=\\{(X_{0},Y_{0}),...,(X_{N},Y_{N})\\}$ containing $N$
input-label pairs, we use the Adam optimizer [21] to minimize the binary
crossentropy (BCE) between the output voxel grids $\hat{Y}$ and their
corresponding ground-truth voxel grids $Y$:

$BCE(Y,\hat{Y})=-\frac{1}{|Y|}\sum^{|Y|}_{i=1}\Big{[}Y^{i}\log(\hat{Y}^{i})+(1-Y^{i})\log(1-\hat{Y}^{i})\Big{]},$
(1)

where $Y^{i}$ and $\hat{Y}^{i}$ denote the ith entries in voxel grids $Y$ and
$\hat{Y}$. This corresponds to a supervised training regime, as the labelled
voxel grids $Y$ are used during training.

In real-world surgical applications, any solution for segmenting surgical
scenes must generalize to new gallbladders not present in the training
dataset, as it is difficult to collect new patient-specific training data and
re-train the network before surgery. We leverage data augmentation to improve
the generalization capacity of the DNN. Random rotation around the coordinate
axes ($rx,ry,rz\in[-30^{\circ},30^{\circ}]$), uniform scaling (factor
$s\in[0.8,1.2]$) and elastic transformation (factor $f\in[0,0.3]$) are applied
with the probability $p=0.5$ to each input-label tuple per epoch. Random gamma
correction ($\gamma\in[0.7,1.5]$), contrast ($\alpha\in[0.7,1.3]$), and
brightness ($\beta\in[-0.3,0.3]$) are always applied.

###  II-B Weakly supervised gallbladder segmentation

The proposed segmentation DNN is trained on labelled input-label pairs via a
supervised learning regime. To ensure the applicability of the approach in
real-world surgical settings and minimize human involvement during training,
we propose to convert it into a weakly-supervised regime by generating high-
quality labelled training data using a clustering-based pipeline. To bootstrap
a large training dataset for the segmentation DNN, we contribute the multi-
stage weakly supervised data processing pipeline outlined in Fig. 2 (2). Its
objective is to generate high-quality labelled training examples for the DNN
by segmenting the gallbladder from a large number of raw RGB-D images of
surgical scenes, while requiring a minimal amount of human intervention.

The proposed pipeline is composed of three distinct stages (cf. Fig. 2 (2)),
which are described below. Human involvement is only required for the very
first training example - all other RGB-D images in the dataset are processed
autonomously.

####  II-B1 Alignment

To ensure robustness against changes in the relative position of the camera
with respect to the scene, the raw RGB-D point cloud is first aligned to a
reference scene $P_{ref}$ via a homogeneous transformation $T_{align}$. Our
approach assumes a static camera pose per dataset, but allows for changing
camera poses between datasets. Aligning all datasets to the same reference
scene permits the network to be trained across many different surgical scenes
covering multiple patients and camera perspectives. For surgical practice,
this implies that the pose of the stereo laparoscope must not change during
surgery, or that the current laparoscope pose must be known (i.e. tracked) at
all times. For our experiments, $T_{align}$ is computed via least-squares
regression over point correspondences between the current and reference point
clouds. For the first point cloud $P_{1}$ of a dataset, a human is asked to
identify at least four point correspondences between $P_{1}$ and $P_{ref}$. In
practice, we found that identifying points at the tip (fundus) as well as at
the neck (infundibulum, collum) of the gallbladder in both $P_{1}$ and
$P_{ref}$ produced sufficient alignment. The computed $T_{align}$ is used for
the remainder of the dataset.

####  II-B2 Cropping

The aligned point cloud is cropped to contain only the surgical scene. This is
particularly important for cameras or stereo laparoscopes with a field of view
much larger than the scene. Again, a human is asked to select a bounding box
for $P_{1}$, which is then applied to the remainder of the dataset.

####  II-B3 Clustering-based gallbladder segmentation

Gallbladder segmentation is performed by an algorithm, which combines
clustering with heuristics specific to the domain of organ segmentation. In a
first step, color-based 3D region growing [22] is leveraged to split the raw
point clouds into clusters of similar color. Similar to the alignment and
cropping stages, a human expert provides a set of gallbladder colors by
picking points in $P_{1}$. Clusters with colors sufficiently different from
any picked gallbladder colors are rejected outright. The remaining clusters
are merged into the final gallbladder cluster $C_{GB}$ via the following
algorithm:

  1. 1.

$C_{GB}$ is initialized with the largest cluster.

  2. 2.

All clusters adjacent to $C_{GB}$ are added to $C_{GB}$. Two clusters are
adjacent if the smallest distance between any two points in the clusters is
below a given threshold.

  3. 3.

The process is iterated until $C_{GB}$ ceases to grow.

The generated training dataset then consists of the aligned and cropped RGB-D
images of the surgical scene as well as the corresponding segmentation
results. Both input and label images are downsampled to an 80x80x80 voxel
grid.

##  III EXPERIMENTAL VALIDATION

We conduct a series of experiments on ex-vivo porcine models to assess the
validity of our approach.

###  III-A Experimental setup

The experimental setup is illustrated in Fig. 1. An operating table is
equipped with a UR5 industrial manipulator (Universal Robots A/S, Odense,
Denmark). The robot’s end effector is a standard laparoscopic gripper (KARL
STORZ SE & Co KG, Tuttlingen, Germany), which is connected to the robot via a
customized mechatronic interface. The segmentation of the instrument is not
within the scope of this work. State-of-the-art algorithms achieve high F1
scores (approx. 0.88) for the semantic segmentation of the complete instrument
[1]. To easily track the instrument, we apply green markers to the gripper
fingers. The surgical procedures are performed on ex-vivo porcine livers by a
human surgeon. Two RGB-D cameras acquire data for the experiments: A Zivid One
industrial RGB-D camera (Zivid AS, Oslo, Norway), operating via structured
light and providing point clouds with high dynamic range at a resolution of
1920 x 1200 and a frame rate of up to 10 Hz; as well as a ZED Mini RGB-D
camera (Stereolabs Inc., San Francisco, U.S.A), with lower dynamic range, but
higher frame rate of 15 Hz at a resolution of 4416x1242 (left and right image
side by side).

###  III-B Data collection

We collected a total of 15 datasets, each containing between 10 and 483 raw
point clouds of the surgical site, from a total of nine ex-vivo porcine
models. The datasets cover different stages of the operation from three
different camera angles. Six datasets were recorded using the Zivid camera,
while the Stereolabs camera was used for the remaining nine. Each dataset was
collected using the following protocol:

  1. 1.

The liver is positioned upside-down compared to the in-vivo anatomy, i.e. with
the gallbladder and the hilum of the liver facing upwards. Thus, the
gallbladder is always visible. This results in a view similar to the
intraoperative view when gallbladder and liver are elevated by a grasper at
the gallbladder fundus.

  2. 2.

Blunt or electrocautery dissection of Calot’s triangle; clipping and cutting
of cystic duct and cystic artery. No RGB-D data is collected during this step.

  3. 3.

Teleoperated grasping of the gallbladder by the robot. The grasp point is
dependent on the state of the operation. Initially, the gallbladder is grasped
at the infundibulum. At later stages, the grasp point advances along the body
toward the fundus to remain close to the respective dissection plane between
liver and gallbladder.

  4. 4.

Execution of random teleoperated end-effector motions for 2-3 minutes. The
teleoperator mimics the range and type of gripper motions (i.e., lateral,
upward, and backward) commonly performed by the surgeon. RGB-D point clouds of
the surgical site are continuously recorded during teleoperation.

  5. 5.

Teleoperated release of the gallbladder and continuation of gallbladder
removal by the surgeon.

  6. 6.

Steps 2-4 are repeated respectively after the removal of the first, second and
third fifths of the gallbladder.

In the first experiments (Stereloabs 1-5, Zivid) the liver and gallbladder
were positioned in a white box, which caused reflections of the gallbladder
visible in the RGB-D images. Thus, for the last four datasets (Stereolabs
6-9), a matte black box was used.

###  III-C Experiments

####  III-C1 Comparison with expert labels

In a first series of experiments, we compare the segmentation results of
LapSeg3D with a total of 67 RGB-D scenes manually segmented by surgeons using
CloudCompare [23]. Three models were trained on three different training
datasets: allRG containing a total of 2,311 RGB-D images of surgical scenes
from both cameras; slRG, containing 1,921 images from the Stereolabs camera;
and zivRG, containing 390 images from the Zivid camera. Training times for the
networks varied with the size of the dataset. For the largest dataset (allRG),
training took approximately 8 hours on a NVIDIA 3090 GPU. All training data
was generated using the region growing-based weakly supervised processing
pipeline (RG). To assess the data generation pipeline, its outputs are also
included in the analysis.

Figure 4: Segmentation results (from left to right): original point cloud
recorded by sensor, annotation derived through region growing (magenta),
annotation predicted through our proposed LapSeg3D (red), compared to manual
annotations of the medical experts (blue).

####  III-C2 4-fold crossvalidation

In a second series of experiments, we perform 4-fold crossvalidation to assess
the capacity of LapSeg3D to generalize to scenes and gallbladders beyond the
training dataset. Four networks were trained, each on a subset of the datasets
collected with the Stereolabs camera: sl678RG, sl689RG, sl789RG and sl679RG,
combinations of data from the 6th, 7th, 8th and 9th gallbladder. The networks
are evaluated on the labels of the respective missing gallbladder provided by
a human surgeon.

##  IV RESULTS

Table I: Performance of LapSeg3D compared to human expert labels.

Model | $\mathcal{D}_{train}$ | $\mathcal{D}_{test}$ | P | R | F1 | IoU
---|---|---|---|---|---|---
LapSeg3D | allRG | allH | 0.94 | 0.95 | 0.94 | 0.89
LapSeg3D | allRG | slH | 0.95 | 0.95 | 0.95 | 0.90
LapSeg3D | allRG | zivH | 0.91 | 0.96 | 0.93 | 0.88
LapSeg3D | slRG | allH | 0.95 | 0.82 | 0.86 | 0.79
LapSeg3D | slRG | slH | 0.97 | 0.91 | 0.94 | 0.89
LapSeg3D | slRG | zivH | 0.92 | 0.67 | 0.72 | 0.61
LapSeg3D | zivRG | allH | 0.93 | 0.84 | 0.87 | 0.78
LapSeg3D | zivRG | slH | 0.91 | 0.80 | 0.84 | 0.73
LapSeg3D | zivRG | zivH | 0.96 | 0.91 | 0.93 | 0.88
Region Growing | - | allH | 0.96 | 0.92 | 0.94 | 0.89
Region Growing | - | slH | 0.96 | 0.92 | 0.94 | 0.88
Region Growing | - | zivH | 0.95 | 0.93 | 0.94 | 0.89

P: Precision, R: Recall, F1: F1 Score, IoU: Intersection over Union.
RG: Labels generated by weakly supervised region growing.
H: Labels annotated by a human surgeon.
sl: Stereolabs camera, ziv: Zivid One camera.

The results of experiment 1 (see Section III-C1) are summarized in Table I.
Trained on a dataset comprising all nine gallbladders, LapSeg3D achieves an F1
score of 0.94 and an IoU score of 0.89 when compared against labels provided
by a medical expert. Echoing the results in [12], training on data from one
camera (e.g. Zivid in zivRG) and evaluating on data from another (e.g.
Stereolabs in slH) lowers F1 and IoU scores. Given the large differences in
the dynamic range and resolution between the two cameras, however, the scores
(F1 = 0.84, IoU = 0.73) are highly competitive, particularly given that the
training set zivRG contained only 390 images of three gallbladders.
Segmentation of raw point clouds with LapSeg3D took a mean of 162 ms
($\sigma$=3 ms) per point cloud, with the DNN itself requiring 17 ms
($\sigma$=1 ms) on an NVIDIA 3080 Ti GPU. The weakly supervised labelling
pipeline also achieves F1 scores of 0.94 and IoU scores above 0.88 on all
datasets, demonstrating its capacity to bootstrap high-quality labels with
little human involvement.

Qualitative analysis of the segmented point clouds confirms the results. Fig.
4 (top) shows a raw RGB-D point cloud from the Stereolabs camera (left),
human-provided ground truth labels (blue) and the segmentation results of
region growing (magenta) and LapSeg3D (red) in the downsampled voxel space,
which both provide very good segmentation. The middle and bottom rows show
segmentation results for Zivid data. In both cases, LapSeg3D provides accurate
segmentation. The bottom row illustrates our observation that LapSeg3D
segments even strongly deformed gallbladders well, including parts of the
collum (marked in Fig. 4 with a white arrow), which region growing failed to
segment.

Fig. 5 shows examples from the test data in allRG, illustrating the capacity
of LapSeg3D to segment gallbladders in different deformation states, during
different phases of the operation, and grasped at different points. Unlike
Fig. 4, Fig. 5 shows the segmentation results upsampled to the original sensor
resolution.

Table II summarizes results of 4-fold crossvalidation. LapSeg3D achieves an
average F1 score of 0.84 and an average IoU of 0.74, indicating that it
generalizes well to new scenes. The results are evidence that training data
from three gallbladders suffice for LapSeg3D to learn sufficiently meaningful
features to segment unseen gallbladders. Given that individual gallbladders
vary greatly from patient to patient and gathering patient-specific data is
costly, they indicate that LapSeg3D and its weakly supervised training scheme
are well suited to real-world surgical applications.

Figure 5: Results of different segmentations upsampled to their original size. Table II: Performance of LapSeg3D under 4-fold crossvalidation. Model | $\mathcal{D}_{train}$ | $\mathcal{D}_{test}$ | P | R | F1 | IoU
---|---|---|---|---|---|---
LapSeg3D | sl678RG | sl9H | 1.0 | 0.75 | 0.86 | 0.75
LapSeg3D | sl679RG | sl8H | 0.83 | 0.96 | 0.89 | 0.81
LapSeg3D | sl689RG | sl7H | 0.93 | 0.85 | 0.89 | 0.83
LapSeg3D | sl789RG | sl6H | 0.69 | 0.81 | 0.73 | 0.58

##  V DISCUSSION

###  V-A Weakly-supervised training pipeline

We propose a pipeline for weakly-supervised generation of training data. While
not fully unsupervised, human involvement during training is reduced to
selecting points and bounding boxes on one image per dataset. Quality of the
input depends on the user’s experience and can lead to sub-optimal results,
when non-representative points of the gallbladder are chosen. The weakly-
supervised training pipeline achieves an F1 score of 0.94. Methods to sort out
remaining bad training data are the subject of future work.

###  V-B LapSeg3D

Other works have shown that it is challenging for a trained neural network to
be used on image data from unknown sensor systems. When LapSeg3D is trained on
the Stereolabs dataset, it still achieves a good F1 score of 0.72 when
evaluated on Zivid point clouds. This rate can be further increased when the
training dataset is enhanced with Zivid data. Future work will investigate how
results on unknown camera systems can be further improved, e.g. through
additional data augmentation.

An additional challenge in biomedical image segmentation is the great variance
in patients’ anatomies and pathologies, which can lead to different colouring,
texture, and dynamic behaviour of the gallbladder. LapSeg3D copes well with
these variations, achieving a highly competitive average F1 score of 0.84 on a
4-fold crossvalidation task.

Our approach is most challenged by different lighting conditions and noise in
the point clouds. This can lead to errors segmenting the borders of the
gallbladder and the transition to the liver, but only small areas around the
gallbladder are affected.

We have shown that LapSeg3D is able to learn to segment structures that are
not present in the training data, such as the infundibulum, which is grasped
by a laparoscopic gripper, as depicted in Fig. 4 (white arrow). In future
work, we will consider the detailed semantic segmentation of the gallbladder
into its anatomical components, with dedicated labels for e.g. cystic duct,
infundibulum and fundus.

The definition of whether voxels are part of the gallbladder may not be always
clear and depend on the annotator. This mainly concerns structures such as the
infundibulum, the collum and Calot’s triangle. Here, we define the gallbladder
starting at the collum, but even for a medical expert this can vary in the
manually annotated datasets.

With an overall F1 score of 0.94, our approach outperforms the state of the
art for segmenting 2D images. This could be an indication that the 3D
structure of an organ is also taken into account during segmentation.

###  V-C Limitations

The acquired point clouds were recorded using two external camera systems. It
can be assumed that the ZED mini camera is comparable to state-of-the-art
stereo laparoscopes, as they provide a similar resolution. 3D reconstruction
was not considered in this work. Both camera systems provide methods to
automatically reconstruct the 3D scene from image data. Therefore, our
approach needs to be transferred to laparoscopic sensors, such as stereo
laparoscopes. The ex-vivo livers were recorded at a distance of approximately
35 cm and were always visible at the center of the point clouds. When
recording data inside the human body, the field of view may be limited. We
have shown that our approach was able to learn the segmentation of difficult
deformation states of the gallbladder. Future work will transfer LapSeg3D to
laparoscopic image data with a limited field of view.

##  VI CONCLUSION

This work presents a method for fast and reliable gallbladder segmentation in
point clouds of surgical scenes as a basis for automated robotic gallbladder
removal. The presented neural network LapSeg3D was trained using a weakly
supervised method for automated generation of training data utilizing a region
growing approach, and was shown to be able to perform voxel-wise segmentation
of laparoscopic scenes with an F1 score of 0.94 in 17 ms.

## References

  * [1] S. Bodenstedt, _et al._ , “Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery,” _ArXiv Prepr. ArXiv180502475_ , 2018.
  * [2] D. Bouget, M. Allan, D. Stoyanov, and P. Jannin, “Vision-based and marker-less surgical tool detection and tracking: A review of the literature,” _Med. Image Anal._ , vol. 35, pp. 633–654, 2017.
  * [3] Z. Chen, _et al._ , “Virtual fixture assistance for needle passing and knot tying,” in _2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)_. IEEE, 2016, pp. 2343–2350.
  * [4] S. Bodenstedt, _et al._ , “Image-based laparoscopic bowel measurement,” _Int. J. Comput. Assist. Radiol. Surg._ , vol. 11, no. 3, pp. 407–419, 2016\.
  * [5] M. Allan, S. Ourselin, S. Thompson, D. J. Hawkes, J. Kelly, and D. Stoyanov, “Toward detection and localization of instruments in minimally invasive surgery,” _IEEE Trans. Biomed. Eng._ , vol. 60, no. 4, pp. 1050–1058, 2012\.
  * [6] D. Pakhomov, V. Premachandran, M. Allan, M. Azizian, and N. Navab, “Deep residual learning for instrument segmentation in robotic surgery,” in _International Workshop on Machine Learning in Medical Imaging_. Springer, 2019, pp. 566–573.
  * [7] L. C. García-Peraza-Herrera, _et al._ , “Real-time segmentation of non-rigid surgical tools based on deep learning and tracking,” in _International Workshop on Computer-Assisted and Robotic Endoscopy_. Springer, 2016, pp. 84–95.
  * [8] A. Agustinos and S. Voros, “2D/3D real-time tracking of surgical instruments based on endoscopic image processing,” in _Computer-Assisted and Robotic Endoscopy_. Springer, 2015, pp. 90–100.
  * [9] S. K. Hasan and C. A. Linte, “U-NetPlus: A modified encoder-decoder U-Net architecture for semantic and instance segmentation of surgical instruments from laparoscopic images,” in _2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)_. IEEE, 2019, pp. 7205–7211.
  * [10] P. M. Scheikl, _et al._ , “Deep learning for semantic segmentation of organs and tissues in laparoscopic surgery,” _Curr. Dir. Biomed. Eng._ , vol. 6, no. 1, 2020.
  * [11] S. Maqbool, A. Riaz, H. Sajid, and O. Hasan, “M2caiseg: Semantic segmentation of laparoscopic images using convolutional neural networks,” _ArXiv Prepr. ArXiv200810134_ , 2020.
  * [12] S. Bodenstedt, _et al._ , “Endoscopic Vision Challenge 2021: HeiChole Surgical Workflow Analysis and Full Scene Segmentation (HeiSurF),” Oct. 2021. [Online]. Available: https://caruscloud.uniklinikum-dresden.de/index.php/s/z7jNWCHQ5TYfpSx
  * [13] N. Haouchine and S. Cotin, “Segmentation and labelling of intra-operative laparoscopic images using structure from point cloud,” in _2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI)_. IEEE, 2016, pp. 115–118.
  * [14] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2017, pp. 652–660.
  * [15] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” _Adv. Neural Inf. Process. Syst._ , vol. 30, 2017.
  * [16] Z. Wu, _et al._ , “3d shapenets: A deep representation for volumetric shapes,” in _Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition_ , 2015, pp. 1912–1920.
  * [17] A. Brock, T. Lim, J. M. Ritchie, and N. Weston, “Generative and discriminative voxel modeling with convolutional neural networks,” _ArXiv Prepr. ArXiv160804236_ , 2016.
  * [18] L. Tchapmi, C. Choy, I. Armeni, J. Gwak, and S. Savarese, “Segcloud: Semantic segmentation of 3d point clouds,” in _2017 International Conference on 3D Vision (3DV)_. IEEE, 2017, pp. 537–547.
  * [19] Ö. Çiçek, A. Abdulkadir, S. S. Lienkamp, T. Brox, and O. Ronneberger, “3D U-Net: Learning dense volumetric segmentation from sparse annotation,” in _Int. Conf. on Medical Image Computing and Computer-Assisted Intervention_. Springer, 2016, pp. 424–432.
  * [20] Y. Alzahrani and B. Boufama, “Biomedical Image Segmentation: A Survey,” _SN COMPUT. SCI._ , vol. 2, no. 4, p. 310, May 2021.
  * [21] D. P. Kingma and J. Ba, “Adam: A Method for Stochastic Optimization,” _ArXiv14126980 Cs_ , Jan. 2017.
  * [22] Q. Zhan, Y. Liang, and Y. Xiao, “Color-based segmentation of point clouds,” _ISPRS Laser Scanning Workshop_ , vol. 38, July 2009.
  * [23] “CloudCompare.” [Online]. Available: http://www.cloudcompare.org/

  *[RAS]: robot assisted surgery
  *[CNNs]: convolutional neural networks
  *[IoU]: Intersection over Union
  *[DNN]: deep neural network
  *[DNNs]: deep neural network
  *[ReLU]: Rectified Linear Unit
  *[BCE]: binary crossentropy