# Higher rank Brill-Noether theory and coherent systems
Open Questions

P. E. Newstead  P. E. Newstead
Department of Mathematical Sciences
University of Liverpool
Peach Street, Liverpool L69 7ZL, UK<EMAIL_ADDRESS>
###### Abstract.

This article presents a list of open questions on higher rank Brill-Noether
theory and coherent systems. Background material and appropriate references
are included.

###### Key words and phrases:

Vector bundles on curves, Brill-Noether theory, coherent systems, Clifford
indices

###### 2010 Mathematics Subject Classification:

Primary: 14H60

The author is a member of the research group VBAC (Vector Bundles on Algebraic
Curves).

###### Contents

  1. 1 Background and definitions
  2. 2 Basic questions
  3. 3 Clifford indices and gonalities
  4. 4 Fixed determinant
  5. 5 Butler’s Conjecture
  6. 6 Coherent systems on the projective line
  7. 7 Coherent systems on elliptic curves
  8. 8 Coherent systems for $g\geq 2$
  9. 9 Final remarks

In this article, we present some of the many open questions on higher rank
Brill-Noether theory and coherent systems on algebraic curves. A previous
problems list including open questions on higher rank Brill-Noether theory was
produced originally in 1991 with an updated version in 1994 for the
foundational meeting of the research group Vector Bundles on Algebraic Curves
(VBAC). Mercat’s Présentation of 2001 contains a further list. In 2003, I
launched the Brill-Noether project aimed at solving the basic questions in 10
years. Of course, this was not achieved (I never really expected it to be),
but much progress has been made. The project and Mercat’s Présentation are
available on my website http://www.liverpool.ac.uk/$\sim$newstead; almost all
the questions stated there are still open in full generality, although they
have been solved in specific cases. The survey article [New11] concludes with
a list of problems on moduli spaces of coherent systems and many open
questions are stated in various papers on the subject.

The questions presented here are a mixture of general problems and more
specific ones. They are far from being a complete list. The list of references
is also incomplete; I have concentrated on those which form the basis for the
questions raised in this article. Most of the questions are unlikely to have a
clear-cut answer; partial solutions and counter-examples are a more plausible
aim.

My thanks are due to Usha Bhosle, Leticia Brambila-Paz, Gavril Farkas, George
Hitching, Michael Hoff, Emanuele Macrí, Alexander Schmitt and Montserrat
Teixidor i Bigas for comments, corrections, suggestions and additional
references. Responsibility for any remaining errors and omissions is entirely
mine.

##  1\. Background and definitions

Brill-Noether theory is concerned with the possible dimensions of the space of
sections $H^{0}(E)$ of a vector bundle on an algebraic variety. This is an
extension of the classical study of meromorphic functions on a smooth complex
algebraic curve (or Riemann surface) $C$ with poles along a given divisor.
This study began with Riemann, Roch and Clifford, followed by a considerable
number of other researchers in the 19th century, most notably for our purposes
Brill and Max Noether. The essence of the theory is the study of projective
embeddings of $C$. In more recent times, the theory has played a significant
role in the study of moduli spaces of curves.

We are concerned here with the higher-rank analogue; in other words, we
consider vector bundles over a projective curve $C$, which we shall suppose to
be defined over the complex numbers; many questions arise for curves defined
over other fields (either not algebraically closed or of finite
characteristic), but we shall not address these here. When
$h^{0}(E)>\text{rk}(E)$, there is a close connection with morphisms to
Grassmannians and with Quot schemes, also with syzygies and with the
geometrical structure of the moduli spaces of curves. In particular, counter-
examples to the Harris-Morrison slope conjecture have been discovered using
divisors in the moduli space which are defined in terms of rank-$2$ Brill-
Noether loci (see, inter alia, [FP05] and [FJP21]).

In what follows, I have attempted to use a consistent set of notations. This
is not the case in the literature.

Except in the final section, we shall suppose that $C$ is smooth and
irreducible. For $E$ a vector bundle, we write $\mu(E):=\frac{\deg
E}{\text{rk}E}$ for the slope of $E$. Recall that a vector bundle $E$ is said
to be stable (semistable) if every proper subbundle $F$ of $E$ satisfies
$\mu(F)<(\leq)\mu(E)$. Let $M(n,d)$ denote the moduli space of stable bundles
of rank $n$ and degree $d$ on $C$, and $\widetilde{M}(n,d)$ the
compactification of $M(n,d)$ whose points correspond to (S-equivalence classes
of) semistable vector bundles; if $\gcd(n,d)=1$, $M(n,d)=\widetilde{M}(n,d)$.
The Brill-Noether loci $B(n,d,k)\subset M(n,d)$ and
$\widetilde{B}(n,d,k)\subset\widetilde{M}(n,d)$ are defined by

$B(n,d,k):=\\{E\in M(n,d)|h^{0}(E)\geq k\\}$

and

$\widetilde{B}(n,d,k):=\\{[E]\in\widetilde{M}(n,d)|h^{0}(\text{gr}E)\geq
k\\},$

where $[E]$ denotes the S-equivalence class of $E$ and $\text{gr}E$ is the
associated graded bundle. (In classical notation, $B(1,d,k)=W^{k-1}_{d}$; in
the general situation, various other notations have been used.)

A coherent system on $C$ of type $(n,d,k)$ is a pair $(E,V)$, where $E$ is a
vector bundle of rank $n$ and degree $d$, and $V$ is a linear subspace of
$H^{0}(E)$ of dimension $k$; when $n=1$, these are referred to as linear
systems. When $k>n$, we say that $(E,V)$ is generated (generically generated)
if the evaluation map $V\otimes\mathcal{O}\to E$ is surjective (surjective at
all but a finite number of points); if $(E,V)$ is generated, it determines a
map from $C$ to the Grassmannian $\operatorname{Gr}(k,n)$. If $(E,H^{0}(E))$
is (generically) generated, we say also that $E$ is (generically) generated.

For any $\alpha\in{\mathbb{R}}$, the $\alpha$-slope of $(E,V)$ is defined by

$\mu_{\alpha}(E,V):=\frac{d+\alpha k}{n}.$

A coherent system $(E,V)$ is said to be $\alpha$-stable ($\alpha$-semistable)
if every proper subsystem $(F,W)$ of $(E,V)$ satisfies
$\mu_{\alpha}(F,W)<(\leq)\mu_{\alpha}(E,V)$. Note that $(E,0)$ is
$\alpha$-stable ($\alpha$-semistable) if and only if $E$ is stable
(semistable). It is easy to see that, if $k\geq 1$, $(E,V)$ can be
$\alpha$-semistable only if $\alpha\geq 0$, $d\geq 0$ and $\alpha(n-k)\leq d$
(note that the last of these conditions is vacuous if $k\geq n$). If $n=1$ and
$\alpha>(\geq)0$, all coherent systems are $\alpha$-stable
($\alpha$-semistable). For any $\alpha$, there exists a quasi-projective
moduli space $G(\alpha;n,d,k)$ of $\alpha$-stable coherent systems, which
possesses a natural compactification $\widetilde{G}(\alpha;n,d,k)$ given by
S-equivalence classes of $\alpha$-semistable coherent systems. When $n=1$,
these spaces are independent of $\alpha>0$ and we write them as $G(1,d,k)$
(classically $G^{k-1}_{d}$).

A key feature for coherent systems is that of a critical value. This can be
defined as a value of $\alpha$ for which there exist $\alpha$-semistable
coherent systems which become unstable (i.e. not semistable) for values of the
parameter on one side or other (possibly both) of $\alpha$. There is a simple
numerical criterion for critical values, which are rational numbers with a
finite set of possible denominators. It follows at once that there are
finitely many critical values when $k<n$ and this can be proved also for
$k\geq n$. We therefore have a sequence of critical values

$0=\alpha_{0}<\alpha_{1}<...<\alpha_{L}<\begin{cases}\frac{d}{n-k}&\text{if
$k<n$},\\\ \infty&\text{if $k\geq n$}.\end{cases}$

The moduli space can change only at a critical value and we write
$G_{i}(n,d,k)$ ($G_{L}(n,d,k)$) for $G(\alpha;n,d,k)$ with
$\alpha_{i}<\alpha<\alpha_{i+1}$ ($\alpha>\alpha_{L}$). The moduli space
$G_{L}(n,d,k)$ may be referred to as the terminal moduli space. One can define
similarly $\widetilde{G}_{i}(n,d,k)$ and $\widetilde{G}_{L}(n,d,k)$. Further
useful definitions are

$U^{s}(n,d,k):=\left\\{(E,V)\in G_{0}(n,d,k)\left|\begin{array}[]{c}(E,V)\in
G(\alpha;n,d,k)\text{ for }\\\
\alpha>0,\alpha(n-k)<d\end{array}\right.\right\\}$

and $U(n,d,k):=\\{(E,V)\in U^{s}(n,d,k)|E\text{ stable}\\}$.

For any critical value $\alpha_{i}$, one can define flip loci

$G_{i}^{-}:=\\{(E,V)\in G_{i-1}(n,d,k)|(E,V)\not\in G_{i}(n,d,k)\\})$

and

$G_{i}^{+}:=\\{(E,V)\in G_{i}(n,d,k)|(E,V)\not\in G_{i-1}(n,d,k)\\}.$

These describe the wall-crossing at $\alpha_{i}$.

The Brill-Noether number $\beta(n,d,k)$ is defined by

$\beta(n,d,k):=n^{2}(g-1)+1-k(k-d+n(g-1)),$

where $g$ is the genus of $C$ (suppressed in the notation). When $k\leq
d-n(g-1)$, $B(n,d,k)=M(n,d)$. Otherwise, the Brill-Noether number is often
referred to as the expected dimension of $B(n,d,k)$ and every irreducible
component of $B(n,d,k)$ has dimension at least $\beta(n,d,k)$. Moreover, with
no condition on $k$, every irreducible component of $G(\alpha;n,d,k)$ has
dimension at least $\beta(n,d,k)$.

The infinitesimal behaviour of $G(\alpha;n,d,k)$ at $(E,V)$ is governed by the
multiplication map

$\mu_{E,V}:V\otimes H^{0}(E^{*}\otimes K)\longrightarrow H^{0}(E\otimes
E^{*}\otimes K),$

where $E^{*}$ is the dual of $E$ and $K$ is the canonical line bundle on $C$.
This map is often referred to as the Petri map at $(E,V)$. When $V=H^{0}(E)$,
we write also $\mu_{E}$. The moduli space $G(\alpha;n,d,k)$ is smooth of
dimension $\beta(n,d,k)$ at $(E,V)$ if and only if $\mu_{E,V}$ is injective.
Moreover, if $E\in B(n,d,k)\setminus B(n,d,k+1)$, then $B(n,d,k)$ is smooth of
dimension $\beta(n,d,k)$ at $E$ if and only if $\mu_{E}$ is injective. If
$k>d-n(g-1)$, then $B(n,d,k+1)$ is contained in the singular set of
$B(n,d,k)$.

If we consider bundles of fixed determinant $\mathcal{L}$ of degree $d$, we
have a moduli space $M(n,\mathcal{L})$, Brill-Noether loci
$B(n,\mathcal{L},k)$ and moduli spaces $G(\alpha;n,\mathcal{L},k)$. It follows
at once from the above that every irreducible component of
$G(\alpha;n,\mathcal{L},k)$ has dimension at least $\beta(n,d,k)-g$, as does
$B(n,\mathcal{L},k)$ when $k>d-n(g-1)$. However, statements about smoothness
are not necessarily true. In particular, $B(2,K,k)$ has a new Brill-Noether
number

$\beta(2,K,k):=3g-3-\frac{k(k+1)}{2},$

and a new Petri map

$\mu_{E}^{K}:S^{2}H^{0}(E)\longrightarrow H^{0}(S^{2}E),$

arising from the symmetry $E\cong E^{*}\otimes K$. All components of
$B(2,K,k)$ have dimension at least $\beta(2,K,k)$ and, if $E\in
B(2,K,k)\setminus B(2,K,k+1)$, then $B(2,K,k)$ is smooth of this dimension at
$E$ if and only if $\mu_{E}^{K}$ is injective.

Given a vector bundle $F$ of rank $n^{\prime}$ and degree $d^{\prime}$, one
can define the twisted Brill-Noether loci $B(n,d,k)(F)$ by

$B(n,d,k)(F):=\\{E\in M(n,d)|h^{0}(E\otimes F)\geq k\\}$

and similarly for $\widetilde{B}(n,d,k)(F)$. There is a new Brill-Noether
number $\beta(n,d,k)(F)$ defined by

$\beta(n,d,k)(F):=n^{2}(g-1)+1-k(k-n^{\prime}d-nd^{\prime}+nn^{\prime}(g-1))$

and a new Petri map, defined as the composite of a multiplication map and a
trace map:

$\displaystyle\mu_{E_{F}}:H^{0}(F\otimes E)\otimes H^{0}(E^{*}\otimes
F^{*}\otimes K)$ $\displaystyle\longrightarrow H^{0}(\mbox{End}(F\otimes
E)\otimes K)$ $\displaystyle\longrightarrow H^{0}(\mbox{End}E\otimes K)$

with the same significance as before. The simplest examples of such loci are
generalised theta divisors, i.e. divisors of the form

$\\{E\in M(n,d)|h^{0}(E\otimes F)>0\\},$

where $F$ is a fixed vector bundle of rank $n^{\prime}$ such that
$\deg(E\otimes F)=nn^{\prime}(g-1)$. These have been studied extensively over
many years, but the extension of this to arbitrary twisted Brill-Noether loci
is more recent. Here we could also allow $V$ to vary in a family, but we shall
not pursue this further here.

One can extend this idea still further by choosing a homogeneous
representation
$\rho:\operatorname{GL}_{n}({\mathbb{C}})\to\operatorname{GL}(H)$, where $H$
is a vector space (this includes, for example, tensor power representations)
and defining

$B(\rho,n,d,k)(F):=\\{E\in M(n,d)|h^{0}(E_{\rho}\otimes F)\geq k),$

where $E_{\rho}$ is the vector bundle induced from $E$ by $\rho$. Again there
is a similar definition for $\widetilde{B}(\rho,n,d,k)(F)$. For all these
twisted Brill-Noether loci, there is an obvious concept of coherent system.

##  2\. Basic questions

Let $C$ be a smooth projective curve defined over $\mathbb{C}$. The following
are basic questions in the Brill-Noether theory of $C$.

######  Question 2.1.

For what values of $(n,d,k)$ is $B(n,d,k)$ non-empty?

######  Question 2.2.

What are the connected components of $B(n,d,k)$? In particular, for what
values of $(n,d,k)$ is $B(n,d,k)$ connected?

######  Question 2.3.

What are the irreducible components of $B(n,d,k)$? In particular, for what
values of $(n,d,k)$ is $B(n,d,k)$ irreducible?

######  Question 2.4.

What is the dimension (of each component) of $B(n,d,k)$?

######  Question 2.5.

Does $B(n,d,k)$ have singularities outside $B(n,d,k+1)$? Could $B(n,d,k)$ be
non-reduced?

Similar questions can be asked for $\widetilde{B}(n,d,k)$,
$B(n,\mathcal{L},k)$ and the twisted Brill-Noether loci. For
$G(\alpha;n,d,k)$, the first four questions make sense, but Question 5 needs
to be replaced by

######  Question 2.6.

Is $G(\alpha;n,d,k)$ smooth?

For $n=1$ (the classical case), the answers to the above questions are all
known (for general $C$) (see [ACGH85] for this and much more).

  * (i)

For any $C$, $G(1,d,k)$ and $B(1,d,k)$ are non-empty if $\beta(1,d,k)\geq 0$
and connected if $\beta(1,d,k)>0$.

  * (ii)

For general $C$, $G(1,d,k)$ is smooth of dimension $\beta(1,d,k)$ (in
particular, it is empty if $\beta(1,d,k)<0$) and it is irreducible if
$\beta(1,d,k)>0$.

  * (iii)

For general $C$, $B(1,d,k)$ has dimension $\beta(1,d,k)$ if

$0\leq\beta(1,d,k)\leq g,$

is irreducible if $\beta(1,d,k)>0$, and its singular set coincides with
$B(1,d,k+1)$ if $\beta(1,d,k)<g$.

Properties (ii) and (iii) follow from the fundamental fact that, for the
general curve $C$ of genus $g$, the Petri map $\mu_{E}$ is injective for every
line bundle $E$ on $C$. A curve with this property is said to be a Petri curve
(the terms Petri general and Brill-Noether-Petri are often used in the
literature, Brill-Noether general being reserved for a weaker concept).

For the twisted Brill-Noether loci $B(1,d,k)(F)$, all this remains true,
provided that, in (ii) and (iii), we take $F$ to be general as well as $C$
[Ghi83, Laz84, Tei14, Hir88] (see also [HHN21, Theorem 2.1]).

If $n\geq 2$, none of the above is true for all values of $(d,k)$ with the
possible exception of connectedness. In fact, the following question remains
(to my knowledge) open.

######  Question 2.7.

Is it true that, if $B(n,d,k)\neq\emptyset$ and $\beta(n,d,k)>0$, then
$B(n,d,k)$ is connected?

There are some cases in which there are good and expected results for
$B(n,d,k)$, for instance for $d\leq 2n$ [BGN97, Mer99a, Mer01] (hence also
$g\leq 3$) and hyperelliptic curves [BMNO00]. There are already considerable
gaps in our knowledge for non-hyperelliptic curves of genus $4$ [LN17]. In
general, the condition $\beta(n,d,k)\geq 0$ is not sufficient for non-
emptiness of $\beta(n,d,k)$ (see, for example, [BGN97]), nor is it necessary
even when $C$ is general (examples can be obtained using the results of
[Mer99b]). However there is a wide range of values for $(n,d,k)$ for which, if
$C$ is general, there exists a component of $B(n,d,k)$ of the expected
dimension and generically smooth; this was proved in [Tei91a] (see [CLT18] for
the smoothness) using degenerations and in [Mer99b] using elementary
transformations. There is an interesting question in connection with this.

######  Question 2.8.

Are the components of $B(n,d,k)$ constructed in [Tei91a] and [Mer99b] the
same?

There is a third construction in [BMNO00], which overlaps with those of
[Tei91a] and [Mer99b]. One can ask whether this also gives the same component
when all are defined.

######  Question 2.9.

What is a “good” definition for Brill-Noether general for rank $n$?

At first sight, this looks like a good question. However, since some of the
answers to the basic questions are negative even on the general curve, and
others are unknown, it is difficult to specify any specific conditions which
must be satisfied on the general curve except by looking at specific values
for $(d,k)$. This certainly allows us to state conditions which are necessary
for a curve to be called Brill-Noether general, and one can also find
sufficient conditions, but these are some distance apart.

######  Question 2.10.

What is a “good” definition of $\alpha$-stability for twisted coherent
systems?

This, on the other hand, is an excellent question. Recently, Schmitt has
proposed two definitions for coherent systems of type $(\rho,n,d,k)(F)$, where
$F$ is a line bundle. The obvious extension of the standard definition for
standard coherent systems is one of these, but presents some problems over the
permissible range for $\alpha$ [Sch19]. The second one is more complicated,
but fits well with Schmitt’s concept of decorated bundles and has some
interesting features [Sch21]. A key requirement for a good definition is that
it leads to a construction of moduli spaces.

##  3\. Clifford indices and gonalities

In determining the possible dimensions for the spaces of sections of line
bundles on $C$, the starting point is the classical theorems of Riemann-Roch
and Clifford. In higher rank, the Riemann-Roch Theorem generalises for any
vector bundle: if $E$ has rank $n$ and degree $d$, then

$h^{0}(E)-h^{0}(E^{*}\otimes K)=d-n(g-1).$

For Clifford’s Theorem, we have to restrict to semistable $E$, when we get,
for $0\leq d\leq n(2g-2)$,

$h^{0}(E)\leq\frac{d}{2}+n.$

Note that, if $E$ is semistable and $d<0$, $h^{0}(E)=0$, while, if
$d>n(2g-2)$, $h^{0}(E)=d-n(g-1)$.

These results depend only on the genus of $C$. To take the complex structure
of $C$ into account, we need more refined invariants. The first of these is
the Clifford index, defined as follows. For any vector bundle $E$ of rank $n$
and degree $d$,

$\operatorname{Cliff}(E):=\frac{1}{n}(d-2(h^{0}(E)-n))=\mu(E)-2\frac{h^{0}(E)}{n}+2.$

For $g\geq 4$, the Clifford indices of $C$ are then defined by

$\operatorname{Cliff}_{n}(C):=\min_{E}\left\\{\operatorname{Cliff}(E)\;\left|\begin{array}[]{c}E\;\mbox{semistable
of rank}\;n\\\ h^{0}(E)\geq 2n,\;\mu(E)\leq g-1\end{array}\right\\}.\right.$

Note that, using Serre duality, we can extend this definition to the full
range $0\leq d\leq n(2g-2)$. For $n=1$, we get the classical Clifford index,
which we denote here by $\operatorname{Cliff}_{1}(C)$. We can extend the
definition to $g\geq 2$ by defining $\operatorname{Cliff}_{n}(C)$ to be $0$
when $g=2$ and when $C$ is hyperelliptic of genus $3$, and to be $1$ when $C$
is non-hyperelliptic of genus $3$. (Clifford indices were defined in [LN10a],
where they were denoted by $\gamma_{n}^{\prime}$; the notation $\gamma_{n}$
has also been used, but this was used in [LN10a] with a different meaning.)

We can define still more refined invariants

$\operatorname{gon}_{n,k}:=\min\\{d|\exists\text{ semistable }E\text{ of rank
}n\text{ with }h^{0}(E)\geq k\\}.$

Classically, $\operatorname{gon}_{1,2}$ is known as the gonality of $C$ and we
shall refer to all the $\operatorname{gon}_{n,k}$ as gonalities. We shall also
define $\operatorname{gon}_{n,k}^{s}$ to be the corresponding quantities when
we replace “semistable” by “stable” in the definition.

Curves of genus $0$ or $1$ are excluded from the definition of Clifford index
and all gonalities are known in these cases, so we assume for the rest of this
section that $g\geq 2$.

It is conventional to represent the non-emptiness of $B(n,d,k)$ graphically by
plotting $\lambda:=\frac{k}{n}$ against $\mu$. On this diagram, the upper
bound given by Clifford’s Theorem is the line $\lambda=\frac{\mu}{2}+1$ (valid
for $0\leq\mu\leq 2g-2$). However, it is certainly possible to find smaller
upper bounds. Mercat has proposed the following problems.

######  Question 3.1.

Find a function $f$ (defined for $0\leq\mu\leq 2g-2$) as small as possible
such that all $\widetilde{B}(n,d,k)$ with $\lambda>f(\mu)$ are empty.

######  Question 3.2.

Find a function $h$ (defined for $0\leq\mu\leq 2g-2$) as large as possible
such that all $B(n,d,k)$ with $\lambda<h(\mu)$ are non-empty.

Precise answers are known for $g=2$, $g=3$ and hyperelliptic curves of genus
$4$ [LN17] and $5$ [LN16]; in fact, in these cases, all non-empty Brill-
Noether loci are known. For hyperelliptic curves of any genus, there are very
good results; in fact, if one weakens the last question to insist only that
$B(n,d,k)\neq\emptyset$ for some $(n,d,k)$ with $d=n\mu$ and $k=n\lambda$,
then again there is a precise answer [BMNO00]. For bielliptic curves, one can
take $h(\mu)=\frac{\mu}{2}$ [Ba98] and, over a certain part of the range for
$\mu$, one can also take $f(\mu)=\frac{\mu}{2}$ [Mer02]. On any curve, in the
range $0<\mu<2$, we have $f(\mu)=h(\mu)=\frac{1}{g}(\mu-1)+1$ [BGN97, Mer99a],
with a corresponding result for $2g-4<\mu<2g-2$.

For fixed $n$, the best known general result for $f$ is
$f(\mu)=\frac{\mu-\operatorname{Cliff}_{n}(C)}{2}+1$ given by the Clifford
index, and, for $h$, the bound of Teixidor [Tei91a] and Mercat [Mer99b] as
extended in [BMNO00]. The bound for $f$ can certainly be improved (see, for
example, the diagrams for $4\leq g\leq 6$ in [LN17, LN16, LN18] and the
example in [Li19] for $g=41$). All of these bounds are piecewise linear (but
not necessarily continuous). However, for a large class of curves on K3
surfaces, a recent result [FL21] establishes the quadratic bound
$f(\mu)<1+\frac{g}{4(g-1)^{2}}\mu^{2}+\frac{1}{g}$ for $\mu\leq g-1$ and the
positivity of the Brill-Noether number also takes a quadratic form (although
it does not provide a bound). It is even possible that the “true” bounds are
fractal.

For twisted Brill-Noether loci, non-emptiness and smoothness results somewhat
analogous to those of [Tei91a] and [Mer99b] have been obtained in [HHN21].
However, there are extra numerical conditions required in the proof.

######  Question 3.3.

Can the additional conditions in [HHN21, Theorem 1.2] for the non-emptiness of
$B(n,d,k)(F)$ and the existence of a component which is generically smooth of
dimension $\beta(n,d,k)(F)$ be relaxed?

We have clearly $\operatorname{Cliff}_{n}(C)\leq\operatorname{Cliff}_{1}(C)$
and indeed $\operatorname{Cliff}_{am}(C)\leq\operatorname{Cliff}_{m}(C)$ for
any positive integer $a$. The following question seems natural.

######  Question 3.4.

Is it true that $\operatorname{Cliff}_{n}(C)\leq\operatorname{Cliff}_{m}(C)$
when $n\geq m$?

An implication of an affirmative answer to this question for a given $n$ is
that every bundle computing $\operatorname{Cliff}_{n}(C)$ would then be
primitive (i.e. both $E$ and $E^{*}\otimes K$ are generated) [LN10b]. However,
the answer can be negative; for $n=3$, there are examples in [LN13c].

In [Mer02], Mercat made a conjecture proposing upper bounds for $h^{0}(E)$ in
the range $1\leq\mu(E)\leq 2g-3$. The main implication of this conjecture can
be stated as

######  Conjecture 1.

$\operatorname{Cliff}_{n}(C)=\operatorname{Cliff}_{1}(C).$

This has come to be known as Mercat’s Conjecture. The conjecture was
originally made for general $C$, but it makes sense for any $C$.

######  Question 3.5.

For which values of $n\geq 2$ and for which curves does Mercat’s Conjecture
hold? More generally, find a good lower bound for
$\operatorname{Cliff}_{n}(C)$.

The conjecture always holds if $\operatorname{Cliff}_{1}(C)\leq 2$, i.e. for
hyperelliptic curves, trigonal curves, tetragonal curves, including bielliptic
curves, and smooth plane curves of degree $6$. It has been proved recently
that it holds for arbitrary smooth plane curves [FL21]. For the class of
curves on K3 surfaces mentioned above, the exact value of
$\operatorname{Cliff}_{n}(C)$ is calculated for $g\geq n^{2}$ and, for $n\geq
3$, it is strictly less than $\operatorname{Cliff}_{1}(C)$ except for $n=3$,
$g=10,14$. When $n=2$, the conjecture also holds for general $\ell$-gonal
curves with $\ell\geq 5$ and $g\geq 4\ell-4$ (possibly for smaller $g$ as
well) [LN13b], for arbitrary $\ell$-gonal curves with $g\geq(\ell-1)(2\ell-4)$
[FO12], and for $C$ general [BaF18]. However, there exist $C$ (even having
maximal Clifford index
$\operatorname{Cliff}_{1}(C)=\lfloor\frac{g-1}{2}\rfloor$) for which it does
not hold [FO11, LN11, FO12]; there are indeed Petri curves for which
$\operatorname{Cliff}_{2}(C)<\operatorname{Cliff}_{1}(C)$ [LC13]. Moreover,
$\operatorname{Cliff}_{2}(C)\geq\min\\{\operatorname{Cliff}_{1}(C),\frac{\operatorname{gon}_{1,5}-4}{2}\\}$
[LN10a] and this bound can be attained for any value of
$\operatorname{Cliff}_{1}(C)$ [LN11].

For $n=3$, the conjecture fails for general $C$ of genus $9$ or $11$ [LMN12]
and for any smooth curve lying on a K3 surface with $g=9$ or $g\geq 11$ and
$\operatorname{Cliff}_{1}(C)=\left\lfloor\frac{g-1}{2}\right\rfloor$ [FO12].
For the best bound known to me which covers all curves, see [LN13c]. In
particular, if $\operatorname{Cliff}_{1}(C)=3$, we have
$\frac{8}{3}\leq\operatorname{Cliff}_{3}(C)\leq 3$.

######  Question 3.6.

Do there exist curves with $\operatorname{Cliff}_{1}(C)=3$ and
$\operatorname{Cliff}_{3}(C)=\frac{8}{3}$?

There are severe restrictions on the existence of such a curve [LN13d]; in
particular, one must have $9\leq g\leq 12$.

Turning now to looking at gonalities, we have the following general question.

######  Question 3.7.

What are the values of $\operatorname{gon}_{n,k}$,
$\operatorname{gon}^{s}_{n,k}$ for a general curve? When is
$\operatorname{gon}_{n,k}=\operatorname{gon}^{s}_{n,k}$?

All values of $\operatorname{gon}_{1,k}=\operatorname{gon}_{1,k}^{s}$ are
known for Petri curves, and also for hyperelliptic curves, trigonal curves,
general tetragonal curves, bielliptic curves and smooth plane curves. For
$n\geq 2$, the values of $d_{n,k}$ and $d_{n,k}^{s}$ are known for $g=2$ and
$g=3$ and for hyperelliptic curves of genus $4$ or $5$.

The gonality $\operatorname{gon}_{n,2n}$ is particularly interesting as
bundles of this gonality are the first candidates for computing the Clifford
index. This happens if and only if
$d_{n,2n}=n(\operatorname{Cliff}_{n}(C)+2)$. In fact, it is easy to see that,
on a general curve of genus $g$, there exist strictly semistable bundles of
any rank $n\geq 2$ and degree
$d=n(\operatorname{Cliff}_{1}(C)+2)=n\lfloor\frac{g+3}{2}\rfloor$, so this
certainly holds if Mercat’s Conjecture holds, in particular if $n=2$. On the
other hand, it is not obvious that there exist stable bundles with this
property. In general, one can ask the following question.

######  Question 3.8.

Which bundles compute the Clifford index? Which, if any, stable bundles do so?

For hyperelliptic and trigonal curves, the bundles computing
$\operatorname{Cliff}_{n}(C)$ are all strictly semistable. For hyperelliptic
curves, they are all direct sums of copies of the hyperelliptic line bundle
[Re98]; for trigonal curves of genus $g\geq 5$, they are direct sums of copies
of the unique trigonal line bundle [LN13a, Hof21]. For $n=2$,[LN13b] contains
several further results as follows. The only bundle computing
$\operatorname{Cliff}_{2}(C)=\operatorname{Cliff}_{1}(C)$ for a smooth plane
curve of degree $\delta\geq 5$ is $H\oplus H$, where $H$ is the hyperplane
bundle. For the general tetragonal curve of genus $g\geq 27$, the only bundle
computing $\operatorname{Cliff}_{2}(C)=\operatorname{Cliff}_{1}(C)$ is
$Q\oplus Q$, where $Q$ is the unique tetragonal bundle. For $\ell\geq 5$, a
similar result holds for the general $\ell$-gonal curve of genus
$g>\max\\{3\ell^{2}-8\ell+7,46\\}$. These lower bounds can certainly be
improved.

In rank $2$, for $C$ general, we have $d_{2,4}=d_{2,4}^{s}=g+3$ if $g$ is odd
[Tei07, FO11] and $d_{2,4}=g+2$ if $g$ is even. This leaves the following
question.

######  Question 3.9.

Does there exist a stable bundle of rank $2$ and degree $g+2$ with $h^{0}=4$
on a general curve of even genus $g$?

The answer is negative for $g\leq 10$, although there do exist Petri curves of
genus $g=10$ for which $B(2,12,4)\neq\emptyset$ [GMN11]. To my knowledge, the
answer to the question is not known for $g\geq 12$. To complete the picture
for bundles of rank $2$ computing $\operatorname{Cliff}_{2}(C)$, we can ask

######  Question 3.10.

Does there exist a bundle $E$ computing $\operatorname{Cliff}_{2}(C)$ on a
general curve with $h^{0}(E)>4$?

It is shown in [BaF18] that there are no such bundles on a curve of even genus
$g\geq 10$, while, on a curve of odd genus $g\geq 15$, the only possibility is
for a semistable bundle $E$ of degree $2g-2$ with $h^{0}(E)=\frac{g+3}{2}$. It
is not known whether any such bundle $E$ exists, but, if it does, $\det
E\not\cong K$. For $6\leq g\leq 9$, $g=11$ and $g=13$, there are bundles of
determinant $K$ with the requisite number of sections. For $g=4$ and $g=5$,
there are no candidates for such a bundle.

Much less is known when $n=3$.

######  Question 3.11.

What is the value of $\operatorname{Cliff}_{3}(C)$ for a general curve $C$ of
genus $g$? Which bundles compute $\operatorname{Cliff}_{3}(C)$?

Certainly $\operatorname{Cliff}_{3}(C)=\operatorname{Cliff}_{1}(C)$ for $g\leq
6$, but (to my knowledge) the answer is not known for $g=7$ (here
$\frac{8}{3}\leq\operatorname{Cliff}_{3}(C)\leq\operatorname{Cliff}_{1}(C)=3$).
For $g=9$, we have $\operatorname{Cliff}_{1}(C)=4$ and
$\operatorname{Cliff}_{3}(C)=\frac{10}{3}$ [LMN12, LN13c]; moreover, in this
case, $\operatorname{Cliff}_{3}(C)$ is computed by a stable bundle of degree
$16$ with $h^{0}=6$. For $g=11$, it was shown in [LMN12, LN13c] that
$4\leq\operatorname{Cliff}_{3}(C)<\operatorname{Cliff}_{1}(C)=5$; recently,
the exact value $\operatorname{Cliff}_{3}(C)=\frac{14}{3}$ was determined in
[FL21]. In general, the results of [FL21] imply that, for $C$ general of genus
$g\geq 9$, we have

$\frac{2}{3}(g-1)-\frac{2}{3}\left\lfloor\frac{g}{3}\right\rfloor\leq\operatorname{Cliff}_{3}(C)\leq\operatorname{Cliff}_{1}(C)=\left\lfloor\frac{g-1}{2}\right\rfloor.$

There are many questions that can be asked for special curves. An interesting
case is that of smooth plane curves of degree $\delta\geq 7$. As seen above,
we know that
$\operatorname{Cliff}_{n}(C)=\operatorname{Cliff}_{1}(C)=\delta-4$, but, to my
knowledge, the following question remains open.

######  Question 3.12.

For $C$ a smooth plane curve of degree $\delta\geq 7$, is it true that the
only bundles computing $\operatorname{Cliff}_{n}(C)$ are direct sums of copies
of the hyperplane bundle $H$?

##  4\. Fixed determinant

We again assume that $g\geq 2$. Let $\mathcal{L}$ be a line bundle of degree
$d$. The naïve lower bound $\beta(n,d,k)-g$ for the dimension of a component
of $G(\alpha;n,\mathcal{L},k)$ (and, when $k\geq d-n(g-1)$, that of a
component of $B(n,\mathcal{L},k)$) cannot always be attained, although we do
expect it to apply for the general $\mathcal{L}$ of any fixed degree; for
circumstances in which it can be attained in the case $n=2$, see [Tei10].
Based on work of Osserman [Oss13a, Oss13b], we pose the following question.

######  Question 4.1.

Is it true that the dimension of $G(\alpha,n,\mathcal{L},k)$ at $(E,V)$ is at
least $\beta(n,d,k)-g+h^{1}(\mathcal{L})\binom{k}{n}$? Can this lower bound be
attained?

Following Osserman’s work, an affirmative answer to the first question has
been given by Zhang [Zh17] for any Petri curve when $(E,V)$ is generically
generated. Osserman also gave some examples in which the bound is attained and
it is shown in [GN14] that the bound is attained for $k\leq n+1$ under a mild
generality condition on $\mathcal{L}$. That some such condition is necessary
is already implicit in Osserman’s work. For $k>n+1$, the second question
remains open.

For the rest of this section, we fix $n$ to be $2$. The case of $B(2,K,k)$ is
particularly interesting. As indicated in Section 1, there is a modified
Brill-Noether number $\beta(2,K,k):=3g-3-\frac{k(k+1)}{2}$ in this case. Many
years ago, Bertram and Feinberg [BF98] conjectured that, on a general curve
$C$, $B(2,K,k)\neq\emptyset$ if and only if $\beta(2,K,k)\geq 0$; the question
was also asked by Mukai in the form of two problems [Mu95, Mu01]. The “only
if” part of this conjecture was proved by Teixidor [Tei08a], who showed that
the Petri map $\mu_{E}^{K}$ is injective for all semistable $E$ of rank $2$
and determinant $K$. It therefore remains to consider the following question.

######  Question 4.2.

Is it true that $B(2,K,k)\neq\emptyset$ whenever $\beta(2,K,k)\geq 0$?

This question has been attacked by degeneration methods [Tei04, Zh16] and,
more recently, by cohomological methods [LNP16], which reduce the problem to a
(complicated) combinatorial one. In fact, this shows that, if $B(2,K,k)$ is
non-empty for one value of $g$, then this also holds for all greater values of
$g$. The answer is now known to be affirmative for $k\leq 9$ and,
correspondingly, for $g\leq 19$ (for the case $k=8$, $g=13$, see [FJP21];
$k=9$, $g=16$ is even more recent and, so far, unpublished). In the case
$k=10$, we can reduce the problem to two particular values of $g$. In fact,
$B(2,K,10)\neq\emptyset$ for $g\geq 22$ (for $g=22$, see [CGZ21]).

######  Question 4.3.

Is $B(2,K,10)$ non-empty for $g=20$ or $g=21$?

It is also known that $B(2,K,k)\neq\emptyset$ whenever $\beta(2,K,k)\geq 0$
for $k=11$, $k=15$, $k=16$, $k=20$ and $k=24$; in fact, the calculations in
[LNP16] and the papers mentioned above give complete answers for all genera
$g<110$ except for $20,21,27,28,32-36,52,58,65,66,78,86-88,93-96$. The
expectation is that these are not genuine exceptions.

A particularly interesting fact is that $\beta(2,K,k)$ can be greater than
$\beta(2,2g-2,k)$, so there is the possibility that $B(2,K,k)$ is contained in
a superabundant component of $B(2,2g-2,k)$ (i.e., one of dimension greater
than $\beta(2,2g-2,k)$). The first time this happens is for $g=5$ [New18].
This can be generalised to $B(2m,m(2g-2),k)$ by considering bundles of rank
$2m$ with a $K$-valued symplectic structure $E\cong E^{*}\otimes K$ [BH21];
the required lower bound for the genus for this construction to give a
superabundant component is $g=50$.

We can also consider $B(2,\mathcal{L},k)$ when $\mathcal{L}\not\cong K$. This
case is of course covered by Question 4.1. It is possible to study
$B(2,\mathcal{L},k)$ using cohomological methods similar to those of [LNP16].
This is done in the case when $d$ is odd in [LNS15]. As for the case
$\mathcal{L}=K$, we reduce the problem (at least for general $\mathcal{L}$) to
a combinatorial one.

##  5\. Butler’s Conjecture

Suppose that $(E,V)$ is a generated coherent system on $C$ of type $(n,d,k)$
with $k>n$. We define a bundle $D_{E,V}$ by the exact sequence

$0\longrightarrow D_{E,V}^{*}\longrightarrow
V\otimes{\mathcal{O}}_{C}\longrightarrow E\longrightarrow 0.$

Provided $h^{0}(E^{*})=0$ (which is certainly the case if $E$ is semistable),
dualising this sequence gives rise to a coherent system
$D(E,V):=(D_{E,V},V^{*})$. This is known as the dual span construction. The
bundle $D_{E,V}^{*}$ is often referred to as a kernel bundle or syzygy bundle
and denoted by $M_{E,V}$. When $V=H^{0}(E)$, we write $D_{E}$ and $M_{E}$ for
$D_{E,V}$ and $M_{E,V}$. Now let $S_{0}(n,d,k)$ be the open subset of
$G_{0}(n,d,k)$ consisting of generated coherent systems. In [Bu97], D. C.
Butler made the following conjecture.

######  Conjecture 2.

Let $C$ be a general curve of genus $g$ and $n,d,k$ positive integers. Then,
for a general $(E,V)\in S_{0}(n,d,k)$, $D(E,V)\in S_{0}(k-n,d,k)$. Moreover,
$S_{0}(n,d,k)$ and $S_{0}(k-n,d,k)$ are birational.

Here, by saying that $(E,V)$ is general, we mean that it belongs to some dense
open subset of $S_{0}(n,d,k)$. Butler’s Conjecture is often stated in the form
of the following question.

######  Question 5.1.

For a general $(E,V)\in S_{0}(n,d,k)$ on a general curve $C$ of genus $g\geq
1$, is $D_{E,V}$ semistable?

In general, this is slightly weaker than Conjecture 2, but is equivalent to it
when $n=1$; note that, in this case, the hypothesis $(E,V)\in S_{0}(1,d,k)$ is
equivalent to the simple assertion that $(E,V)$ is generated. This case has
been examined by a number of authors and finally proved to be true in [BBN15]
. In fact, when $n=1$ and $g\geq 3$, $D_{E,V}$ is very frequently stable,
rather than just semistable. This is always true if $k\leq 5$ [BBN08] and also
if $k\geq 6$ and $g\geq 2k-6$. For $g=3$, the problem of stability would be
completely solved by an answer to the following question.

######  Question 5.2.

Suppose that $C$ is a non-hyperelliptic curve of genus $g=3$. Is it true that,
for general $(E,V)\in G(1,2k,k)$ with $k\geq 6$, $D_{E,V}$ is stable?

An interesting concept, which dates back to Mumford, is that of linear
stability for a generated rank $1$ coherent system $(E,V)\in G(1,d,k)$. Linear
stability is linked with the Chow stability of the image of $C$ in
${\mathbb{P}}^{k-1}$ under the map defined by $(E,V)$; for some recent work on
this link, see [BT16]. Stability (semistability) of $D_{E,V}$ implies linear
stability (semistability) of $(E,V)$, but the converse is not clear.

######  Question 5.3.

Suppose that $(E,V)\in G(1,d,k)$ is generated and linearly stable
(semistable). Is it true that $D_{E,V}$ is stable (semistable)?

In [MS12], this was shown to be true in many cases and some counterexamples
were given. When $V=H^{0}(E)$, the question has been answered in the
affirmative for Petri curves and hyperelliptic curves [CT18], but counter-
examples are known for smooth plane curves of genus $7$ [CMT20].

There is another condition on bundles, namely that of cohomological stability.
In fact, cohomological semistability is equivalent to semistability, but
cohomological stability is stronger than stability. The concept has been used
in proving Butler’s Conjecture in some cases (see, for example, [MS12]).

Note that any generated coherent system of type $(n,d,n+1)$ is of the form
$D(E,V)$ for some generated $(E,V)$ of type $(1,d,n+1)$. If one can prove that
none of these $D(E,V)$ is stable, it follows that $B(n,d,n+1)$ contains no
generated bundles.

######  Question 5.4.

Let $C$ be a general curve. Do there exist non-empty Brill-Noether loci
$B(n,d,n+1)$ which contain no generated bundles?

On special curves, such loci exist even for $n=1$. On a smooth plane curve of
degree $7$, the example constructed in [CMT20] shows that $B(2,15,3)$, which
is non-empty, contains no generated elements. So far as I am aware, this
particular example has not previously been observed.

For higher rank, much less is known. Conjecture 2 is true on any curve of
genus $g\geq 2$ when $V=H^{0}(E)$ and $d>2ng$ (see [Bu94],[Mer99a],
[BMGNO19]). It is also proved in [BMGNO19] that the conjecture holds when
$V=H^{0}(E)$ and $d=2ng$ provided $C$ is not hyperelliptic. Another
interesting fact proved in [BMGNO19] is that, if $(E,V)\in S_{0}(n,d,k)$ and
$D(E,V)\in S_{0}(k-n,d,k)$, then $(E,V)\in G(\alpha;n,d,k)$ for all
$\alpha>0$, or, in other words, $(E,V)\in U^{s}(n,d,k)$; this is a possible
source of counter-examples to Butler’s Conjecture.

######  Question 5.5.

Does Conjecture 2 hold for $(E,V)\in S_{0}(2,d,4)$?

This is the first case to consider for $n\geq 2$ where $V\neq H^{0}(E)$. The
conjecture is shown to be true for a range of values of $d$ when $C$ is a
general curve of genus $g\geq 3$ in [BMGNO19], but there are many other values
to be considered.

##  6\. Coherent systems on the projective line

It is well known that the only vector bundles on ${\mathbb{P}}^{1}$ are direct
sums of line bundles ${\mathcal{O}}_{C}(a)$. This means that the Brill-Noether
theory is completely trivial. However, there is a very interesting theory of
coherent systems (see [LN04, LN07, LN08a, NT21], also [PP08], where the
closely related concept of holomorphic triples is studied).

The spaces $G(\alpha;n,d,k)$ are always smooth and irreducible of dimension
$\beta(n,d,k)$ whenever they are non-empty. Moreover, the set
$I(n,d,k):=\\{\alpha|G(\alpha;n,d,k)\neq\emptyset\\}$ is always an open
interval (possibly semi-infinite) and all $G_{i}(\alpha;n,d,k)$ are
birational.

######  Question 6.1.

When is $I(n,d,k)$ non-empty? If $I(n,d,k)\neq\emptyset$ and we write
$I(n,d,k)=]\alpha_{m},\alpha_{M}[$, where $\alpha_{M}$ can be $\infty$, what
are the values of $\alpha_{m}$ and $\alpha_{M}$?

In many cases, the answer is known; for example, if $d=an$ for some integer
$a\geq 2$ and $k\geq n$, then $I(n,d,k)\neq\emptyset$ if and only if
$\beta(n,d,k)=k((a+1)n-k)-n^{2}+1\geq 0$. Moreover, if this condition holds,
then $I(n,d,k)=]0,\infty[$. A complete answer is also known when $k\leq 3$.
For $k<n$, there is a known upper bound for $\alpha_{M}$, so
$\alpha_{M}\neq\infty$. For $k\geq n$, there are conjectures concerning the
non-emptiness of $I(n,d,k)$ in [NT21] and, in particular, we can ask the
following question.

######  Question 6.2.

Suppose that $d=an-t$ with $a\geq 2$, $1\leq t\leq n-1$ and $k\geq n$. If
$G(\alpha;n,d,k)$ is non-empty for some $\alpha$, is it true that
$\alpha_{M}=\infty$?

I know of no cases in which the answer is no, but there is an indication for
possible counter-examples in [NT21].

######  Question 6.3.

Can one determine the flip loci for coherent systems of type $(n,d,k)$ on
${\mathbb{P}}^{1}$ and use this information to compute Hodge and Poincaré
polynomials of $G_{i}(n,d,k)$?

In the case $k=1$, this is done in [LN08a] and the information is sufficient
to determine the Hodge polynomials of the smooth projective varieties
$G_{i}(n,d,1)$. In the general case, the description of the flip loci in
[BGMN03] works for the case $g=0$, but these loci are more complicated than in
the case $k=1$ and it has not so far been possible to carry out the
cohomological calculations.

##  7\. Coherent systems on elliptic curves

When $g=1$, the basic questions for $G(\alpha;n,d,k)$ are fully answered in
[LN05] (see also [PP07] for holomorphic triples). Precise conditions for non-
emptiness are given and are entirely as expected, and all non-empty
$G(\alpha;n,d,k)$ are smooth and irreducible of the expected dimension. As
with ${\mathbb{P}}^{1}$, the set
$I(n,d,k):=\\{\alpha|G(\alpha;n,d,k)\neq\emptyset\\}$ is always an open
interval and all $G_{i}(\alpha;n,d,k)$ are birational. In fact, if it is non-
empty,

$I(n,d,k)=\begin{cases}]0,\frac{d}{n-k}[&\text{ if }k<n\\\ ]0,\infty[&\text{
if }k\geq n.\end{cases}$

There is a very interesting use of Fourier-Mukai transforms in [HT08], which
shows that $G_{0}(n,d,k)\cong G_{0}(n+ad,d,k)$ (and $G_{L}(n,d,k)\cong
G_{L}(n+ad,d,k)$ if $k<n$) for all integers $a$. In particular, the birational
type of $G(\alpha;n,d,k)$ depends only on $n\bmod d$. The following question
is raised in [LN09].

######  Question 7.1.

Let $\mathcal{L}$ be a line bundle of degree $d$. Is
$G(\alpha;n,\mathcal{L},k)$ a rational variety?

This is proved to be true when $\gcd(n,d)=1$ and in some other cases in
[LN09].

The main question remaining is the analogue of Question 6.3.

######  Question 7.2.

When $g=1$, can one determine the flip loci for coherent systems of type
$(n,d,k)$ and use this information to compute Hodge and Poincaré polynomials
of $G_{i}(n,d,k)$?

For some computations in this direction, covering also the fixed determinant
case, see [LN09]. In particular, the Hodge polynomial is computed for all
$G_{i}(2+ad,d,1)$.

The results of [LN05] can be used to obtain results on coherent systems on
bielliptic curves [Ba06a].

##  8\. Coherent systems for $g\geq 2$

We now assume that $g\geq 2$. There is a version of Clifford’s Theorem for
$\alpha$-semistable coherent systems $(E,V)$ of type $(n,d,k)$ (just replace
$h^{0}(E)$ by $k$ in the classical statement) [LN08b]. There are also obvious
definitions of Clifford indices $\operatorname{Cliff}_{\alpha,n}(C)$, but I do
not know of any work on the subject outside that of [LN08b].

######  Question 8.1.

What values can $\operatorname{Cliff}_{\alpha,n}(C)$ take on a curve with
specified $\operatorname{Cliff}_{1}(C)$?

Somewhat related to this question, we have

######  Question 8.2.

Find a good bound (either dependent on or independent of $\alpha$) for
$h^{0}(E)$ when $(E,V)\in G(\alpha;n,d,k)$.

It is known that such a bound exists [BGMN03], but this bound can certainly be
improved.

The major new feature for coherent systems is the variation of the moduli
spaces with $\alpha$. Recall that $U^{s}(n,d,k)$ consists of those $(E,V)$
which are $\alpha$-stable for all $\alpha>0$. The following question was
raised by Ballico in [Ba06a].

######  Question 8.3.

What is the smallest integer $d^{\prime}_{n,k}$ such that, for all $d\geq
d^{\prime}_{n,k}$, there exists a coherent system $(E,V)\in U(n,d,k)$ with
$(E,V)$ generated and both the Petri map and the natural map
$\psi:\bigwedge^{n}(V)\to H^{0}(\det E)$ injective?

Ballico proved that $d^{\prime}_{n,k}$ exists and gave a crude estimate for
it. The significance of the injectivity of the natural map $\psi$ is that it
is equivalent to saying that the image of $C$ under the map to projective
space defined as the map to $\operatorname{Gr}(k,n)$ given by $(E,V)$ followed
by the Plücker embedding is non-degenerate. It is clear that
$d^{\prime}_{n,k}\geq d_{n,k}$. The map $\psi$ is investigated in [Tei98] in
the case where $V=H^{0}(E)$ and injectivity is proved for general $E$ when
$d=ng+1$ and $d=ng+2$. It follows that, in these cases, $\psi$ is injective
for all $(E,V)$ with $V$ any subspace of $H^{0}(E)$. This suggests the
possibility that $d^{\prime}_{n,n+1}=ng+1$ and $d^{\prime}_{n,n+2}=ng+2$.

A somewhat similar question (stated as a “Model Theorem”) was proposed in
[New11].

######  Question 8.4.

Suppose that $n\geq 2$. Does there exist an integer $d_{0}(n,k)$ with the
following properties

  * (a)

$G(\alpha;n,d,k)\neq\emptyset$ if and only if $\alpha>0$, $(n-k)\alpha<d$ and
$d\geq d_{0}$;

  * (b)

$B(n,d,k)\neq 0$ if and only if $d\geq d_{0}$;

  * (c)

if $d\geq d_{0}$, then $U(n,d,k)\neq\emptyset$?

For $k<n$, this is true and $d_{0}(n,k)=n-g(n-k)\geq 0$ [BGN97, Tei08b,
BGMMN09]; moreover, if $d\leq 2n$, all the non-empty moduli spaces are
irreducible of the expected dimension [BGMMN09]. More generally, if we drop
the “only if” from (a) and (b) and slightly modify (b), this is again true
with the same value of $d_{0}(n,k)$ as is given by Teixidor’s bound mentioned
in connection with Question 3.2 [Tei08b].

For $k<n$, the moduli space $G_{L}(n,d,k)$ is described explicitly in [BG02]
(see also [BGMN03]). There are also good results for $k=n$ and $k=n+1$ and
some information for $k>n+1$. [BGMN03] contains also a description of the flip
loci and estimates of their codimensions. If both flip loci have positive
codimension, then there exist $(E,V)$ belonging to both $G_{i-1}(n,d,k)$ and
$G_{i}(n,d,k)$. Some cases for low values of $k$ are worked out in detail.
Very much related to this is the following question.

######  Question 8.5.

Suppose that $C$ is general and $k>n$. Is it true that, if
$G(\alpha;n,d,k)\neq\emptyset$ for some $\alpha>0$, then
$G_{L}(n,d,k)\neq\emptyset$?

One expects that this question usually has an affirmative answer, but there
are counter-examples [BGMMN09]. This shows in particular that the answer to
Question 8.4 may also be negative. Without any assumption about $C$, one can
also ask the following question, which is simpler than the previous two
questions.

######  Question 8.6.

When is $U^{s}(n,d,k)$ $(U(n,d,k))$ non-empty?

Papers which address this question directly include [BP08, BO09a, BO09b,
BMGNO19, BM20]. When $C$ is general and $k=n+1$, the problem is related to
Butler’s Conjecture and is completely solved for $U^{s}(n,d,n+1)$ and solved
in most cases for $U(n,d,n+1)$ [BBN15]. For $n=2$, $k=4$, see [Tei07, FO11,
BMGNO19]; here, $U(2,d,4)$ is non-empty for $g\geq 4$ and $d\geq g+3$ and this
is best possible for $g$ odd. For $g$ even, $U^{s}(2,g+2,4)=\emptyset$ (hence
also $U(2,g+2,4)=\emptyset$) for $g\leq 10$. The following question (a version
of Question 3.9) remains.

######  Question 8.7.

For a general curve $C$ of even genus $g\geq 12$, is $U^{s}(2,g+2,4)$ non-
empty?

Finally, we consider the question of a more detailed study of the wall-
crossings, leading to a comparison of the cohomology of $G_{i-1}(n,d,k)$ and
that of $G_{i}(n,d,k)$ and hopefully, by induction, relating the cohomology of
$G_{0}(n,d,k)$ to that of $G_{L}(n,d,k)$.

######  Question 8.8.

Can one obtain an expression for the change in the Hodge-Deligne polynomial
(or Poincaré polynomial) when crossing the critical value $\alpha_{i}$?

We use the term Hodge-Deligne polynomial because of the possibility of
singularities in the moduli spaces. We have already discussed the case $g=0$
(Question 6.3, where there are complete results for $(n,d,1)$, and $g=1$
(Question 7.2). For type $(2,\mathcal{L},1)$, with $\mathcal{L}$ a line bundle
of degree $d$, this has been worked out by Thaddeus [Th94], and it is easy to
deduce the results for type $(2,d,1)$. In this case, the wall-crossings are
genuine flips, which makes the calculations easier. These results were
reproved in [MOV07]. The geometry of flips was discussed in the nicest case in
[BGMMN07] and results obtained for type $(n,d,n-2)$ with $n\geq 3$ and
critical values close to the upper bound $\frac{d}{2}$ for $\alpha$. For
$(3,d,1)$, see [Mun08]. The case $(4,d,1)$ is partially covered in the thesis
of M. Tommasini and a start made on $(2,d,2)$ (unpublished - I hope that some
of this work might be published soon). The related topic of triples of rank
$(2,2)$ is discussed in [MOV09].

##  9\. Final remarks

In this section, we look at some topics not covered above.

######  Remark 9.1.

One can study Brill-Noether loci for singular curves; the definition of
coherent systems in [KN95] works for any polarised curve (connected scheme of
pure dimension $1$ with a polarisation). Most work has concentrated on the
case of a curve with nodal (or possibly cuspidal) singularities. Brill-Noether
theory on irreducible nodal and cuspidal curves has been studied, for example,
in [Bh07, BhS13]. In particular, the results of [BGN97] are generalised in
[Bh07], while kernel bundles are discussed in [BhS13], including a proof of
semistability (stability) in the case $d\geq 2ng$ ($d>2ng$) (compare [Bu94]);
[BhS13] also contains generalisations of the results of [Mer99a], [Mer01].
Coherent systems on integral curves are discussed in [Ba06b, BaP07] and on
irreducible nodal curves in [Bh09], where results from [BG02] and [BGMN03] are
generalised, including a description of $G_{L}(\alpha;n,d,k)$ when $k\leq n$.
The case of a nodal curve of (arithmetic) genus $1$ is discussed in [Bh11].
Open questions are in general similar to those for smooth curves. One problem
which has been solved in the smooth case, but remains open in the nodal case
is the following.

######  Question 9.1.

For a general $(E,V)\in G(1,d,k)$ on a general nodal curve, is $D_{E,V}$
semistable?

The paper [BhP14] is interesting in its own right, but contains results which
are necessary for solving this problem.

For a reducible nodal curve, the components of the moduli space $M(n,d)$ were
determined in [Tei91b, Tei95] together with a useful criterion for the
existence of stable bundles. For recent work on coherent systems on reducible
nodal curves, see [BrF20b, BrF21]. The failure of Butler’s conjecture on a
reducible nodal curve with $2$ components and one node is discussed in
[BrF20a].

######  Remark 9.2.

The constructions in [KN95] also work in finite characteristic. There is no
doubt that much of the theory applies in that case, but, to my knowledge,
there has been no systematic work on this.

######  Remark 9.3.

If $E$ is a bundle of rank $n$, one can define the Segre invariant of rank $r$
in terms of the maximal degree of a subbundle of rank $r$. There is
substantial study of these invariants, including the stratifications of
$M(n,d)$ which they determine [BL98, RT99].

######  Question 9.2.

How are the Segre stratifications related to the stratification induced by the
Brill-Noether loci in $M(n,d)$?

Segre stratifications for the moduli space of coherent systems are defined in
[Roa20]. These should give information on Question 9.2 and can also be used to
describe certain wall-crossings [Roa20].

######  Remark 9.4.

The problem of Brill-Noether loci in rank $1$ for general curves of fixed
gonality $m:=d_{1,2}$ has recently been solved [LLV20](following [Pfl17,
CPJ19, Lar20]). The methods involve degeneration to a chain of elliptic curves
and some intricate combinatorial computations using the splitting type of the
direct images of line bundles under the natural $m$-fold covering
$C\to{\mathbb{P}}^{1}$.

######  Question 9.3.

Study the Brill-Noether loci and coherent systems in higher rank for a curve
$C$ through the morphism $C\to{\mathbb{P}}^{1}$ induced by a line bundle of
degree $d_{1,2}$.

In view of the somewhat complicated arguments of [LLV20], this looks a hard
problem in general. One could start with the trigonal case, where the rank-$1$
situation has been well understood for some time.

######  Remark 9.5.

One can ask for Torelli theorems for Brill-Noether loci and moduli spaces of
coherent systems, in other words, whether these loci determine the curve.

######  Question 9.4.

To what extent do Brill-Noether loci and moduli spaces of coherent systems
determine the curve $C$?

## References

  * [ACGH85] E. Arbarello, M. Cornalba, P. A. Griffiths and J. Harris: _Geometry of Algebraic Curves I_. Grundlehren math. Wiss. 267, Springer, New York, 1985.
  * [BH21] A. Bajravani and G. H. Hitching: _Brill-Noether loci on moduli spaces of symplectic bundles over curves_. Collectanea Math. 72 (2021), 443–469.
  * [BaF18] B. Bakker and G. Farkas: _The Mercat conjecture for stable rank $2$ vector bundles on generic curves_. Amer. J. Math. 140 (2018), 1277–1295.
  * [Ba98] E. Ballico: _Brill-Noether theory for vector bundles on projective curves_. Proc. Camb. Phil. Soc. 124 (1998), 483–499.
  * [Ba06a] E. Ballico: _Coherent systems with many sections on projective curves_. Internat. J. Math. 17 (2006), 263–267.
  * [Ba06b] E. Ballico: _Stable coherent systems on integral projective curves: an asymptotic existence theorem_ , Int. J. Pure Appl. Math. 27 (2006), 205–214.
  * [BaP07] E. Ballico and F. Prantil: _Coherent systems on singular genus one curves_. Int. J. Contemp. Math. Sci. 2 (2007), 1527–1543, doi: 10.12988/ijcms.2007.07160.
  * [BF98] A. Bertram and B. Feinberg, _On stable rank two bundles with canonical determinant and many sections_. In: Algebraic Geometry (Catania, 1993/Barcelona 1994), 259–269, Lecture Notes in Pure and Appl. Math. 200, Marcel Dekker, New York, 1998.
  * [Bh07] U. N. Bhosle: _Brill-Noether theory on nodal curves_. Internat. J. Math. 18 (2007), 1133–1150.
  * [Bh09] U. N. Bhosle: _Coherent systems on a nodal curve_. In: Moduli spaces and vector bundles, London Mathematical Society Lecture Note Series Vol. 359, 437–455, Cambridge University Press, 2009.
  * [Bh11] U. N. Bhosle: _Coherent systems on a nodal curve of genus $1$_. Math. Nachr. 284 (2011), 1829–1845, doi: 10.1002/mana.200910133.
  * [BBN08] U. N. Bhosle, L. Brambila-Paz and P. E. Newstead: _On coherent systems of type $(n,d,n+1)$ on Petri curves_. Manuscripta Math. 126 (2008), 409–441.
  * [BBN15] U. N. Bhosle, L. Brambila-Paz and P. E. Newstead: _On linear series and a conjecture of D. C. Butler_. Internat. J. Math. 26 (2015), 1550007 (18 pages).
  * [BhP14] U. N. Bhosle and A. J. Parameswaran: _Picard bundles and Brill- Noether loci on the compactified Jacobian of a nodal curve_. IMRN Vol. 2014, Issue 15 (2014), 4241–4290, doi: 10.1093/imrn/rnt069.
  * [BhS13] U. N. Bhosle and S. K. Singh: _Brill-Noether loci and generated torsionfree sheaves over nodal and cuspidal curves_. Manuscripta Math. 141 (2013), 241–271.
  * [BG02] S. B. Bradlow and O. García-Prada: _An application of coherent systems to a Brill-Noether problem_. J. Reine Angew. Math. 551 (2002), 123–143.
  * [BGMN03] S. B. Bradlow, O. García-Prada, V. Muñoz and P. E. Newstead: _Coherent systems and Brill-Noether theory_. Internat. J. Math. 14 (2003), 683–733.
  * [BGMMN07] S. B. Bradlow, O. García-Prada, V. Mercat, V. Muñoz and P. E. Newstead: _On the geometry of moduli spaces of coherent systems on algebraic curves_. Internat. J. Math. 18 (2007), 411–453.
  * [BGMMN09] S. B. Bradlow, O. García-Prada, V. Mercat, V. Muñoz and P. E. Newstead: _Moduli spaces of coherent systems of small slope on algebraic curves_. Comm. in Alg. 37 (2009), 2649–2678.
  * [BP08] L. Brambila-Paz: _Non-emptiness of moduli spaces of coherent systems_. Internat. J. Math. 19 (2008), 779–799.
  * [BGN97] L. Brambila-Paz, I. Grzegorczyk and P. E. Newstead: _Geography of Brill-Noether loci for small slopes_. J. Alg. Geom. 6 (1997), 645-669.
  * [BL98] L. Brambila-Paz and H. Lange: _A stratification of the moduli space of vector bundles on curves (dedicated to Martin Kneser on the occasion of his 70th birthday)_. J. Reine Angew. Math. 494 (1998), 173–187.
  * [BM20] L. Brambila-Paz and O. Mata-Gutierrez: _$(t,\ell)$ -stability and coherent systems_. Glasgow Math. J. 62 (2020), 661–672.
  * [BMGNO19] L. Brambila-Paz, O. Mata-Gutierrez, P. E. Newstead and A. Ortega: _Generated coherent systems and a conjecture of D. C. Butler_. Internat. J. Math. 30 (2019), 1950024 (25 pages), doi: 10.1142/S0129167X19500241.
  * [BMNO00] L. Brambila-Paz, V. Mercat, P. E. Newstead and F. Ongay: _Nonemptiness of Brill-Noether loci_. Internat. J. Math. 11 (2000), 737–760.
  * [BO09a] L. Brambila-Paz and A. Ortega: _Brill-Noether bundles and coherent systems on special curves_. In: Moduli spaces and vector bundles, 456–472, London Mathematical Society Lecture Note Series Vol. 359, Cambridge University Press, Cambridge, 2009.
  * [BO09b] L. Brambila-Paz and A. Ortega: _Estabilidad de sistemas coherentes_. Aportaciones Mat. Comun. 40 (2009), 15–26.
  * [BT16] L. Brambila-Paz and H. Torres-López: _On Chow stability for algebraic curves_. Manuscripta Math. 151 (2016), 289–304.
  * [BrF20a] S. Brivio and F. F. Favale: _On kernel bundles over reducible curves with a node_. Internat. J. Math. 31 (2020), 2050054 (15 pages), doi: 10.1142/S0129167X12500371.
  * [BrF20b] S. Brivio and F. Favale: _Coherent systems on curves of compact type_. J. Geom. Phys. 158 (2020), 103850, doi: 10.1016/j.geomphys.2020.103850.
  * [BrF21] S. Brivio and F. Favale: _Coherent systems and BGN extensions on nodal reducible curves_. arXiv: 2104.06883
  * [Bu94] D. C. Butler: _Normal generation of vector bundles over a curve_. J. Diff. Geom. 39 (1994), 1-34.
  * [Bu97] D. C. Butler: _Birational maps of moduli of Brill-Noether pairs_. arXiv:alg-geom/9705009.
  * [CLT18] A. Castorena, A. López Martín and M. Teixidor i Bigas: _Petri map for vector bundles near good bundles_. J. Pure Appl. Algebra. 222 (2018), 1692–1703.
  * [CMT20] A. Castorena, E. C. Mistretta and H. Torres-López: _On linear stability and syzygy stability for rank $2$ linear series_. arXiv: 2001.03609.
  * [CT18] A. Castorena and H. Torres-López: _Linear stability and stability of syzygy bundles_. Internat. J. Math. 29 (2018), 1850080 (14 pages).
  * [CPJ19] K. Cook-Powell and D. Jensen: _Components of Brill-Noether loci for curves of fixed gonality_. arXiv: 1907.08366.
  * [CGZ21] D. Cotterill, A. Alonso Gonzalo and N. Zhang: _The strong maximal rank conjecture and higher rank Brill-Noether theory_. J. London Math. Soc. 104 (2021), 169–205.
  * [FJP21] G.Farkas, D. Jensen and S. Payne: _The non-abelian Brill-Noether divisor on $\overline{\mathcal{M}}_{13}$ and the Kodaira dimension of $\overline{\mathcal{R}}_{13}$._ arXiv: 2110.09553.
  * [FO11] G. Farkas and A. Ortega: _The maximal rank conjecture and rank two Brill-Noether theory_. Pure and Appl. Math. Quarterly 7 no. 4 (2011) (Special Issue: In Memory of Eckart Viehweg), 1265–1296.
  * [FO12] G. Farkas and A. Ortega: _Higher rank Brill-Noether theory on sections of K3 surfaces_. Internat. J. Math. 23 (2012), 1250075.
  * [FP05] G. Farkas and M. Popa: _Effective divisors on $\overline{M}_{g}$, curves on K3 surfaces, and the slope conjecture_. J. Alg. Geom. 14 (2005), 241–267.
  * [FL21] S. Feyzbakhsh and Chunyi Li: _Higher rank Clifford indices of curves on a K3 surface_. Selecta Math. (N.S.) 27 (2021), no. 3, Paper No. 48 (34 pages).
  * [Ghi83] F. Ghione: _Un problème du type Brill-Noether pour les fibrés vectoriels_. In: Algebraic Geometry - Open Problems (Ravello, 1982), 197–209, Lecture Notes in Mathematics Vol. 997, Springer, Berlin, 1983.
  * [GMN11] I. Grzegorczyk, V. Mercat and P. E. Newstead: _Stable bundles of rank 2 with four sections_. Internat. J. Math. 22 (2011), 1743–1762, doi: 10.1142/S0129167X11007434.
  * [GN14] I. Grzegorczyk and P. E. Newstead: _On coherent systems with fixed determinant_. Internat. J. Math. 25 (2014), 1450045 (11 pages), doi: 10.1142/S0129167X14500451.
  * [HT08] D. Hernández Ruipérez and C. Tejero Prieto: _Fourier-Mukai transforms for coherent systems on elliptic curves_. J. London Math. Soc. 77 (2008), 15–32.
  * [Hir88] A. Hirschowitz: _Problèmes de Brill-Noether en rang supérieur_. C. R. Acad. Sci. Paris Sér. I Math. 307 (1988), 153–156.
  * [HHN21] G. H. Hitching, M. Hoff and P. E. Newstead: _Nonemptiness and smoothness of twisted Brill-Noether loci_. Ann. Mat. Pura Applicata 200 (2021), 521–546, doi: 10.1007/s10231-020-01009-x (open access).
  * [Hof21] M. Hoff: _A note on syzygies and normal generation for trigonal curves_. arXiv: 2108.06106
  * [KN95] A. King and P. E. Newstead: _Moduli of Brill-Noether pairs on algebraic curves_. Internat. J.Math. 6 (1995), 733–748.
  * [LMN12] H. Lange, V. Mercat and P. E. Newstead: _On an example of Mukai_. Glasgow Math. J. 54 (2012), 261–271, doi:10.1017/S0017089511000577.
  * [LN04] H. Lange and P. E. Newstead: _Coherent systems of genus $0$_. Internat. J. Math. 15 (2004), 409–424.
  * [LN05] H. Lange and P. E. Newstead: _Coherent systems on elliptic curves_. Internat. J. Math. 16 (2005), 787–805.
  * [LN07] H. Lange and P. E. Newstead: _Coherent systems of genus $0$ II: Existence results for $k\geq 3$_. Internat. J. Math. 18 (2007), 363–393.
  * [LN08a] H. Lange and P. E. Newstead: _Coherent systems of genus $0$ III: Computation of flips for $k=1$_. Internat. J. Math. 19 (2008), 1103–1119.
  * [LN08b] H. Lange and P. E. Newstead: _Clifford’s theorem for coherent systems_. Arch. Math. 90 (2008), 209–216, doi:10.1007/s00013-007-2534-3.
  * [LN09] H. Lange and P. E. Newstead: _Hodge polynomials and birational types of moduli spaces of coherent systems on elliptic curves_. Manuscripta Math. 130 (2009), 1–19.
  * [LN10a] H. Lange and P. E. Newstead: _Clifford indices for vector bundles on curves_. In: A. Schmitt (Ed.) Affine Flag Manifolds and Principal Bundles, 165–202, Trends in Mathematics, Birkhäuser, Basel, 2010.
  * [LN10b] H. Lange and P. E. Newstead: _Generation of vector bundles computing Clifford indices_. Arch. Math. 94 (2010), 529–537, doi: 10.1007/S00013-017-0126-0.
  * [LN11] H. Lange and P. E. Newstead: _Further examples of stable bundles of rank 2 with 4 sections_. Pure Appl. Math. Quarterly 7 no.4 (2011) (Special Issue: In Memory of Eckart Viehweg), 1517–1528.
  * [LN13a] H. Lange and P. E. Newstead: _Bundles computing Clifford indices on trigonal curves_. Arch. Math. 101 (2013), 21–31, doi: 10.1007/S00013-013-0540-1.
  * [LN13b] H. Lange and P. E. Newstead: _Vector bundles of rank $2$ computing Clifford indices_. Comm. in Algebra 41 (2013), 2317–2345.
  * [LN13c] H. Lange and P. E. Newstead: _On bundles of rank 3 computing Clifford indices_. Kyoto J. Math. 53, no. 1 (Memorial issue for Masaki Maruyama) (2013), 25–54.
  * [LN13d] H. Lange and P. E. Newstead: _Bundles of rank 3 on curves of Clifford index 3_. J. Symbolic Computation 57 (2013), 3–18, doi: 10.1016/j.jsc.2013.05.002.
  * [LN17] H. Lange and P. E. Newstead: _Higher rank BN-theory for curves of genus $4$_. Comm. in Algebra 45 (2017), 3948–3966, doi: 10.1080/00927872.2016.1251938.
  * [LN16] H. Lange and P. E. Newstead: _Higher rank BN-theory for curves of genus $5$_. Rev. Mat. Complut. 29 (2016), 691–717.
  * [LN18] H. Lange and P. E. Newstead: _Higher rank BN-theory for curves of genus $6$_. Internat. J. Math. 29 (2018), 1850014 (40 pages), doi: 10.1142/S0129167X18500143.
  * [LNP16] H. Lange, P. E. Newstead and Seong Suk Park. _Nonemptiness of Brill-Noether loci in M(2,K)_. Comm. in Algebra 44 (2016), 746–767, doi: 10.1080/00927872.2014.990020.
  * [LNS15] H.Lange, P. E. Newstead and V. Strehl: _Nonemptiness of Brill-Noether loci in M(2,L)_. Internat. J. Math. 26 (2015), 1550108 (26 pages), doi: 10.1142/S0129167X15501086.
  * [LLV20] E. Larson, H. K. Larson and I. Vogt: _Global Brill-Noether theory over the Hurwitz space_. arXiv: 2008.10765.
  * [Lar20] H. K. Larson: _Refined Brill-Noether theory for all trigonal curves_. arXiv: 2002.00142.
  * [Laz84] R. Lazarsfeld: _Some applications of the theory of positive vector bundles_. In: Complete intersections (Acireale, 1983), 29–61, Lecture Notes in Math. Vol. 1092, Springer, Berlin, 1984.
  * [LC13] M. Lelli-Chiesa: _Stability of rank- $3$ Lazarsfeld-Mukai bundles on K3 surfaces_. Proc. London Math. Soc. 107 (2013), 451–479, doi:10.1112/plms/pds087.
  * [Li19] Chunyi Li: _On stability conditions for the quintic threefold_. Invent. Math. 218 (2019), 301–-340.
  * [Mer99a] V. Mercat: _Le problème de Brill-Noether pour des fibrés stables de petite pente_. J. reine angew. Math. 506 (1999), 1–41.
  * [Mer99b] V. Mercat: _Le problème de Brill-Noether et le théorème de Teixidor_. Manuscripta Math. 98 (1999), 75–85.
  * [Mer01] V. Mercat: _Fibrés stables de pente 2_. Bull. London Math. Soc. 33 (2001), 535–542.
  * [Mer02] V. Mercat: _Clifford’s theorem and higher rank vector bundles_. Internat. J. Math. 13 (2002), 785–796.
  * [MS12] E. C. Mistretta and L. Stoppino: _Linear series on curves: stability and Clifford index_. Internat. J. Math. 23 (2012), 1250121 (25 pages).
  * [Mu95] S. Mukai: _Vector bundles and Brill-Noether theory_. In Current topics in complex algebraic geometry (Berkeley, CA, 1992/93), 145–158, Math. Sci. Res. Inst. Publ., 28, Cambridge Univ. Press, Cambridge, 1995.
  * [Mu01] S. Mukai: _Non-abelian Brill-Noether theory and Fano 3-folds_. Sugaku Expositions 14 (2001), 125–153.
  * [Mun08] V. Muñoz: _Hodge polynomials of the moduli spaces of rank $3$ pairs_. Geom. Dedicata 136 (2008), 17–46.
  * [MOV07] V. Muñoz, D. Ortega and M. J. Vázquez-Gallo: _Hodge polynomials of the moduli space of pairs_. Internat. J. Math. 18 (2007), 695–721.
  * [MOV09] V. Muñoz, D. Ortega and M. J. Vázquez-Gallo: _Hodge polynomials of the moduli space of triples of rank $(2,2)$_. Quart. J. Math. 60 (2009), 235–272.
  * [New11] P. E. Newstead: Existence of $\alpha$-stable coherent systems on algebraic curves. In: Grassmannians, Moduli Spaces and Vector Bundles, 121–139, Clay Math. Proc. Vol. 14, American Mathematical Society, Providence, RI, 2011.
  * [New18] P. E. Newstead: _Some examples of rank-2 Brill-Noether loci_. Rev. Mat. Complut. 21 (2018), 201–215, doi: 10.1007/s13163-017-0241-6.
  * [NT21] P. E. Newstead and M. Teixidor i Bigas: _Coherent systems on the projective line_. Quart. J. Math. 72 (2021), 115–136, doi:10.1093/qmathj/haaa024.
  * [Oss13a] B. Osserman: _Brill-Noether loci with fixed determinant in rank $2$_. Internat. J. Math. 24 (2013), 1350019 (24 pages).
  * [Oss13b] B. Osserman: _Special determinants in higher-rank Brill-Noether theory_. Internat. J. Math. 24 (2013), 1350084 (20 pages).
  * [PP07] S. Pasotti and F. Prantil: _Holomorphic triples on elliptic curves_. Results Math. 50 (2007), 227–239.
  * [PP08] S. Pasotti and F. Prantil: _Holomorphic triples of genus $0$_. Central European J. Math. 6 (2008), 129–142.
  * [Pfl17] N. Pflueger: _Brill-Noether varieties of $k$-gonal curves_. Adv. Math. 312 (2017), 46–63.
  * [Re98] R. Re: _Multiplication of sections and Clifford bounds for stable vector bundles on curves_. Comm. in Alg. 26 (1998), 1931-1944.
  * [Roa20] L. Roa-Leguizamón: _Segre invariant and a stratification of the moduli space of coherent systems_. Internat. J. Math. 31 (2020), 2050117 (32 pages).
  * [RT99] B. Russo and M. Teixidor i Bigas: _On a conjecture of Lange_. J. Alg. Geom. 8 (1999), 483–496.
  * [Sch19] A. Schmitt: _A general notion of coherent systems_. hal-02391836 (39 pages).
  * [Sch21] A. Schmitt: _Notes on coherent systems_. Rev. Mat.: Teoria y Apl. 28 (2021), 1–38.
  * [Tei91a] M. Teixidor i Bigas: _Brill-Noether theory for stable vector bundles_. Duke Math. J. 62 (1991), 385–400.
  * [Tei91b] M. Teixidor i Bigas: _Moduli spaces of (semi)stable vector bundles on tree-like curves_. Math. Ann. 290 (1991), 341–348.
  * [Tei95] M. Teixidor i Bigas: _Moduli spaces of vector bundles on reducible curves_. Amer. J. Math. 117 (1995), 125–139.
  * [Tei98] M. Teixidor i Bigas: _Curves in Grassmannians_. Proc. Amer. Math. Soc. 126 (1998), 1597–1603.
  * [Tei04] M. Teixidor i Bigas: _Rank two vector bundles with canonical determinant_. Math. Nachr. 265 (2004), 100–106.
  * [Tei07] M. Teixidor i Bigas: _Existence of coherent systems of rank two and dimension four_. Collect. Math. 58 (2007), 193–198.
  * [Tei08a] M. Teixidor i Bigas: _Petri map for rank two bundles with canonical determinant_. Compos. Math. 144 (2008), 705–720.
  * [Tei08b] M. Teixidor i Bigas: _Existence of coherent sysytems II_. Internat. J. Math. 19 (2008), 1269–1283.
  * [Tei10] M. Teixidor i Bigas: _Existence of vector bundles of rank two with fixed determinant and sections_. Proc. Japan Acad. 86 Ser. A (2010), 113–118.
  * [Tei14] M. Teixidor i Bigas: _Injectivity of the Petri map for twisted Brill-Noether loci_. Manuscripta Math. 145 (2014), 389–397.
  * [Th94] M. Thaddeus: _Stable pairs, linear systems and the Verlinde formula_. Invent. Math. 117 (1994), 317–353.
  * [Zh16] N. Zhang: _Towards the Bertram-Feinberg-Mukai Conjecture_. J. Pure Appl. Algebra 220 (2016), 1588–1654.
  * [Zh17] N. Zhang: _Expected dimensions of higher rank Brill-Noether loci_. Proc. Amer. Math. Soc. 145 (2017), 3735–3746.


# Toward Robust LiDAR based 3D Object Detection via
Density-Aware Adaptive Thresholding

Eunho Lee1, Minwoo Jung2 and Ayoung Kim2∗ †This work was supported by
Institute of Information & communications Technology Planning & Evaluation
(IITP) grant funded by the Korea government(MSIT) No.2022-0-00480, Development
of Training and Inference Methods for Goal-Oriented Artificial Intelligence
Agents1E. Lee is with the Interdisciplinary Program in Artifical Intelligence,
SNU, Seoul, S. Korea<EMAIL_ADDRESS>Jung and A. Kim are with the Dept.
of Mechanical Engineering, SNU, Seoul, S. Korea [moonshot<EMAIL_ADDRESS>
###### Abstract

Robust 3D object detection is a core challenge for autonomous mobile systems
in field robotics. To tackle this issue, many researchers have demonstrated
improvements in 3D object detection performance in datasets. However, real-
world urban scenarios with unstructured and dynamic situations can still lead
to numerous false positives, posing a challenge for robust 3D object detection
models. This paper presents a post-processing algorithm that dynamically
adjusts object detection thresholds based on the distance from the ego-
vehicle. 3D object detection models usually perform well in detecting nearby
objects but may exhibit suboptimal performance for distant ones. While
conventional perception algorithms typically employ a single threshold in
post-processing, the proposed algorithm addresses this issue by employing
adaptive thresholds based on the distance from the ego-vehicle, minimizing
false negatives and reducing false positives in urban scenarios. The results
show performance enhancements in 3D object detection models across a range of
scenarios, not only in dynamic urban road conditions but also in scenarios
involving adverse weather conditions.

##  I Introduction

3D object detection is one of the fundamental components of autonomous mobile
systems in field robotics. This system often handles the unstructured and
dynamic real-world environments that invoke unpredictable situations [1, 2,
3]. However, 3D object detectors trained in datasets perform less in various
real-world environments than in datasets, leading to system malfunction.

Robust 3D object detection is crucial to function for their purpose in highly
dynamic environments like urban roads, where dynamic objects, unpredictable
obstacles, and sensor noise present significant challenges. While urban roads
might seem structured, they are also one of the key environments that field
robotics must handle because they are dynamic and challenging [4]. These
environments are filled with moving vehicles, pedestrians, and ghost obstacles
worsened by adverse weather [5, 6]. This leads to numerous false positives,
potentially causing sudden stops and fatal accidents. Therefore, developing a
robust perception module that minimizes false positives on urban roads is a
key research focus in field robotics.

Figure 1:  The dynamic environment involves moving vehicles, pedestrians, and
obstacles such as bushes and road signs (RGB image). 3D object detection using
a single threshold misclassifies the bushes as vehicles, which results in
false positives (single threshold). Our method significantly improves the
robustness of 3D object detection in complex urban scenarios by effectively
minimizing false detections, thus enhancing autonomous driving systems’
overall performance and safety (ours).

Traditionally, 3D object detection usually utilizes LiDAR, offering high
precision and efficient 3D environmental data for robust perception. LiDAR-
based 3D object detection models [7, 8, 9, 10, 11, 12, 13] use 3D point clouds
to predict an object’s class, location, and confidence scores, which are then
refined by confidence score-based post-processing using a single threshold
hyper-parameter. Detection accuracy varies with object distance due to sensor
characteristics like resolution and range and training dataset diversity.
Objects near the ego-vehicle have higher accuracy and confidence due to the
increased density of point clouds, whereas distant objects have lower recall
and confidence scores due to the corresponding decrease in point cloud
density. These observations indicate that for autonomous mobile systems to
drive safely on real roads, it is more suitable to prioritize precision for
objects closer and recall for objects farther away. Consequently, employing a
single threshold in post-processing is inadequate for autonomous mobile
systems operating across diverse real-world environments.

Figure 2:  Framework of the proposed algorithm. 3D Detectors [7, 8, 9, 10, 11,
12, 13] normally apply the post processing using the single threshold. In
contrast, the adaptive thresholding.

While existing methods on the post-processing algorithm utilized adaptive
thresholding [14, 15] have explored adaptive thresholding techniques, their
focus has remained on 2D image processing, which does not fully satisfy 3D
object detections. To tackle this issue, our work introduces a novel adaptive
thresholding algorithm specifically designed for 3D object detection, making
it better suited for implementation in autonomous mobile systems. This
innovation also offers a straightforward integration process that does not
require extra training or add unnecessary complexity to existing detection
frameworks. By optimizing the balance between minimizing false positives for
nearby objects and lowering missed detections for distant objects, our
approach enhances the overall efficacy of 3D object detection in datasets and
real-world scenarios. This contributes to safer and more reliable autonomous
navigation, as demonstrated in our qualitative analysis presented in Fig. 1.

Differing from previous methods, our method presents the following
contributions:

  * •

By applying adaptive thresholding based on the distance from the ego-vehicle
in the post-processing of 3D object detection models, we significantly reduce
false positives and false negatives according to distance. This improvement
strengthens the robustness of models and makes the model more suitable for
autonomous mobile systems by enabling more stable driving.

  * •

The proposed algorithm simplifies the integration into various 3D object
detection frameworks and effectively boosts detection performance in terms of
mAP (mAP) and the balance between Recall and Precision. Our experiments across
multiple frameworks have confirmed these improvements.

  * •

The reduction in misidentification issues within the model has been
qualitatively validated based on actual driving data from diverse urban roads
and under varying weather conditions.

##  II related work

###  II-A Adaptive Thresholding in Object Detection

Previous researches on adaptive thresholding [14, 15] have been conducted for
detecting moving objects in 2D images. Zhang et al. [14] enhanced detection
accuracy by employing adaptive thresholding based on distance. This was
achieved by detecting objects within an image and subsequently estimating the
objects’ states and distances through pixel comparison with previous frames.
Lin and Huang [15] utilizes adaptive thresholding to determine thresholds for
distinguishing between moving and stationary objects during depth estimation,
a critical step in converting images into a 3D representation. This approach
involves considering features within the image in pairs, with the threshold
adjusting dynamically based on the distance between these features, thereby
facilitating the determination of an object’s motion status. These studies
have focused on 2D images, making them unsuitable for 3D object detection
models required in autonomous driving, which necessitates representation
within a 3D space for various conditions and object recognition.

###  II-B LiDAR based 3D Object Detection

3D object detection can be categorized into LiDAR-based, Camera-based, Radar-
based, and Sensor-Fusion-based modalities. This paper is dedicated solely to
exploring LiDAR-based 3D object detection strategies. Within the spectrum of
LiDAR-based methodologies, we classify approaches into voxel, pillar, and raw
point cloud-based.

VoxelNet [7] aggregates and downsamples 3D point clouds into voxels for
feature representation and executes object detection via 3D convolution
operations facilitated by a Region Proposal Network (RPN) [8]. SECOND [9]
introduces an effective voxel computation strategy for handling the sparse
attributes present in 3D environments. PointPillars [10] adopts a pillar
representation to facilitate 2D convolution operations, optimizing
computational and memory efficiencies while ensuring robust performance.
PointRCNN [11] distinguishes objects from the background using raw point
clouds, employing this distinction for object detection to compromise high
precision. PV-RCNN [12] leverages both raw point clouds and voxels for object
detection, preserving the intrinsic properties of point clouds despite
significant computational requirements, thus demonstrating remarkable
efficiency. These methods rely on anchor-based detection paradigms.
Progressing beyond, Duan et al. [13] proposed CenterNet, advocating an anchor-
free paradigm, thereby challenging traditional anchor-dependent frameworks.

Most 3D object detection models are trained and evaluated on specific datasets
representing typical, non-adversarial road environments. However, these models
struggle to adequately address the more diverse and challenging situations
autonomous vehicles encounter on actual urban roads. On actual roads, varying
weather conditions such as rain, snow, and fog can introduce noise into sensor
data, and elements like bushes or road signs may be mistakenly identified as
vehicles or obstacles, leading to false positives. We propose a method that
enables autonomous vehicles to adaptively handle false positives, ensuring
stable driving on real urban roads.

##  III Adaptive Thresholding Framework

We illustrate our framework in Fig. 2. The 3D object detection model that
receives the point clouds as input detects surrounding objects and outputs the
class, location, and confidence score. Then, the adaptive thresholding module
receives the distance from the ego-vehicle for each class of objects and the
confidence score as input and finally predicts the 3D bounding boxes of
objects accurately. This study used PointPillars [10] as our 3D object
detection model. We analyze how distinct single threshold values, such as 0.3,
0.5, and 0.7, impact the distance-based performance results of the 3D detector
within the Kitti 3D object detection dataset [16]. In analyzing detection
performance with a threshold of 0.5, it is observed that the 3D object
detection algorithm effectively identifies objects closer to the ego-vehicle
(d $<$ 30m), resulting in the number of detections that surpass the ground
truth counts. However, its performance drops over extended ranges (d $>$ 30m),
where fewer objects are detected than the ground truth. It highlights a
variable recall rate with higher recall at closer proximities and lower recall
at extended distances. The inverse relationship between Recall and Precision
indicates lower precision at close distances and higher precision at longer
distances. To enhance precision for objects at closer distances, a higher
threshold of 0.7 was applied, leading to a general decrease in detection
counts. Notably, the detection rate for objects beyond 40m dropped
significantly, resulting in lower recall and increased false negatives.
Conversely, a lower threshold of 0.3 was employed to increase recall for
distant objects, which improved recall and increased the number of detected
objects within the 10 to 40m range beyond the ground truth, thereby reducing
overall precision.

Figure 3:  Confidence score tendency with a single threshold (=0.5). The red
dots represent the score mean values at each distance (10 m), and the green-
shaded area indicates the standard deviation.

The analytical outcomes, employing single thresholds of 0.5 for proximal
distances (below 40m) and 0.3 for more extended distances (above 40m),
facilitated the computation of the standard deviation across intervals of 10m,
as described in Fig. 2. After an examination of confidence score trends
depicted in Fig. 2, it was inferred that a quadratic function most
appropriately captures the variability of confidence scores across distances,
leading to the formulation of the distance-based adaptive thresholding
equation as described in Eq. 1.

$\text{ Score Threshold }=\left\\{\begin{array}[]{c}\alpha d^{2}+\beta
d+\gamma(0<d\leq\delta)\\\ k(d>\delta)\end{array}\right.$ (1)

This equation, $d$, represents the distance from the ego-vehicle to the
object, measured in meters. The parameters $\alpha$, $\beta$, and $\gamma$
determine the shape of the quadratic curve that models the variation of
confidence scores with distance. $\delta$ is the parameter that defines the
maximum distance at which the algorithm is applicable, while $k$ represents
the constant value of the quadratic curve at a distance $\delta$. The value of
$\delta$ may vary depending on the detection range capabilities of the LiDAR
and the performance of the 3D object detection models. To prevent the
confidence threshold from decreasing too much for distances beyond $\delta$,
thus increasing the risk of false positives, $k$ is set accordingly. These
parameters enable the application of the algorithm not only to the
PointPillars [10] model but also across various 3D object detection models.

$\text{ Score Mean}_{d}=\frac{\sum_{i=d}^{d+1}\text{ Score
}_{i}}{N_{d}}(\mathrm{~{}d}=0,1,2,3,4,5)$ (2)

$\displaystyle\text{Score Std}_{d}$
$\displaystyle=\sqrt{\frac{1}{N_{d}}\sum_{i=d}^{N_{d}}\left(\text{Score}_{i}-\text{Score
Mean}_{d}\right)^{2}}$ (3) $\displaystyle(\mathrm{d}$
$\displaystyle=0,1,2,3,4,5)$ (4)

Eq. 2 segments the range from 0 to 60 meters into intervals of 10 meters each,
designating each as the $d^{th}$ interval. Within each interval, the sum of
the confidence scores of detected objects is divided by the number of detected
objects in that interval to calculate the average confidence score. Similarly,
Eq. 3 computes the standard deviation of confidence scores within each
$d^{th}$ interval, divided by the interval’s average confidence score. A
quadratic trend line is derived by adjusting the six average confidence scores
obtained from Eq. 2 within the standard deviations specified in Eq. 3. This
process determines the values of the parameters $\alpha$, $\beta$, $\gamma$
and $k$ for Eq. 1.

##  IV Experiments

###  IV-A Evaluation Metrics

Two distinct datasets were utilized for the evaluation: the Kitti 3D Object
Detection Dataset [16], a publicly available dataset designed for qualitative
assessment of our algorithms, and the custom urban road dataset specifically
gathered for quantitative analysis.

The custom urban road dataset comprises data acquired from the Hyundai Elec-
city, outfitted with six Velodyne 32CH LiDAR sensors. This vehicle conducted
data collection drives around Blue House and Gyeongbokgung in Seoul, capturing
diverse environmental conditions ranging from clear to rainy weather
scenarios. Additionally, for further data diversity, the Kia Carnival,
equipped with four Velodyne 32CH LiDAR sensors, drove in the foggy and rainy
environments of Gangneung roads, acquiring a comprehensive dataset
encompassing various weather conditions.

###  IV-B Quantitative Results

Our algorithm is a car class that accounts for 93% of Kiti 3D object detection
datasets and quantitatively evaluates whether the 3D object detection model is
suitable for autonomous vehicle systems. 3D detector’s suitability for
autonomous vehicles, ensuring safe and efficient driving from Recall,
Precision, and mAP is quantitatively indicated by stable or improving mAP
values with balanced Recall and Precision. This implies the model’s accurate
object detection and recognition efficiency while minimizing false positives,
ensuring safe vehicle operation. Therefore, Performance metrics such as
Recall, Precision, their Trade-off, and mAP were employed to evaluate the
algorithm’s impact on 3D detectors before and after our algorithm’s
implementation.

TABLE I:  Recall, Precision and mAP comparison for PointPillars [10] applying
various single threshold and adaptive threshold

Method | Recall | Precision | Trade-Off | mAP
---|---|---|---|---
Single Threshold | 0.3 | 0.895 | 0.646 | 0.249 | 77.28
0.5 | 0.807 | 0.847 | 0.040 | 77.29
0.7 | 0.655 | 0.943 | 0.288 | 77.49
Ours | 0.786 | 0.813 | 0.025 | 77.29

TABLE II: Recall, Precision and mAP comparison for various 3D object detection
models applying single threshold 0.5 and adaptive thresholding.

| 3D Detectors | Recall | Precision | Trade-Off | mAP
---|---|---|---|---|---
Single Threshold | PointPillars [10] | 0.807 | 0.847 | 0.040 | 77.28
SECOND [9] | 0.808 | 0.856 | 0.048 | 78.62
PointRCNN [11] | 0.899 | 0.848 | 0.051 | 78.74
PV-RCNN [12] | 0.969 | 0.731 | 0.238 | 79.25
Ours | PointPillars [10] | 0.786 | 0.813 | 0.023 (-0.015) | 77.28
SECOND [9] | 0.792 | 0.823 | 0.031 (-0.016) | 78.62
PointRCNN [11] | 0.849 | 0.815 | 0.034 (-0.017) | 78.73
PV-RCNN [12] | 0.893 | 0.792 | 0.101 (-0.137) | 79.49

Table. I contrasts the outcomes of implementing diverse single thresholds on
PointPillars [10] with those of the proposed algorithm. Optimal parameter
values were established through iterative experimentation. The algorithm’s
parameters, as indicated in Eq. 1, are $\alpha$ = -0.00002, $\beta$ = -0.0061,
$\gamma$ = 0.6828, and $k$ = 0.6. Our algorithm resulted in a decreased trade-
off compared to a single threshold, maintaining comparable mAP performance.
This highlights the algorithm’s ability to fine-tune Recall and Precision for
objects near and far, improving Precision for closer objects and increasing
Recall for those at extended distances.

Further validation of the algorithm’s efficiency was conducted through its
implementation to additional LiDAR-based 3D object detection models: SECOND
[9], PointRCNN [11], and PV-RCNN [12]. These models were subject to
performance evaluation based on parameters derived from the PointPillars
[10]’s experiment. The outcomes with a single 0.5 threshold for each model are
tabulated in Table. II, while results leveraging the proposed algorithm are
documented in Table. II. As depicted in Table. II, the proposed algorithm for
various 3D detectors significantly reduced trade-offs relative to a single
threshold, demonstrating that the algorithm improves prediction performances
suited for autonomous driving.

###  IV-C Qualitative Results

Figure 4:  Results of our algorithm. In challenging urban road scenarios, such
as fog and rain, our algorithm enhances the performance of 3D object detection
models by reducing false positives and accurately distinguishing vehicles from
point clouds caused by adverse weather conditions. This leads to improved
overall precision of object detection, ensuring safer driving for autonomous
vehicles.

Using our urban road dataset, we conducted a qualitative assessment using the
PointPillars[10] model to address the issue of false positives in various real
urban roads and weather conditions.

Fig. 1 shows urban road data acquired in clear weather conditions near Blue
House and Gyeongbokgung in Seoul. In the RGB image of Fig. 1., bushes exist
horizontally along the right side of the road, from which many point clouds
emerge, resembling the shape of vehicles. This leads the model to misidentify
bushes as vehicles when applying a single threshold. During actual autonomous
driving, such false positives were mistakenly perceived as vehicles changing
lanes to the driving lane of the ego-vehicle, causing the vehicle to suddenly
stop. When the adaptive thresholding algorithm proposed in this paper was
applied to the model, it no longer recognized bushes as vehicles while still
accurately detecting vehicles in the left lane and ahead.

Fig. 4’s fog presents urban road data captured in environments with heavy fog
around Gangneung. In the RGB image of Fig. 4, vehicles can be observed in the
left lane, indicating the presence of significant fog. A characteristic of fog
data is its appearance at a similar height to LiDAR, resembling smoke-like
point noise. Consequently, when a single threshold is applied, false positives
of densely fogged areas as floating vehicles in the air can be observed. Our
algorithm does not recognize the falsely identified fog, while vehicles
existing in the left lane continue to be accurately detected.

Fig. 4’s rain presents urban road data acquired in heavy rainfall around Blue
House and Gyeongbokgung in Seoul. The RGB image in Fig. 4 indicates a
situation with substantial precipitation. Although vehicles exist in the left
lane, it is observed that no vehicles are present in the driving lane ahead or
the right lane. False positives of vehicles occur due to the large amount of
point clouds reflected by the heavy rain, and similar to Fig. 1, false
positives caused by bushes on the right can be observed. Our algorithm
improves Precision at close distances for false positives caused by rain and
bushes while vehicles in the left lane remain accurately detected.

##  V Conclusion & Discussion

This research presented an adaptive thresholding algorithm for post-processing
in 3D object detection models, which are pivotal for the perception module of
autonomous vehicles, a key component of Field Robotics. Through comparative
experiments on an open dataset between single thresholds and our algorithm,
we’ve demonstrated that our approach enhances the robustness of the perception
module. It effectively minimizes false positives by dynamically adjusting
thresholds according to the distance from the autonomous vehicle. Subsequent
experiments demonstrated the algorithm’s effectiveness and adaptability across
diverse 3D object detection frameworks. Additionally, qualitative results in
the custom dataset have verified the algorithm’s practical effectiveness under
dynamic urban roads and adverse weather conditions, such as fog and rain. This
shows our algorithm’s ability to enhance detection accuracy and robustness for
the perception module of autonomous driving.

For future work, we will extend the applicability of our methodology beyond
LiDAR-based 3D object detection models to those leveraging cameras, radars,
and sensor fusion, comprehensive studies on the distinct characteristics of
each sensor type. Moreover, this algorithm has the potential to evolve into a
learning-based approach for real-time adaptation to various scenarios.

## References

  * Wijayathunga et al. [2023] L. Wijayathunga, A. Rassau, and D. Chai, “Challenges and solutions for autonomous ground robot scene understanding and navigation in unstructured outdoor environments: A review,” _Applied Sciences_ , vol. 13, no. 17, p. 9877, 2023.
  * Ginerica et al. [2024] C. Ginerica, M. Zaha, L. Floroian, D. Cojocaru, and S. Grigorescu, “A vision dynamics learning approach to robotic navigation in unstructured environments,” _Robotics_ , vol. 13, no. 1, p. 15, 2024.
  * Itkina et al. [2019] M. Itkina, K. Driggs-Campbell, and M. J. Kochenderfer, “Dynamic environment prediction in urban scenes using recurrent representation learning,” in _2019 IEEE Intelligent Transportation Systems Conference (ITSC)_. IEEE, 2019, pp. 2052–2059.
  * Yurtsever et al. [2020] E. Yurtsever, J. Lambert, A. Carballo, and K. Takeda, “A survey of autonomous driving: Common practices and emerging technologies,” _IEEE Access_ , vol. 8, pp. 58 443–58 469, 2020.
  * Mirza et al. [2021] M. J. Mirza, C. Buerkle, J. Jarquin, M. Opitz, F. Oboril, K.-U. Scholl, and H. Bischof, “Robustness of object detectors in degrading weather conditions,” in _2021 IEEE International Intelligent Transportation Systems Conference (ITSC)_ , 2021, pp. 2719–2724.
  * Hahner et al. [2021] M. Hahner, C. Sakaridis, D. Dai, and L. Van Gool, “Fog simulation on real lidar point clouds for 3d object detection in adverse weather,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)_ , October 2021, pp. 15 283–15 292.
  * Zhou and Tuzel [2018] Y. Zhou and O. Tuzel, “Voxelnet: End-to-end learning for point cloud based 3d object detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 4490–4499.
  * Ren et al. [2015] S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,” _Advances in neural information processing systems_ , vol. 28, 2015.
  * Yan et al. [2018] Y. Yan, Y. Mao, and B. Li, “Second: Sparsely embedded convolutional detection,” _Sensors_ , vol. 18, no. 10, p. 3337, 2018.
  * Lang et al. [2019] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 12 697–12 705.
  * Shi et al. [2019] S. Shi, X. Wang, and H. Li, “Pointrcnn: 3d object proposal generation and detection from point cloud,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 770–779.
  * Shi et al. [2020] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 10 529–10 538.
  * Duan et al. [2019] K. Duan, S. Bai, L. Xie, H. Qi, Q. Huang, and Q. Tian, “Centernet: Keypoint triplets for object detection,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 6569–6578.
  * Zhang et al. [2021] K. Zhang, S. Tong, H. Shi, G. Yue, and J. Zhao, “Moving object detection of assembly components based on improved background subtraction algorithm,” in _IOP Conference Series: Materials Science and Engineering_ , vol. 1009, no. 1. IOP Publishing, 2021, p. 012063.
  * Lin and Huang [2018] S.-F. Lin and S.-H. Huang, “Moving object detection from a moving stereo camera via depth information and visual odometry,” in _2018 IEEE International Conference on Applied System Invention (ICASI)_. IEEE, 2018, pp. 437–440.
  * Geiger et al. [2012] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _2012 IEEE conference on computer vision and pattern recognition_. IEEE, 2012, pp. 3354–3361.

  *[mAP]: