# Low rank orthogonal bundles and quadric fibrations

Insong Choe  Department of Mathematics, Konkuk University, 1 Hwayang-dong,
Gwangjin-Gu, Seoul 143-701, Korea<EMAIL_ADDRESS>and  George H. Hitching
Oslo Metropolitan University, Postboks 4, St. Olavs plass, 0130 Oslo, Norway
<EMAIL_ADDRESS>
###### Abstract.

Let $C$ be a curve and $V\to C$ an orthogonal vector bundle of rank $r$. For
$r\leq 6$, the structure of $V$ can be described using tensor, symmetric and
exterior products of bundles of lower rank, essentially due to the existence
of exceptional isomorphisms between $\mathrm{Spin}(r,{\mathbb{C}})$ and other
groups for these $r$. We analyze these structures in detail, and in particular
use them to describe moduli spaces of orthogonal bundles. Furthermore, the
locus of isotropic vectors in $V$ defines a quadric subfibration
$Q_{V}\subset{\mathbb{P}}V$. Using familiar results on quadrics of low
dimension, we exhibit isomorphisms between isotropic Quot schemes of $V$ and
certain ordinary Quot schemes of line subbundles. In particular, for $r\leq 6$
this gives a method for enumerating the isotropic subbundles of maximal degree
of a general $V$, when there are finitely many.

###### Key words and phrases:

Orthogonal vector bundle, curve, quadric fibration, isotropic subbundle

###### 2010 Mathematics Subject Classification:

14H60; 14M17

The first named author was supported by the National Research Foundation of
Korea: NRF-2020R1F1A1A01068699.

##  1\. Introduction

Let $C$ be a complex projective smooth curve of genus $g\geq 2$, and $L\to C$
a line bundle. A vector bundle $V\to C$ is said to be $L$-valued orthogonal if
there is a nondegenerate symmetric bilinear form $\sigma\colon V^{\otimes
2}\to L$. Like symplectic bundles (defined similarly, but with the form being
skew-symmetric), orthogonal bundles are interesting examples of decorated
vector bundles. Moreover, they are associated bundles of principal $G$-bundles
for $G$ an orthogonal group $\mathrm{SO}(r,{\mathbb{C}})$,
$\mathrm{O}(r,{\mathbb{C}})$ or $\mathrm{GO}(r,{\mathbb{C}})$, and provide
useful tools and intuition for questions regarding these principal bundles and
their moduli. See for example [Bea08], [BG10], [CH14], [CH15], [Serm12],
[CCH21b] and many other works.

Orthogonal bundles and their moduli enjoy remarkable geometric properties. One
reason for this is that the group $\mathrm{SO}(r,{\mathbb{C}})$ is not simply
connected as $\mathrm{SL}(r,{\mathbb{C}})$ and $\mathrm{Sp}(2n,{\mathbb{C}})$
are, but has fundamental group ${\mathbb{Z}}_{2}$ and universal covering given
by the spin group $\mathrm{Spin}(r,{\mathbb{C}})$. The point of departure for
the present work is the fact that for $3\leq r\leq 6$, there are exceptional
isomorphisms between $\mathrm{Spin}(r,{\mathbb{C}})$ and other familiar
groups:

$\begin{array}[]{c||c|c|c|c}r&3&4&5&6\\\
\hline\cr\mathrm{Spin}(r,{\mathbb{C}})&\mathrm{SL}(2,{\mathbb{C}})&\mathrm{SL}(2,{\mathbb{C}})\times\mathrm{SL}(2,{\mathbb{C}})&\mathrm{Sp}(4,{\mathbb{C}})&\mathrm{SL}(4,{\mathbb{C}})\end{array}$

Moreover, in each case there are interesting geometric interpretations for the
map $\mathrm{Spin}(r,{\mathbb{C}})\to\mathrm{SO}(r,{\mathbb{C}})$; see for
example [FH91, Lectures 18–19]. Taking a cue from the existence of these
isomorphisms, we describe the structures of orthogonal bundles of rank $r\leq
6$ using sum and tensor operations on bundles of lower rank. This extends the
classification in [Mum71, p. 185–187] of $\mathrm{SO}(r,{\mathbb{C}})$-bundles
for $r\leq 4$. Of particular interest is the fact that topologically,
orthogonal bundles of rank $r$ with trivial determinant are distinguished by
another topological invariant, the second Stiefel–Whitney class $w_{2}(V)\in
H^{2}(C,{\mathbb{Z}}_{2})\cong{\mathbb{Z}}_{2}$. This was characterized in
[Serre90] in terms of theta characteristics of $C$, and we discuss it further
in § 2.3. Here is an overview of the structure theorems for the case
$L={{{\mathcal{O}}}_{C}}$ and $\det(V)\cong{{{\mathcal{O}}}_{C}}$ and
$w_{2}(V)=0$. (The statements for the other cases are given at the relevant
points in the paper; compare also with [Mum71, pp. 185-187].)

$\begin{array}[]{c|cc}\hbox{Rank}&\hbox{Structure}&\\\ \hline\cr 2&N\oplus
N^{-1}:&N\in\mathrm{Pic}(C)\\\ 3&\mathrm{Sym}^{2}E\otimes(\det
E)^{-1}:&\hbox{$E$ of rank two}\\\ 4&E_{1}\otimes E_{2}:&\hbox{$E_{1}$,
$E_{2}$ of rank two, $\det E_{2}\cong(\det E_{1})^{-1}$}\\\
5&\mathrm{Ker}(\wedge^{2}W_{1}\to M):&\hbox{$M$ a square root of
${{{\mathcal{O}}}_{C}}$, and}\\\ &&\hbox{$W_{1}$ a rank four $M$-valued
symplectic bundle}\\\ 6&\wedge^{2}W:&\hbox{$W$ of rank four and trivial
determinant}\end{array}$

These identifications imply in particular the existence of natural maps
between certain moduli spaces of bundles over $C$, which are described in the
course of the paper.

An important point is that when $r=2n$ is even, our proofs rely on the
existence of a rank $n$ isotropic subbundle in $V$. In [CCH21b, § 2], it was
shown that this is equivalent to the condition $\det V\cong L^{n}$ (in
general, the isomorphism $V\xrightarrow{\sim}V^{*}\otimes L$ only gives $(\det
V)^{2}\cong L^{2n}$). Moreover, in [CH14] and [CH15] it was shown that when
$L={{{\mathcal{O}}}_{C}}=\det V$, the invariant $w_{2}(V)$ coincides with the
parity of the degrees of the rank $n$ isotropic subbundles of $V$. In Lemma
3.2 and Theorem 2.3 we offer more elementary proofs of these results.

Turning our attention to the “quadric fibrations” in the title: The locus of
isotropic lines in fibers of an orthogonal bundle $V$ is a subfibration
$Q_{V}\subset{\mathbb{P}}V$ in smooth quadric hypersurfaces, giving
${\mathbb{P}}V$ the structure of an “enveloping bundle” (see § 2.5). For
$r\leq 6$, these quadrics are very well understood; see for example [GH78]:

$\begin{array}[]{c||c|c|c|c|c}r&2&3&4&5&6\\\ \hline\cr\hbox{Quadric}&\hbox{Two
smooth points}&\hbox{Conic
}{\mathbb{P}}^{1}&{\mathbb{P}}^{1}\times{\mathbb{P}}^{1}&\mathrm{LG}(2,4)&\mathrm{Gr}(2,4)\end{array}$

Here $\mathrm{LG}(2,4)\subset\mathrm{Gr}(2,4)$ is the Lagrangian Grassmannian
of two-dimensional subspaces of ${\mathbb{C}}^{4}$ which are isotropic with
respect to a fixed nondegenerate skew-symmetric bilinear form. We use well-
known properties of these quadrics to analyze the structures of orthogonal
bundles, and in particular their isotropic subbundles. For $3\leq r\leq 6$, it
turns out that the isotropic Quot scheme $\mathrm{IQ}^{\circ}_{m,e}(V)$, which
parameterizes isotropic subbundles of rank $m$ and degree $e$ in $V$, is
isomorphic to a union of Quot schemes of subbundles of certain bundles of
lower rank. This allows us in particular to enumerate the maximal isotropic
subbundles of a general orthogonal bundle of rank $r\leq 6$, when the number
is finite. This gives an analog for low rank orthogonal bundles of the
enumeration results in [Hol04] for maximal degree subbundles of a general
vector bundle, and in [CCH21a] for maximal degree Lagrangian subbundles of a
general symplectic bundle.

Some results in the paper are proven in or would follow easily from [CCH21b],
[CH14] and [CH15]. However, our policy is, whenever possible, to give
arguments which are elementary and/or use the rich and familiar geometry of
the low rank situation.

The paper is organized as follows. We begin by recalling background material
on orthogonal bundles and their moduli, isotropic subbundles and isotropic
Quot schemes, and quadric subfibrations of projective bundles. Then we give a
detailed treatment of the rank two case. This is relatively well known and
straightforward, but provides a paradigm for the other cases; moreover, the
rank two case has some useful corollaries for higher rank bundles. We then
turn to the higher rank cases. It emerges that the results in rank three and
five follow naturally from special cases of rank four and six respectively.
Therefore, we treat the latter first in each case.

For the present, as mentioned above, we have restricted ourselves in the even
rank cases to orthogonal bundles which admit isotropic subbundles of half
rank. The structure of bundles not possessing any such subbundle is of
interest, and amenable to study by similar methods. This topic will be studied
in a future project.

### Acknowledgements

We are grateful to Daewoong Cheong for stimulating conversations, and for
sharing valuable knowledge of enumerative geometry. We thank Han-Bom Moon for
a useful suggestion on the morphisms between moduli spaces. The second author
thanks Raquel Mallavibarrena for interesting communication about enveloping
bundles.

##  2\. Preliminaries

###  2.1. Orthogonal bundles and isotropic subbundles

Here we recall background material on orthogonal bundles. For more
information, we refer to [Bea08], [Mum71], [Ram81] and [CCH21b].

Let $C$ be a complex projective smooth curve, and $L\to C$ a line bundle. A
vector bundle $V\to C$ of rank $r$ is said to be $L$-valued orthogonal if
there is a nondegenerate symmetric bilinear form $\sigma\colon V^{\otimes
2}\to L$; equivalently, if there is a symmetric isomorphism
$V\xrightarrow{\sim}V^{*}\otimes L$.

In particular, for such a $V$ we have $(\det V)^{2}\cong L^{r}$. It follows
that if $r=2n+1$ is odd,

(2.1) $\deg(L)\hbox{ is even, and}\det V\cong M^{2n+1}\hbox{ for a uniquely
determined $M$}\\\ \hbox{such that }M^{2}\cong L.$

On the other hand, if $r=2n$ is even, then

(2.2) $\det V\ \cong\ L^{n}\otimes\eta\hbox{ for some $\eta$ satisfying
}\eta^{2}\ \cong\ {{{\mathcal{O}}}_{C}}.$

The question of which $\eta$ arise in (2.2) is linked to the existence of
certain isotropic subbundles, which we now discuss.

Let $V$ be $L$-valued orthogonal of rank $r$. For any subbundle $E\subset V$,
the isomorphism $V\xrightarrow{\sim}V^{*}\otimes L$ induces a sequence $0\to
E^{\perp}\to V\to E^{*}\otimes L\to 0$, where

$E^{\perp}\ :=\ \\{v\in V:\sigma(v\otimes e)=0\hbox{ for all }e\in E\\}$

is the orthogonal complement of $E$ with respect to $\sigma$. A subbundle
$E\subset V$ is isotropic if $E\subseteq E^{\perp}$; equivalently, if
$\sigma(E\otimes E)=0$.

Suppose $\mathrm{rk}\,(V)=2n$ and $V$ contains an isotropic subbundle $E$ of
rank $n$. Then $E=E^{\perp}$ and $V$ is an extension $0\to E\to V\to
E^{*}\otimes L\to 0$. In particular,

(2.3) if $V$ admits a rank $n$ isotropic subbundle, then $\det V\cong L^{n}$;

that is, $\eta={{{\mathcal{O}}}_{C}}$ in (2.2). In fact, by [CCH21b, Lemma
2.5] the converse of (2.3) also holds; in Proposition 3.2, we shall offer an
elementary proof of this fact.

Moreover, in general, not every extension $0\to E\to V\to E^{*}\otimes L\to 0$
arises from an orthogonal structure. We shall use the following
characterization [Hit07, Criterion 2.1].

######  Criterion 2.1.

Let $0\to E\to V\to E^{*}\otimes L\to 0$ be an extension of vector bundles,
with class $\delta(V)\in H^{1}(C,E\otimes E\otimes L^{-1})$. Then $V$ admits
an orthogonal structure with respect to which $E$ is isotropic if and only if,
up to the action of $\mathrm{Aut}\,(E)\times\mathrm{Aut}\,(E^{*}\otimes L)$,
the extension class $\delta(V)$ belongs to $H^{1}(C,\wedge^{2}E\otimes
L^{-1})$.

###  2.2. Moduli of orthogonal bundles

For any line bundle $N$, let ${\mathcal{S}}{\mathcal{U}}_{C}(r,N)$ be the
moduli space of S-equivalence classes of semistable vector bundles of rank $r$
and determinant $N$ over $C$. As this is noncanonically isomorphic to the
moduli space of semistable principal $\mathrm{SL}(2,{\mathbb{C}})$-bundles
over $C$, by [Rth96, Theorem 5.9] it is an irreducible projective variety of
dimension $(r^{2}-1)(g-1)$. We consider the following closed subloci of
${\mathcal{S}}{\mathcal{U}}_{C}(r,{{{\mathcal{O}}}_{C}})$ and
${\mathcal{S}}{\mathcal{U}}_{C}(r,{{{\mathcal{O}}}_{C}}(nx))$ for a fixed
$x\in C$.

  * •

${\mathcal{M}}{\mathcal{O}}_{C}(2n,{{{\mathcal{O}}}_{C}})$, the moduli space
of semistable ${{{\mathcal{O}}}_{C}}$-valued orthogonal bundles of rank $2n$
and trivial determinant

  * •

${\mathcal{M}}{\mathcal{O}}_{C}(2n,{{{\mathcal{O}}}_{C}}(x))$, the moduli
space of semistable ${{{\mathcal{O}}}_{C}}(x)$-valued orthogonal bundles of
rank $2n$ and determinant ${{{\mathcal{O}}}_{C}}(nx)$

  * •

${\mathcal{M}}{\mathcal{O}}_{C}(2n+1,{{{\mathcal{O}}}_{C}})$, the moduli space
of semistable ${{{\mathcal{O}}}_{C}}$-valued orthogonal bundles of rank $2n+1$
and trivial determinant

We shall use the notation ${\mathcal{S}}{\mathcal{U}}_{C}(r,N)_{\mathrm{st}}$
and ${\mathcal{M}}{\mathcal{O}}_{C}(r,L)_{\mathrm{st}}$ when we wish to
restrict to the stable loci. We write $F\sim F^{\prime}$ to indicate that $F$
and $F^{\prime}$ are S-equivalent semistable vector bundles.

Let us explain why we consider only these three kinds of moduli space.
Firstly, when $r=2n+1$, any $L$-valued orthogonal bundle is a twist by a line
bundle of one which is ${{{\mathcal{O}}}_{C}}$-valued with trivial
determinant. When $r=2n$, any $L$-valued orthogonal bundle is a twist by a
line bundle of either an ${{{\mathcal{O}}}_{C}}$-valued or an
${{{\mathcal{O}}}_{C}}(x)$-valued orthogonal bundle. Furthermore, we shall
consider only $L$-valued orthogonal bundles which admit rank $n$ isotropic
subbundles. Hence in view of (2.3), we get the restriction on the determinant
as above for $r=2n$.

The following seems to be well known, but we include a proof for completeness.

######  Lemma 2.2.

Every component of the moduli spaces above is of dimension
$\frac{1}{2}r(r-1)(g-1)$, where $r$ is the rank.

###### Proof.

Recall that the conformal orthogonal group $\mathrm{GO}(r,{\mathbb{C}})$ is
the image in $\mathrm{GL}(r,{\mathbb{C}})$ of the multiplication map
$\mathrm{O}(r,{\mathbb{C}})\times{\mathbb{C}}^{*}\to\mathrm{GL}(r,{\mathbb{C}})$.
As explained in [BG10, § 2], the moduli space of rank $2n$ vector bundles
admitting an $L$-valued symmetric form for some $L$ of degree $\ell$ is the
image of certain components of the moduli space of semistable
$\mathrm{GO}(2n,{\mathbb{C}})$-bundles by the forgetful map to
${\mathcal{U}}\left(r,\frac{r\ell}{2}\right)$. Furthermore, by [BS02, Theorem
8.5 and Remark 8.6], this map is finite. Thus by [Rth96, Theorem 5.9], this
image has dimension

$(g-1)\cdot\dim\mathrm{GO}(r,{\mathbb{C}})+\dim
Z(\mathrm{GO}(r,{\mathbb{C}}))\ =\ \left(\frac{1}{2}r(r-1)+1\right)(g-1)+1.$

Fixing $\ell$ and $L$, we obtain a locus of codimension $g$; that is,
dimension $\frac{1}{2}r(r-1)(g-1)$ as desired. ∎

###  2.3. The second Stiefel–Whitney class

As will be discussed later, the moduli space
${\mathcal{M}}{\mathcal{O}}_{C}(r,{{{\mathcal{O}}}_{C}}(x))$ is irreducible.
However, ${\mathcal{M}}{\mathcal{O}}_{C}(r,{{{\mathcal{O}}}_{C}})$ is in
general reducible, due to the presence of another discrete invariant for
orthogonal bundles which we shall now discuss.

Firstly, we dispose of a special case. The group $\mathrm{SO}(2,{\mathbb{C}})$
is isomorphic to ${\mathbb{C}}^{*}$ via the map

$z\ \mapsto\ \begin{pmatrix}\frac{z+1/z}{2}&\frac{-z+1/z}{2\sqrt{-1}}\\\
\frac{z-1/z}{2\sqrt{-1}}&\frac{z+1/z}{2}\end{pmatrix}.$

Hence it has infinite cyclic fundamental group, unlike
$\mathrm{SO}(r,{\mathbb{C}})$ for $r\geq 3$. Therefore, during this section we
shall assume that $r\geq 3$.

For $r\geq 3$, let
$1\to{\mathbb{Z}}_{2}\to\mathrm{Spin}(r,{\mathbb{C}})\to\mathrm{SO}(r,{\mathbb{C}})\to
1$ be the exact sequence associated to the universal covering of
$\mathrm{SO}(r,{\mathbb{C}})$; see for example [FH91, Lecture 20]. By for
example [Gro58, Chapter 5], this gives rise to a sequence of nonabelian
cohomology sets

$\cdots\ \to\ H^{1}(C,{\mathbb{Z}}_{2})\ \to\
H^{1}(C,\mathrm{Spin}(r,{\mathbb{C}}))\ \to\
H^{1}(C,\mathrm{SO}(r,{\mathbb{C}}))\ \to\ H^{2}(C,{\mathbb{Z}}_{2}).$

Choosing a set of transition functions for $V$ defines an element of
$H^{1}(C,\mathrm{SO}(r,{\mathbb{C}}))$. Then $w_{2}(V)$ is the image of this
element in $H^{2}(C,{\mathbb{Z}}_{2})$. By [Gro58, Proposition 5.7.2], the
class $w_{2}(V)$ is trivial if and only if $V$ is obtained from a
$\mathrm{Spin}(r,{\mathbb{C}})$-bundle by the extension of structure group
given by $\mathrm{Spin}(r,{\mathbb{C}})\to\mathrm{SO}(r,{\mathbb{C}})$. For
$i\in{\mathbb{Z}}_{2}$, we denote by
${\mathcal{M}}{\mathcal{O}}_{C}^{i}(r,{{{\mathcal{O}}}_{C}})$ the component of
${\mathcal{M}}{\mathcal{O}}_{C}(r,{{{\mathcal{O}}}_{C}})$ consisting of
bundles with $w_{2}(V)=i\in\\{0,1\\}$.

The class $w_{2}(V)$ has several properties. Firstly, by [Serre90, Theorem 2],
for every theta characteristic $\kappa$ on $C$ we have

(2.4) $w_{2}(V)\ \equiv\ h^{0}(V\otimes\kappa)+r\cdot h^{0}(\kappa)\mod 2.$

Furthermore, it is shown in [CH14] and [CH15] that $w_{2}(V)$ coincides with
the parity of the degree of any rank $n$ isotropic subbundle of $V$, where
$n=\left\lfloor\frac{\mathrm{rk}\,(V)}{2}\right\rfloor$. We now offer a more
elementary proof of this fact.

######  Theorem 2.3.

Let $V$ be an ${{{\mathcal{O}}}_{C}}$-valued orthogonal bundle of rank $2n$ or
$2n+1$ and trivial determinant. If $F\subset V$ is a rank $n$ isotropic
subbundle, then $w_{2}(V)\equiv\deg(F)\mod 2$. In particular, all rank $n$
isotropic subbundles of $V$ have degree of the same parity.

######  Caution 2.4.

Note that we have not proven that $V$ has any such subbundle $F$. This will be
shown in Proposition 3.2.

###### Proof.

Suppose firstly that $r=2n$. By hypothesis and using Criterion 2.1, there is
an exact sequence $0\to F\to V\to F^{*}\to 0$ such that the cohomology class
$[V]$ belongs to $H^{1}(C,\wedge^{2}F)$. For any theta characteristic
$\kappa$, this yields a cohomology sequence

(2.5) $0\ \to\ H^{0}(C,F\otimes\kappa)\ \to\ H^{0}(C,V\otimes\kappa)\ \to\
H^{0}(C,F^{*}\otimes\kappa)\ \xrightarrow{\cup[V]}\ H^{1}(C,F\otimes\kappa)\
\to\ \cdots.$

Now by Serre duality, there is an identification
$H^{1}(C,F\otimes\kappa)\xrightarrow{\sim}H^{0}(C,F^{*}\otimes\kappa)^{*}$. We
claim that the map $\cup[V]$ is antisymmetric. To see this: It is well known
that the cup product map $\cup\colon H^{1}(C,F\otimes
F)\to\mathrm{Hom}\left(H^{0}(C,F^{*}\otimes\kappa),H^{1}(C,F\otimes\kappa)\right)$
is dual to the Petri map

$\mu\colon H^{0}(C,F^{*}\otimes\kappa)^{\otimes 2}\ \to\ H^{0}(C,F^{*}\otimes
F^{*}\otimes K_{C}).$

Thus we must show that

(2.6)
$\left\langle[V],\mu\left(\mathrm{Sym}^{2}H^{0}(C,F^{*}\otimes\kappa)\right)\right\rangle\
=\ 0,$

the pairing being defined by Serre duality. But since

$\mu\left(\mathrm{Sym}^{2}H^{0}(C,F^{*}\otimes\kappa)\right)\ \subseteq\
H^{0}(C,\mathrm{Sym}^{2}F^{*}\otimes K_{C})$

while $[V]\in H^{1}(C,\wedge^{2}F)\cong H^{0}(C,\wedge^{2}F^{*}\otimes
K_{C})^{*}$, we obtain (2.6).

By the above claim, the map $\cup[V]$ in (2.5) has even rank. We write

$h\ :=\ h^{0}(C,F^{*}\otimes\kappa)\ =\ h^{1}(C,F\otimes\kappa).$

Moreover, $\chi(F\otimes\kappa)=\deg(F)$, so
$h^{0}(C,F\otimes\kappa)=\deg(F)+h$. By exactness of (2.5), then,

$h^{0}(C,V\otimes\kappa)\ =\
h^{0}(C,F\otimes\kappa)+\dim\mathrm{Ker}(\cup[V])\ =\
\left(\deg(F)+h\right)+\left(h-\mathrm{rk}\,(\cup[V])\right).$

Since $\mathrm{rk}\,(\cup[V])$ is even,
$h^{0}(C,V\otimes\kappa)\equiv\deg(F)\mod 2$. The statement now follows from
(2.4), since $r=2n$.

Suppose now that $\mathrm{rk}\,(V)=2n+1$. We use a technique applied
frequently in [CH15] and [CCH21b, § 8], which will appear several times in
this paper. The orthogonal direct sum $V\perp{{{\mathcal{O}}}_{C}}$ is
${{{\mathcal{O}}}_{C}}$-valued orthogonal of rank $2n+2$ and trivial
determinant. Let $E$ be any rank $n$ isotropic subbundle of $V$. Using for
example a local trivialization, we can complete $E$ to a rank $n+1$ isotropic
subbundle $\overline{E}$ of $V$ in two distinct ways, each one an extension
$0\to E\to\overline{E}\to{{{\mathcal{O}}}_{C}}\to 0$. Thus
$\deg(E)=\deg(\overline{E})$. It follows by the even rank case above that

all rank $n$ isotropic subbundles of $V$ have parity equal to
$w_{2}(V\perp{{{\mathcal{O}}}_{C}})$.

On the other hand, an easy computation with (2.4) shows that
$w_{2}(V)=w_{2}(V\perp{{{\mathcal{O}}}_{C}})$. The statement for $r=2n+1$ now
follows. ∎

###  2.4. Isotropic Quot schemes

For any bundle $W$, we have the Quot scheme

$\mathrm{Quot}^{\mathrm{rk}\,(W)-m,\deg(W)-d}(W)$

parameterizing coherent quotients of $W$ of rank $\mathrm{rk}\,(W)-m$ and
degree $\deg(W)-d$; equivalently, locally free subsheaves of $W$ of rank $m$
and degree $d$. We depart from convention and denote this by
$\mathrm{Quot}_{m,d}(W)$. We denote the open subset of saturated subsheaves by
$\mathrm{Quot}^{\circ}_{m,d}(W)$.

If $V$ is an orthogonal bundle of rank $2n$ or $2n+1$, then for $1\leq m\leq
n$ we consider the closed sublocus

$\mathrm{IQ}^{\circ}_{m,e}(V)\ :=\ \left\\{[j\colon E\hookrightarrow
V]\in\mathrm{Quot}^{\circ}_{m,e}(V):\hbox{$j(E)$ isotropic of rank $m$ and
degree $e$}\right\\}.$

Let us say a little more about isotropic subbundles of maximal rank $n$. By
[CCH21b, Proposition 3.3], the expected dimension of
$\mathrm{IQ}^{\circ}_{n,e}(V)$ is

(2.7) $\begin{cases}-(n-1)e-\frac{n(n-1)}{2}(g-1-\ell)&\hbox{ if }r=2n;\\\
-n\left(e+\frac{\ell}{2}\right)-\frac{n(n+1)}{2}(g-1-\ell)&\hbox{ if
}r=2n+1.\end{cases}$

Note that when $r=2n$ and $\ell$ is even, $\mathrm{IQ}^{\circ}_{n,e}(V)$ is in
general disconnected, by [CCH21b, Theorem 1.4 (a)]. In Corollary 4.7 and
Corollary 6.10, we shall see this concretely for $r=4$ and $r=6$ respectively.

#### Zero dimensional Quot schemes

Of particular interest for us will be the question of enumerating the points
of $\mathrm{IQ}^{\circ}_{n,e}(V)$ for a general $V$ when this scheme is
finite. This corresponds to the maximal value of $e$ for which
$\mathrm{IQ}^{\circ}_{n,e}(V)$ is nonempty. The analogous question for maximal
subbundles of vector bundles was answered in [Hol04, Theorem 4.2], and for
maximal Lagrangian subbundles of symplectic bundles in [CCH21a]. We quote a
special case [Hol04, Corollary 4.3] which we shall use later (see also [Oxb00,
Lemma 2.2]).

######  Theorem 2.5.

Let $W$ be a general stable bundle of rank $r$. If $\deg(W)\equiv 1-g\mod r$
then $W$ has $r^{g}$ maximal line subbundles of degree
$\frac{1}{r}\left(\deg(W)-(r-1)(g-1)\right)$.

Using (2.7), we now decide which $\mathrm{IQ}^{\circ}_{n,e}(V)$ have expected
dimension zero. The following can be viewed as a special case of [CCH21b,
Theorem 1.4 and 1.5].

######  Lemma 2.6.

Write $\ell:=\deg(L)$. We assume that $\ell\in\\{0,\frac{r}{2}\\}$ when $r$ is
even, and $\ell=0$ when $r$ is odd. Assume that $V$ is $L$-valued orthogonal
of rank $r$ and general in moduli. In the range $3\leq r\leq 6$, the scheme
$\mathrm{IQ}^{\circ}_{n,e}(V)$ has expected dimension zero if the following
conditions hold:

$\begin{array}[]{|r|c|}\hline\cr r=3&e=-g+1\\\ \hline\cr r=4&e=\ell-g+1\\\
\hline\cr r=5&g\text{ is odd and }e=-\frac{3}{2}(g-1)\\\ \hline\cr
r=6&\ell-g\text{ is odd and }e=\frac{3}{2}(\ell-g+1)\\\ \hline\cr\end{array}$

In these cases, each component (that is, point) of
$\mathrm{IQ}^{\circ}_{n,e}(V)$ corresponds to an isotropic subbundle of $V$ of
rank $\lfloor\frac{r}{2}\rfloor$ and degree $e$. Later we shall give
enumerative results on the number of such isotropic subbundles of rank $m$,
for $1\leq m\leq n$.

###  2.5. Quadric fibrations

Finally, we discuss a certain quadric subfibration $Q_{V}\subset{\mathbb{P}}V$
for an orthogonal bundle $V$. This is closely related to the notion of an
enveloping bundle as studied in [LM20] and [LMP15], to which we refer the
reader for more information. Note however that our situation is slightly
different, as we begin with the vector bundle $V$ instead of the quadric
fibration $Q_{V}$; and we do not consider ${\mathbb{P}}V$ or $Q_{V}$ in a
projective embedding, but only $Q_{V}$ relatively embedded in ${\mathbb{P}}V$
over $C$.

Let $V\to C$ be a vector bundle of rank $r\geq 2$, and
$\pi\colon{\mathbb{P}}V\to C$ the associated projective bundle. Let $\sigma$
be a nonzero element of $H^{0}(C,\mathrm{Sym}^{2}V^{*}\otimes L)\cong
H^{0}({\mathbb{P}}V,{\mathcal{O}}_{{\mathbb{P}}V}(2)\otimes\pi^{*}L)$. We
denote by $Q_{V}$ the divisor $(\sigma)\subset{\mathbb{P}}V$. It is a quadric
fibration over $C$. If we further assume that $\sigma|_{p}$ defines a
symmetric isomorphism $V|_{p}\to(V^{*}\otimes L)|_{p}$ for each fiber, then
$\sigma$ defines an $L$-valued orthogonal structure on $V$, and $Q_{V}\to C$
is a fibration in _smooth_ quadrics. (This was observed for $r=3$ and
$C={\mathbb{P}}^{1}$ in [LM20, Remark 1, p. 9].) In this case,
$Q_{V}\subset{\mathbb{P}}V$ parameterizes lines in fibers of $V$ which are
isotropic with respect to the quadratic form $\sigma$.

A useful application of the above is:

######  Proposition 2.7.

Let $V$ be an $L$-valued orthogonal bundle of rank $r\geq 3$. Then $V$ admits
an isotropic line subbundle.

###### Proof.

The fibers of $Q_{V}$ are quadric hypersurfaces in ${\mathbb{P}}^{r-1}$, so
are rationally connected for $r\geq 3$. Thus by [GHS03], there is a section
$C\to Q_{V}\cong\mathrm{OG}(1,V)$, defining an isotropic line subbundle of
$V$. ∎

##  3\. Rank two

We now consider orthogonal bundles of rank two. These results are
straightforward and can be obtained without mentioning quadric fibrations, but
we do this as practice for the higher rank cases. Firstly, we give a variation
on [CCH21b, Proposition 2.10].

######  Theorem 3.1.

Let $V$ be an $L$-valued orthogonal bundle of rank two and determinant $L$.
Then $V\cong N\oplus N^{-1}L$ for some line bundle $N$. Moreover, $V$ has
precisely two isotropic subbundles, which are isomorphic to $N$ and $N^{-1}L$
respectively.

###### Proof.

Let $N$ be a line subbundle of maximal degree in $V$. Then the orthogonal form
on $V$ induces the surjective bundle map $V\to N^{-1}L$ whose kernel is
$N^{\perp}$. Thus we have $N^{\perp}\cong N$.

If $N$ is isotropic, Criterion 2.1 shows that $V$ is isomorphic to an
extension $0\to N\to V\to N^{-1}L\to 0$ determined by a class in
$H^{1}(C,L^{-1}\otimes\wedge^{2}N)$. As $\wedge^{2}N=0$, we have $V\cong
N\oplus N^{-1}L$.

If $N$ is not isotropic, then $N\oplus N^{\perp}$ is a rank two subsheaf of
$V$. Thus in fact $N\oplus N$ is a subsheaf of $V$. As $\deg(N)$ was assumed
to be maximal, this does not drop rank anywhere, so $V\cong N\oplus N$. Since
$\det V\cong L$, we have $N\cong N^{-1}L$. In particular, $E\cong
N\otimes{\mathcal{O}}_{C}^{\oplus 2}$, where $N$ is an orthogonal line
subbundle. Hence by choosing a copy of $N\subset V$ whose fiber is isotropic,
we can also write $E\cong N\oplus N^{-1}L$, with respect to which the summands
are the isotropic subbundles. ∎

We now investigate the structure of ${\mathbb{P}}V$ as an enveloping bundle,
where $V$ is $L$-valued orthogonal of rank two and determinant $L$. Here the
locus $Q_{V}$ of isotropic lines in ${\mathbb{P}}V$ is a fibration in smooth
zero-dimensional quadrics; that is, pairs of points. By Theorem 3.1, we have

$Q_{V}\ =\ {\mathbb{P}}N\sqcup{\mathbb{P}}(N^{-1}L)\ \cong\ C\sqcup C.$

An isotropic line subbundle corresponds to a section of ${\mathbb{P}}V$
belonging to $C\sqcup C$ at each point; that is, one of the copies of $C$.
Thus $\mathrm{IQ}_{1,e}(V)^{\circ}$ is empty unless $e=\deg(N)$ or
$e=\deg(N^{-1}L)$. (If $e=\deg(N)=\frac{\ell}{2}$ then it consists of two
points.)

As applications of our results so far, and to make the present paper more
self-contained, we offer alternative proofs of some results in [CCH21b] and
[CH15] which we shall require later.

######  Proposition 3.2.

Let $V$ be an $L$-valued orthogonal bundle of rank $2n$. Then $V$ admits an
isotropic subbundle of rank $n$ if and only if $\det V\cong L^{n}$.

###### Proof.

(See also [CCH21b, Lemma 2.5].) The “only if” implication is (2.3). For “if”,
we proceed by induction on $n$. The case $n=1$ follows from Theorem 3.1.
Suppose $n\geq 2$. By Proposition 2.7, we can find an isotropic line subbundle
$H\subset V$. Then $H^{\perp}/H$ is $L$-valued orthogonal of rank $2n-2$. From
the exact sequence $0\to H^{\perp}\to V\to H^{-1}L\to 0$, we deduce that

$\det\left(H^{\perp}/H\right)\ \cong\ \det(H^{\perp})\otimes H^{-1}\ \cong\
(\det V)\otimes H\otimes L^{-1}\otimes H^{-1}\ \cong\ (\det V)\otimes L^{-1}\
\cong\ L^{n-1}.$

By induction, $H^{\perp}/H$ has an isotropic subbundle $\overline{E}$ of rank
$n-1$. The inverse image of $\bar{E}$ in $H^{\perp}\subset V$ is a rank $n$
isotropic subbundle of $V$. ∎

We now give an analog of Theorem 2.3 for the odd degree case (compare with
[CCH21b, Theorem 1.4 (b)]).

######  Lemma 3.3.

Suppose $\deg(L)$ is odd, and let $V$ be $L$-valued orthogonal of rank $2n$.
Suppose $V$ satisfies the equivalent conditions of Proposition 3.2. Then $V$
has isotropic rank $n$ subbundles both of even degree and of odd degree.

###### Proof.

Let $E\subset V$ be a rank $n$ isotropic subbundle. Let $H\subset E$ be any
rank $n-1$ subbundle, which is necessarily isotropic. Then $H^{\perp}/H$ is
$L$-valued orthogonal of rank two and determinant $L$, and admits $E/H$ as an
isotropic line subbundle. By Theorem 3.1, in fact

$\frac{H^{\perp}}{H}\ \cong\
\frac{E}{H}\oplus\left(\left(\frac{E}{H}\right)^{*}\otimes L\right).$

Then $(E/H)^{*}\otimes L$ is of the form $F/H$ for a rank $n$ isotropic
subbundle $F\subset H^{\perp}\subset V$, and

$\deg(F)\ =\ -(\deg(E)-\deg H)+\deg(L)+\deg(H)\ \equiv\ \deg(E)+\deg(L)\mod
2.$

As $\deg(L)$ is assumed to be odd, $\deg(F)$ and $\deg(E)$ have opposite
parity. ∎

##  4\. Rank four

We now turn our attention to orthogonal bundles of rank four, as the rank
three case turns out to follow from a special case of rank four.

###  4.1. Rank four orthogonal bundles with rank two isotropic subbundles

As noted in [Mum71, p. 186], given two bundles $E_{1}$ and $E_{2}$ of rank
two, the pairing

(4.1) $(E_{1}\otimes E_{2})\otimes(E_{1}\otimes E_{2})\ \to\
\wedge^{2}E_{1}\otimes\wedge^{2}E_{2}\ =:\ L$

defines an $L$-valued orthogonal structure on $E_{1}\otimes E_{2}$.
Furthermore, for any line subbundles $N_{1}\subset E_{1}$ and $N_{2}\subset
E_{2}$, the subbundles $N_{1}\otimes E_{2}$ and $E_{1}\otimes N_{2}$ are
isotropic of rank two. We shall now see that this induces finite surjective
morphisms of moduli spaces.

######  Theorem 4.1.

  1. (a)

The association $(F_{1},F_{2})\mapsto F_{1}\otimes F_{2}$ defines finite
surjective morphisms

$\Theta_{4}^{0}\colon{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}^{0}(4,{{{\mathcal{O}}}_{C}})$

and

$\Theta_{4}^{1}\colon{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}}(x))\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}}(-x))\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}^{1}(4,{{{\mathcal{O}}}_{C}}).$

In particular, ${\mathcal{M}}{\mathcal{O}}_{C}(4,{{{\mathcal{O}}}_{C}})$ has
two irreducible components.

  2. (b)

For $L\cong{{{\mathcal{O}}}_{C}}(x)$: The association $(F_{1},F_{2})\mapsto
F_{1}\otimes F_{2}$ defines a finite surjective morphism

$\Phi_{4}\colon{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}}(x))\to{\mathcal{M}}{\mathcal{O}}_{C}(4,{{{\mathcal{O}}}_{C}}(x))$

In particular, ${\mathcal{M}}{\mathcal{O}}_{C}(4,{{{\mathcal{O}}}_{C}}(x))$ is
irreducible.

###### Proof.

Let us firstly define the morphism $\Theta_{4}^{0}$. As ${\mathbb{C}}$ has
characteristic zero, $F_{1}\otimes F_{2}$ is semistable if $F_{1}$ and $F_{2}$
are so. Therefore, passing to a suitable étale cover, we can define
$\Theta_{4}^{0}$ on the open subset
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$
using a Poincaré bundle. Note that by Theorem 2.3, the images of
$\Theta_{4}^{0}$ and $\Theta_{4}^{1}$ are contained in
${\mathcal{M}}{\mathcal{O}}_{C}^{0}(4,{{{\mathcal{O}}}_{C}})$ and
${\mathcal{M}}{\mathcal{O}}_{C}^{1}(4,{{{\mathcal{O}}}_{C}})$, respectively.

Now it is straightforward to check that if $F_{1}\sim F_{1}^{\prime}$ and
$F_{2}\sim F_{2}^{\prime}$, then $F_{1}\otimes F_{2}\sim F_{1}^{\prime}\otimes
F_{2}^{\prime}$. Hence, as any semistable bundle is a limit of stable ones, it
follows that $\Theta_{4}^{0}$ has a continuous extension to
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$.
Now since $\Theta_{4}^{0}$ is algebraic on
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$,
it is bounded on a neighborhood of any point. Therefore, by the Riemann
extension theorem, the continuous extension to
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$
given by the association $(F_{1},F_{2})\mapsto F_{1}\otimes F_{2}$ is a
morphism.

Now by [DN89, Théorème B (b)], the moduli space
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$ has Picard number 1.
Let ${\mathcal{N}}$ be any very ample line bundle on
${\mathcal{M}}{\mathcal{O}}_{C}^{0}(4,{{{\mathcal{O}}}_{C}})$. Then
$\left(\Theta_{4}^{0}\right)^{*}{\mathcal{N}}\cong{\mathcal{L}}^{a}\boxtimes{\mathcal{L}}^{b}$
for some integers $a,b$, where ${\mathcal{L}}$ is the ample generator of
$\mathrm{Pic}\left({\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\right)$.
Since $\Theta_{4}^{0}$ is symmetric, we have $a=b>0$. Therefore,
$\Theta_{4}^{0}$ must be a finite morphism. As
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\times{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$
and ${\mathcal{M}}{\mathcal{O}}_{C}^{0}(4,{{{\mathcal{O}}}_{C}})$ are
projective varieties of the same dimension, $\Theta_{4}^{0}$ is surjective.

Similar arguments show that $\Theta_{4}^{1}$ and $\Phi_{4}$ are finite
surjective morphisms. (Note that since
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}}(\pm x))$ is a fine
moduli space, we can define $\Theta_{4}^{1}$ without using the Riemann
extension theorem.) The irreducibility statements follow from the fact that
${\mathcal{S}}{\mathcal{U}}_{C}(2,L)$ is irreducible. ∎

It is natural to ask if an arbitrary (possibly unstable) $L$-valued orthogonal
bundle of rank four which admits an isotropic subbbundle of rank two can be
obtained in this way. The answer is Yes, and the proof of the following
theorem shows how to find the expression(s).

######  Theorem 4.2.

Let $V$ be a rank four $L$-valued orthogonal bundle. Then $V$ admits a rank
two isotropic subbundle if and only if $V$ is isomorphic to $E_{1}\otimes
E_{2}$ for some rank two bundle $E_{2}$ with $\det E_{2}\cong\det
E_{1}^{*}\otimes L$.

###### Proof.

One implication follows from (4.1) and the discussion after it. Conversely,
suppose $E_{1}\subset V$ is a rank two isotropic subbundle. Then Criterion 2.1
shows that $V$ is isomorphic to an extension $0\to E_{1}\to V\to
E_{1}^{*}\otimes L\to 0$ defined by a class

$\delta\ \in\ H^{1}(C,\wedge^{2}E_{1}\otimes L^{*})\ \subset\
H^{1}(C,E_{1}\otimes E_{1}\otimes L^{*}).$

Now $\delta$ also defines an extension of line bundles
$0\to{{{\mathcal{O}}}_{C}}\to E_{2}\to\det E_{1}^{*}\otimes L\to 0$. Tensoring
by $E_{1}$, we obtain an extension $0\to E_{1}\to E_{1}\otimes E_{2}\to
E_{1}\otimes\det E_{1}^{*}\otimes L\to 0$ of class

$\mathrm{Id}_{E_{1}}\otimes\delta\ \in\
H^{1}(C,({{{\mathcal{O}}}_{C}}\cdot\mathrm{Id}_{E_{1}})\otimes\det
E_{1}\otimes L^{*})\ \subset\ H^{1}(C,\mathrm{End}(E_{1})\otimes\det
E_{1}\otimes L^{*}).$

Now since $E_{1}$ has rank two, there is an isomorphism
$\mathrm{Hom}(E_{1},E_{1}\otimes\det E_{1})\xrightarrow{\sim}E_{1}\otimes
E_{1}$ inducing an isomorphism of split exact sequences

$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{({{{\mathcal{O}}}_{C}}\cdot\mathrm{Id}_{E_{1}})\otimes\det
E_{1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\wr}$$\textstyle{\mathrm{End}(E_{1})\otimes\det
E_{1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\wr}$$\textstyle{\mathrm{End}_{0}(E_{1})\otimes\det
E_{1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{\wr}$$\textstyle{0}$$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\wedge^{2}E_{1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{E_{1}\otimes
E_{1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\mathrm{Sym}^{2}E_{1}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0.}$

Under the induced identification $H^{1}(C,\wedge^{2}E_{1}\otimes
L^{*})\xrightarrow{\sim}H^{1}(C,{{{\mathcal{O}}}_{C}}\cdot\mathrm{Id}_{E_{1}}\otimes\det
E_{1}\otimes L^{*})$, the extension class $\mathrm{Id}_{E_{1}}\otimes\delta$
corresponds simply to $\delta$. This shows that $V$ and $E_{1}\otimes E_{2}$
are isomorphic vector bundles. Also the natural orthogonal structure of $V$
induced from the extension $\delta$ coincides with that of $E_{1}\otimes
E_{2}$ in view of Criterion 2.1. ∎

###  4.2. Quadric fibration and isotropic subbundles

Let $V=E_{1}\otimes E_{2}$ be an $L$-valued orthogonal bundle as above. A
computation using (4.1) shows that the isotropic vectors in $E_{1}\otimes
E_{2}$ are exactly those of the form $e_{1}\otimes e_{2}$ for $e_{i}\in
E_{i}$. Therefore:

######  Proposition 4.3.

The quadric subfibration $Q_{V}\subset{\mathbb{P}}V$ parameterizing isotropic
lines is the image of the relative Veronese embedding
${\mathbb{P}}E_{1}\times_{C}{\mathbb{P}}E_{2}\ \hookrightarrow\
{\mathbb{P}}(E_{1}\otimes E_{2})$.

In particular, ${\mathbb{P}}V$ is an enveloping bundle for
${\mathbb{P}}E_{1}\times_{C}{\mathbb{P}}E_{2}$. Moreover, we observe:

######  Remark 4.4.

  1. (a)

There are natural projections $\rho_{1}\colon Q_{V}\to{\mathbb{P}}E_{1}$ and
$\rho_{2}\colon Q_{V}\to{\mathbb{P}}E_{2}$, which are morphisms over $C$.

  2. (b)

Via the identification $E_{1}\otimes
E_{2}\xrightarrow{\sim}\mathrm{Hom}(E_{1}^{*},E_{2})$, the locus $Q_{V}$ is
the projectivization of the set of maps of rank one.

  3. (c)

The two-dimensional isotropic subspaces of a fiber $E_{1}\otimes E_{2}|_{p}$
are exactly those of the form $\Lambda_{1}\otimes E_{2}|_{p}$ and
$E_{1}|_{p}\otimes\Lambda_{2}$, where $\Lambda_{i}$ is a line in $E_{i}|_{p}$.

We now use the enveloping bundle structure to study isotropic subbundles of
$V$.

######  Proposition 4.5.

Suppose $V=E_{1}\otimes E_{2}$ as above. Let $F\subset V$ be a rank two
isotropic subbundle. Then $F$ is either of the form $E_{1}\otimes N_{2}$ for a
line bundle $N_{2}\subset E_{2}$, or $N_{1}\otimes E_{2}$ for a line bundle
$N_{1}\subset E_{1}$. In each case, $N_{i}$ is uniquely determined as a
subbundle of $E_{i}$.

###### Proof.

The inclusion $F\hookrightarrow V$ induces a map
${\mathbb{P}}F\hookrightarrow{\mathbb{P}}V$ of ruled varieties over $C$. By
isotropy, ${\mathbb{P}}F$ is contained in $Q_{V}$, so we have the composed
maps ${\mathbb{P}}F\hookrightarrow
Q_{V}\xrightarrow{\rho_{i}}{\mathbb{P}}E_{i}$. Clearly at least one of these
is dominant, and hence surjective, being a morphism of projective varieties.
Moreover, since $F$ is isotropic, by Remark 4.4 (c), for all $p\in C$ the
fiber $F|_{p}$ is of the form $\Lambda_{1}\otimes E_{2}|_{p}$ or
$E_{1}|_{p}\otimes\Lambda_{2}$ for some line $\Lambda_{i}\subset E_{i}|_{p}$.
Thus ${\mathbb{P}}F$ cannot project surjectively to both ${\mathbb{P}}E_{1}$
and ${\mathbb{P}}E_{2}$, so exactly one of the projections
${\mathbb{P}}F\to{\mathbb{P}}E_{i}$ is surjective.

After reordering if necessary, we may assume that
${\mathbb{P}}F\cong{\mathbb{P}}E_{1}$. As ${\mathbb{P}}F$ is not dominant over
${\mathbb{P}}E_{2}$, for general $p\in C$ the fiber ${\mathbb{P}}F|_{p}\subset
Q_{V}|_{p}$ is the line

${\mathbb{P}}E_{1}|_{p}\times\\{\mu(p)\\}\ \subset\
{\mathbb{P}}E_{1}|_{p}\times{\mathbb{P}}E_{2}|_{p}$

for some $\mu(p)\in{\mathbb{P}}E_{2}|_{p}$. We thus obtain a section
$\mu\colon C\to{\mathbb{P}}E_{2}$, defining a line subbundle $N_{2}\subset
E_{2}$, and $F=E_{1}\otimes N_{2}$. ∎

######  Remark 4.6.

If ${\mathbb{P}}E_{1}\cong{\mathbb{P}}E_{2}$ then $E_{1}\otimes E_{2}$ is a
twist of
$\mathrm{End}\,E_{1}={{{\mathcal{O}}}_{C}}\oplus\mathrm{End}_{0}\,E_{1}$, and
in particular is strictly semistable. Thus if $V=E_{1}\otimes E_{2}$ is a
stable vector bundle, then ${\mathbb{P}}E_{1}\not\cong{\mathbb{P}}E_{2}$. The
situation where ${\mathbb{P}}E_{1}\cong{\mathbb{P}}E_{2}$ will arise in the
next section in the context of rank three orthogonal bundles.

In what follows, we follow the convention that a Quot scheme
$\mathrm{Quot}_{r,d}(W)$ is empty if $d$ is not an integer.

######  Corollary 4.7.

Let $V=E_{1}\otimes E_{2}$ be a rank four $L$-valued orthogonal bundle as
above. Write $d_{i}:=\deg(E_{i})$. For each $e$, there is an isomorphism

$\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e-d_{2})}(E_{1})\ \sqcup\
\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e-d_{1})}(E_{2})\ \xrightarrow{\sim}\
\mathrm{IQ}^{\circ}_{2,e}(V),$

via the map

$\begin{cases}N\mapsto N\otimes E_{2}\hbox{ if }N\subset E_{1}\\\ N\mapsto
E_{1}\otimes N\hbox{ if }N\subset E_{2}.\end{cases}$

######  Remark 4.8.

Notice that $d_{1}+d_{2}=\ell$. Then a Riemann–Roch computation shows that
$\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e-d_{2})}(E_{1})$ and
$\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e-d_{1})}(E_{2})$ have expected
dimension $\ell-e-(g-1)$. This coincides with the expected dimension of
$\mathrm{IQ}^{\circ}_{2,e}(V)$ given in (2.7), agreeing with Corollary 4.7.

######  Remark 4.9.

Note that the ruled surface ${\mathbb{P}}E_{1}$ arises both as the
projectivization of the isotropic subbundle $E_{1}$ and as a component of the
isotropic Grassmannian bundle
$\mathrm{OG}(V)\subset{\mathbb{P}}(\wedge^{2}V)$; that parameterizing
isotropic planes of the form $(N\otimes E_{2})|_{p}$ for $p\in C$.

As shown by Theorem 2.3, if $L={{{\mathcal{O}}}_{C}}$ then all rank two
isotropic subbundles have degree of the same parity (which coincides with the
invariant $w_{2}(V)$, by Theorem 2.3). The above results make this
particularly visible for bundles of rank four:

######  Corollary 4.10.

Let $V=E_{1}\otimes E_{2}$, where $\det E_{1}\otimes\det
E_{2}\cong{{{\mathcal{O}}}_{C}}$. Then $E_{1}$ and $E_{2}$ have degrees of the
same parity, which is also the parity of any rank two isotropic subbundle of
$E_{1}\otimes E_{2}$.

###### Proof.

We have $\deg(E_{1})\equiv\deg(E_{2})\mod 2$ since $\det E_{1}\otimes\det
E_{2}\cong{{{\mathcal{O}}}_{C}}$. By Proposition 4.5, any rank two isotropic
subbundle $F\subset V$ is isomorphic to $E_{i}\otimes N$ for some line
subbundle $N$ and for some $i\in\\{1,2\\}$. Thus

$\deg(F)\ =\ \deg(E_{i}\otimes N)\ =\ \deg(E_{i})+2\deg N\ \equiv\
\deg(E_{i})\mod 2.\qed$

We turn now to isotropic line subbundles. By Proposition 4.3, an isotropic
line subbundle $N$ of $V=E_{1}\otimes E_{2}$ corresponds to a section of
$Q_{V}={\mathbb{P}}E_{1}\times_{C}{\mathbb{P}}E_{2}$. Hence $N$ is of the form
$N_{1}\otimes N_{2}$ where $N_{i}$ is a line subbundle of $E_{i}$ for $i=1,2$.
Therefore, we obtain at once:

######  Theorem 4.11.

Let $V=E_{1}\otimes E_{2}$ be as above. Then $(N_{1},N_{2})\mapsto
N_{1}\otimes N_{2}$ defines an isomorphism

$\bigcup_{d_{1}+d_{2}=d}\mathrm{Quot}^{\circ}_{1,d_{1}}(E_{1})\times\mathrm{Quot}^{\circ}_{1,d_{2}}(E_{2})\
\xrightarrow{\sim}\ \mathrm{IQ}^{\circ}_{1,d}(E_{1}\otimes E_{2}).$

###  4.3. Enumeration of maximal degree isotropic subbundles

Firstly, we discuss the number of isotropic subbundles of rank two with
maximal degree. In particular, we obtain another proof of Lemma 2.6 for $r=4$.

######  Theorem 4.12.

Let $V$ be a stable rank four orthogonal bundle which is general in its
component of moduli.

  1. (a)

Suppose
$V\in{\mathcal{M}}{\mathcal{O}}_{C}(4,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$.
Then the number of isotropic subbundles of rank two of maximal degree is given
as follows.

$\begin{array}[]{c|c|c}\hline\cr&{\mathcal{M}}{\mathcal{O}}_{C}^{0}(4,{{{\mathcal{O}}}_{C}})&{\mathcal{M}}{\mathcal{O}}_{C}^{1}(4,{{{\mathcal{O}}}_{C}})\\\
\hline\cr g\ \text{even}&\infty\ \text{of degree }-g&2\cdot 2^{g}\ \text{of
degree }1-g\\\ \hline\cr g\ \text{odd}&2\cdot 2^{g}\ \text{of degree
}1-g&\infty\ \text{of degree }-g\\\ \hline\cr\end{array}$

  2. (b)

Suppose
$V\in{\mathcal{M}}{\mathcal{O}}_{C}(4,{{{\mathcal{O}}}_{C}}(x))_{\mathrm{st}}$.
Then $V$ admits $2^{g}$ isotropic subbundles of rank two and degree $2-g$.

  3. (c)

Suppose $g\geq 3$ is odd. Then $V$ has $4^{g}$ maximal isotropic line
subbundles of degree $1-g$.

###### Proof.

(a) A computation using Corollary 4.7 shows that for each value of $w_{2}(V)$,
the scheme of maximal rank two isotropic subbundles of $V=E_{1}\otimes E_{2}$
is canonically identified with the union of the Quot schemes of maximal line
subbundles of $E_{1}$ and $E_{2}$. By Theorem 4.1, we may assume that $E_{1}$
and $E_{2}$ are general in moduli. The statement now follows from Theorem 2.5.
The proof of (b) is similar.

As for (c): It follows from Theorem 4.11 that the degree of an isotropic line
subbundle $N_{1}\otimes N_{2}\subset E_{1}\otimes E_{2}$ is maximal among the
isotropic line subbundles if and only if $N_{i}$ is a maximal line subbundle
of $E_{i}$ for $i=1,2$. By Theorem 2.5, if $g$ is odd, then a general
$E_{i}\in{\mathcal{S}}{\mathcal{U}}_{C}(2,{\mathcal{O}}_{C})$ has $2^{g}$
maximal line subbundles of degree $-\frac{1}{2}(g-1)$. Statement (c) follows.
∎

##  5\. Rank three

Suppose $V$ is an $L$-valued orthogonal bundle of rank three. By (2.1), the
line bundle $L$ has even degree and $\det V\cong M^{3}$ for some $M$
satisfying $M^{2}\cong L$. We now analyse the structure of orthogonal bundles
of rank three, expanding upon [Mum71, p. 185] and [CH15, Lemma 4.1].

###  5.1. Rank three orthogonal bundles

As noted in the case $L={{{\mathcal{O}}}_{C}}$ in [Mum71, § 2], for any rank
two bundle $E$, the bundle $M\otimes(\det E)^{-1}\otimes\mathrm{Sym}^{2}E$ has
an $L$-valued quadratic form induced by

(5.1) $(e_{1}\cdot e_{2})\otimes(f_{1}\cdot f_{2})\ \mapsto\
2\left(e_{1}\wedge f_{1}\otimes e_{2}\wedge f_{2}+e_{2}\wedge f_{1}\otimes
e_{1}\wedge f_{2}\right).$

We shall see that this induces finite surjective morphisms of moduli spaces.
Noting that any rank three orthogonal bundle is a twist of an
${{{\mathcal{O}}}_{C}}$-valued orthogonal bundle of determinant
${{{\mathcal{O}}}_{C}}$, we now set $L=M={{{\mathcal{O}}}_{C}}$ and describe
the moduli space ${\mathcal{M}}{\mathcal{O}}_{C}(3,{{{\mathcal{O}}}_{C}})$ in
more detail.

######  Theorem 5.1.

  1. (a)

The association $E\mapsto\mathrm{Sym}^{2}E$ defines a finite surjective
morphism

$\Theta_{3}^{0}\colon{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}^{0}(3,{{{\mathcal{O}}}_{C}}).$

  2. (b)

For any $x\in C$, the association
$E\mapsto\mathrm{Sym}^{2}E\otimes{{{\mathcal{O}}}_{C}}(-x)$ defines a finite
surjective morphism

$\Theta_{3}^{1}\colon{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}}(x))\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}^{1}(3,{{{\mathcal{O}}}_{C}}).$

###### Proof.

Let us firstly define the morphism $\Theta_{3}^{0}$. As ${\mathbb{C}}$ has
characteristic zero, $\mathrm{Sym}^{2}E$ is semistable if $E$ is. Therefore,
passing to a suitable étale cover, we can define $\Theta_{3}^{0}$ on the open
subset ${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$
using a Poincaré bundle. Note that the bundle $\mathrm{Sym}^{2}E$ and
$\mathrm{Sym}^{2}E\otimes{{{\mathcal{O}}}_{C}}(-x)$ have isotropic line
subbundles of the form $N^{2}$ and $N^{2}(-x)$ respectively. Hence by Theorem
2.3, the images of $\Theta_{3}^{0}$ and $\Theta_{3}^{1}$ land on the
components ${\mathcal{M}}{\mathcal{O}}_{C}^{0}(3,{{{\mathcal{O}}}_{C}})$ and
${\mathcal{M}}{\mathcal{O}}_{C}^{1}(3,{{{\mathcal{O}}}_{C}})$, respectively.

Now, using for example [Har77, Exercise II.5.16], it is straightforward to
check that if $E\sim E^{\prime}$, then
$\mathrm{Sym}^{2}E\sim\mathrm{Sym}^{2}E^{\prime}$. Hence, as any semistable
bundle is a limit of stable ones, it follows that $\Theta^{3}_{0}$ has a
continuous extension to
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$. As in the proof of
Theorem 4.1, we can show that $\Theta_{3}^{0}$ is a morphism by using the
Riemann extension theorem.

As already noted, the moduli space
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$ has Picard number 1.
Let ${\mathcal{N}}$ be any very ample line bundle on
${\mathcal{M}}{\mathcal{O}}_{C}^{0}(3,{{{\mathcal{O}}}_{C}})$. Then
$\left(\Theta_{3}^{0}\right)^{*}{\mathcal{N}}\cong{\mathcal{L}}^{a}$ for some
positive integer $a$, where ${\mathcal{L}}$ is as before the ample generator
of
$\mathrm{Pic}\left({\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})\right)$.
Therefore, as $\Theta_{3}^{0}$ is a morphism, it must be finite. Counting
dimensions of the projective varieties
${\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}})$ and
${\mathcal{M}}{\mathcal{O}}_{C}^{0}(3,{{{\mathcal{O}}}_{C}})$, it must be
surjective. This proves (a). The proof of (b) is similar. ∎

As in the rank four case, the following theorem ensures that an arbitrary
(possibly non-stable) $L$-valued orthogonal bundle of rank three has a similar
expression, and its proof shows how to find it.

######  Theorem 5.2.

Let $V$ be a rank three $L$-valued orthogonal bundle of determinant $M^{3}$ as
above. Then $V$ is isomorphic as an orthogonal bundle to
$\mathrm{Sym}^{2}E\otimes(\det E)^{-1}\otimes M$ for some rank two bundle $E$.

###### Proof.

We use an idea from [CH15], also applied in [CCH21b]. The isomorphism
$M^{2}\xrightarrow{\sim}L$ gives $M$ the structure of an $L$-valued orthogonal
bundle. The orthogonal direct sum $V\perp M$ is then $L$-valued orthogonal of
rank four and determinant $M^{4}=L^{2}$. By Proposition 3.2, we can find a
rank two isotropic subbundle in $V\perp M$. By Theorem 4.2, there exist rank
two bundles $E_{1}$, $E_{2}$ such that $V\perp M\cong E_{1}\otimes E_{2}$.

Now the subbundle $M\subset E_{1}\otimes E_{2}$ is nowhere isotropic, being an
orthogonal direct summand. But as noted in Remark 4.4 (b), the locus of
isotropic vectors corresponds precisely to the locus of maps $E_{1}^{*}\to
E_{2}$ of rank at most one. Therefore, the image of $M\to E_{1}\otimes E_{2}$
is indecomposable at all points. Hence it defines an isomorphism
$E_{1}^{*}\otimes M\xrightarrow{\sim}E_{2}$. Moreover, $E_{1}^{*}\otimes
M\cong E_{1}\otimes(\det E_{1})^{-1}\otimes M$ since $E_{1}$ has rank two. It
follows that

$V\perp M\ \cong\ \left(\mathrm{Sym}^{2}E_{1}\otimes(\det E_{1})^{-1}\otimes
M\right)\oplus M.$

By uniqueness of direct sum decomposition, we conclude that
$V\cong\mathrm{Sym}^{2}E_{1}\otimes(\det E_{1})^{-1}\otimes M$. By [Gro57,
Proposition 3.1], composing with an automorphism of $V$ if necessary, this may
be assumed to be an isomorphism of orthogonal bundles. ∎

###  5.2. Quadric fibration and isotropic subbundles

We consider an $L$-valued rank three orthogonal bundle
$V=\mathrm{Sym}^{2}E\otimes(\det E)^{-1}\otimes M$ of determinant $M^{3}$ as
above. We now describe the locus of isotropic lines
$Q_{V}\subset{\mathbb{P}}V$.

######  Proposition 5.3.

The fibration $Q_{V}$ coincides with the relative Segre embedding

${\mathbb{P}}E\ \hookrightarrow\ {\mathbb{P}}\mathrm{Sym}^{2}E\
\xrightarrow{\sim}\ {\mathbb{P}}(\mathrm{Sym}^{2}E\otimes(\det E)^{-1}\otimes
M).$

###### Proof.

This follows easily from (5.1). For a geometric argument: By Theorem 5.2, we
may assume that the quadratic form is induced from that $V\perp M\cong
E\otimes E\otimes(\det E)^{-1}\otimes M$. Thus $Q_{V}=Q_{V\perp
M}\cap{\mathbb{P}}V$. By Proposition 4.3, this is

$\left({\mathbb{P}}E\times_{C}{\mathbb{P}}E\right)\cap\left({\mathbb{P}}(\mathrm{Sym}^{2}E\otimes(\det
E)^{-1}\otimes M)\right),$

which is precisely the image of the relative Segre embedding, as desired. ∎

######  Theorem 5.4 (Isotropic Quot schemes in rank three).

Let $V=\mathrm{Sym}^{2}E\otimes(\det E)^{-1}\otimes M$ be as above. Write
$d:=\deg(E)$ and $m:=\deg(M)$.

  1. (a)

For each integer $e$, there is an isomorphism

$\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e+d-m)}(E)\ \xrightarrow{\sim}\
\mathrm{IQ}^{\circ}_{1,e}(V)$

given by $N_{1}\mapsto N_{1}^{2}\otimes(\det E)^{-1}\otimes M$.

  2. (b)

In particular, if $N\subset V$ is any isotropic line subbundle, then
$\deg(N)\equiv d-m\mod 2$.

###### Proof.

Let $N\subset V$ be an isotropic subbundle of degree $e$. Then $N\otimes(\det
E)\otimes M^{-1}$ is a line subbundle of $\mathrm{Sym}^{2}E$. By Proposition
5.3, this belongs to the cone over the relative Segre embedding
${\mathbb{P}}E\hookrightarrow{\mathbb{P}}\mathrm{Sym}^{2}E$. Thus it is of the
form $\mathrm{Sym}^{2}N_{1}$ for some $N_{1}\subset E$ of degree
$\frac{1}{2}(e+d-m)$. In particular, $e\equiv d-m\equiv 0\mod 2$. Clearly
$N_{1}$ is uniquely determined as an element of
$\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e+d-m)}(E)$. The statement follows. ∎

######  Remark 5.5.

A Riemann–Roch computation shows that
$\mathrm{Quot}^{\circ}_{1,\frac{1}{2}(e+d-m)}(E)$ has expected dimension
$m-e-(g-1)$, which coincides with the expected dimension of
$\mathrm{IQ}^{\circ}_{1,e}(V)$ given in (2.7), in agreement with Corollary
4.7.

###  5.3. Enumeration of isotropic line subbundles

Together with Theorem 2.5, the above results allow us to count the number of
isotropic line subbundles of maximal degree of a general stable
$V\in{\mathcal{M}}{\mathcal{O}}_{C}^{i}(3,{{{\mathcal{O}}}_{C}})$, when this
is finite.

######  Theorem 5.6.

Let $V$ be a stable ${{{\mathcal{O}}}_{C}}$-valued orthogonal bundle of rank
three and trivial determinant.

  1. (a)

If $g$ is even and $V$ is general in
${\mathcal{M}}{\mathcal{O}}_{C}^{1}(3,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$,
then $V$ has $2^{g}$ maximal isotropic line subbundles of degree $1-g$.

  2. (b)

If $g$ is odd, and $V$ is general in
${\mathcal{M}}{\mathcal{O}}_{C}^{0}(3,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$,
then $V$ has $2^{g}$ maximal isotropic line subbundles of degree $1-g$.

###### Proof.

By Theorem 5.4, maximal line subbundles of $E$ are in bijection with maximal
isotropic line subbundles of $\mathrm{Sym}^{2}E\otimes(\det E)^{-1}$. If $g$
is even, then by Theorem 2.5, a general
$E\in{\mathcal{S}}{\mathcal{U}}_{C}(2,{{{\mathcal{O}}}_{C}}(x))_{\mathrm{st}}$
has $2^{g}$ line subbundles $N_{1}$ of degree $1-\frac{g}{2}$. By Theorem 5.4,
these induce $2^{g}$ isotropic line subbundles
$N_{1}^{2}(-x)\subset\mathrm{Sym}^{2}E\otimes{{{\mathcal{O}}}_{C}}(-x)$ of
degree $1-g$, and we obtain (a). The proof of (b) is similar. ∎

##  6\. Rank six

As was the case for orthogonal bundles of ranks three and four, the rank five
case turns out to be closely related to the rank six case. Therefore we now
turn to $L$-valued orthogonal bundles of rank six admitting rank three
isotropic subbundles; equivalently, in view of Lemma 3.2, having determinant
$L^{3}$.

###  6.1. Rank six orthogonal bundles with rank three isotropic subbundles

If $W$ is any bundle of rank four and determinant $L$, then there is a natural
map

(6.1) $\wedge^{2}W\otimes\wedge^{2}W\ \to\ \wedge^{4}W\ =\ L$

given by the determinant. This endows $\wedge^{2}W$ with the structure of an
$L$-valued orthogonal bundle. Furthermore, every rank three subbundle
$F\subset W$ defines a rank three subbundle $\wedge^{2}F\subset\wedge^{2}W$,
which is isotropic since $\wedge^{4}F=0$. We shall now show that up to a
twist, any rank six orthogonal bundle admitting a rank three isotropic
subbundle is of the form $\wedge^{2}W$. (In § 6.2, however, we shall see that
not every rank three isotropic subbundle need be of the form $\wedge^{2}F$,
even up to twisting.)

As in the previous sections, we shall use this to give a description of the
moduli spaces of rank six orthogonal bundles.

######  Theorem 6.1.

Let $x$ be any point of $C$.

  1. (a)

The association $W\mapsto\wedge^{2}W$ defines a finite surjective morphism

$\Theta_{6}^{0}\colon{\mathcal{S}}{\mathcal{U}}_{C}(4,{{{\mathcal{O}}}_{C}})\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}^{0}(6,{{{\mathcal{O}}}_{C}}).$

  2. (b)

The association $W_{1}\mapsto\wedge^{2}W_{1}\otimes{{{\mathcal{O}}}_{C}}(x)$
defines a finite surjective morphism

$\Theta_{6}^{1}\colon{\mathcal{S}}{\mathcal{U}}_{C}(4,{{{\mathcal{O}}}_{C}}(-2x))\to{\mathcal{M}}{\mathcal{O}}_{C}^{1}(6,{{{\mathcal{O}}}_{C}}).$

  3. (c)

The association $W_{2}\mapsto\wedge^{2}W_{2}$ defines a finite surjective
morphism

$\Phi_{6}\colon{\mathcal{S}}{\mathcal{U}}_{C}(4,{{{\mathcal{O}}}_{C}}(x))\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}(6,{{{\mathcal{O}}}_{C}}(x)).$

###### Proof.

This is similar to the rank three case. As ${\mathbb{C}}$ has characteristic
zero, $\wedge^{2}W$ is semistable whenever $W$ is. Using for example [Har77,
Exercise II.5.16], one checks that if $W$ and $W^{\prime}$ are S-equivalent
semistable bundles of rank four, then $\wedge^{2}W$ and $\wedge^{2}W^{\prime}$
are S-equivalent. By a similar argument to that in Theorem 5.1, we obtain the
existence, finiteness and surjectivity of the morphisms $\Theta^{i}_{6}$ and
$\Phi_{6}$. The target component of each $\Theta_{6}^{i}$ is determined by
Theorem 2.3 and the parity of

$\deg(\wedge^{2}F)=2\cdot\deg(F)\ \text{ and }\
\deg(\wedge^{2}F)\otimes{{{\mathcal{O}}}_{C}}(x)=2\cdot\deg(F)+3$

for rank three subbundles $F\subset W$. ∎

As in the previous cases, we describe a construction required for the next
proposition which shows how to get the “inverses” of the morphisms
$\Theta_{6}^{i}$ and $\Phi_{6}$. Let $V$ be a rank six $L$-valued orthogonal
bundle admitting a rank three isotropic subbundle $E$. Then by Criterion 2.1,
there is an exact sequence $0\to E\to V\to E^{*}\otimes L\to 0$, defining an
extension class $\delta\in H^{1}(C,\wedge^{2}E\otimes L^{-1})$. Then $\delta$
also defines an extension $0\to\wedge^{2}E\to W\to L\to 0$ of rank four.

######  Proposition 6.2.

Let $V$ be an $L$-valued orthogonal bundle of rank six. Let $E$, $\delta$ and
$W$ be as above. Then $V$ is isomorphic to $(\wedge^{2}W)\otimes(\det
E)^{-1}$.

###### Proof.

To ease notation, write ${\widehat{E}}:=\wedge^{2}E$, a subbundle of $W$. We
consider the antisymmetrization map $a\colon W\otimes W\to\wedge^{2}W$. By
linear algebra, using the fact that ${\widehat{E}}$ has corank one, the
restriction of $a$ to $W\otimes{\widehat{E}}$ is surjective with kernel
$\mathrm{Sym}^{2}{\widehat{E}}$. Therefore, we have a diagram

$\textstyle{\mathrm{Sym}^{2}{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{=}$$\textstyle{\mathrm{Sym}^{2}{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{{\widehat{E}}\otimes{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{a}$$\textstyle{W\otimes{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{a}$$\textstyle{L\otimes{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0}$$\textstyle{0\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\wedge^{2}{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{\wedge^{2}W\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{L\otimes{\widehat{E}}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{0.}$

In particular, $\wedge^{2}W$ is an extension of class

(6.2) $a_{*}(\delta\otimes\mathrm{Id}_{\widehat{E}})\ \in\
H^{1}(C,\mathrm{Hom}(L\otimes{\widehat{E}},\wedge^{2}{\widehat{E}})).$

Now write $M:=\wedge^{3}E=\det E$. The association

$u\wedge v\ \mapsto\ \left(w\ \mapsto u\wedge v\wedge w\right)$

gives a canonical identification

(6.3) ${\widehat{E}}\ =\ \wedge^{2}E\ =\ \mathrm{Hom}(E,\det E)\
\xrightarrow{\sim}\ E^{*}\otimes M.$

Similarly, we have canonical identifications

(6.4) $\wedge^{2}{\widehat{E}}\ \xrightarrow{\sim}\ \wedge^{2}(E^{*}\otimes
M)\ \cong\ \wedge^{2}E^{*}\otimes M^{2}\ \xrightarrow{\sim}\ E\otimes M.$

Thus $\wedge^{2}W$ is an extension $0\to E\otimes M\to\wedge^{2}W\to
E^{*}\otimes M\otimes L\to 0$, and

$\mathrm{Hom}(L\otimes{\widehat{E}},\wedge^{2}{\widehat{E}})\ =\ L^{-1}\otimes
E\otimes M^{-1}\otimes E\otimes M\ \cong\ L^{-1}\otimes E\otimes
E\otimes\mathrm{End}\;M.$

Therefore, to prove the theorem, in view of (6.2), (6.3) and (6.4) it will
suffice to prove the following lemma. ∎

######  Lemma 6.3.

The element $a_{*}(p\otimes\mathrm{Id}_{\widehat{E}})$ corresponds to
$-p\otimes\mathrm{Id}_{M}$ for each $p\in L^{-1}\otimes\wedge^{2}E$.

###### Proof.

In what follows, we shall assume that $L={{{\mathcal{O}}}_{C}}$, as the
general case is only notationally more cumbersome. Let $p$ be any element of
$L^{-1}\otimes\wedge^{2}E|_{x}=\wedge^{2}E|_{x}$ for some $x\in C$. As $E$ has
rank three, we can write $p=u\wedge v$ for some $u,v\in E|_{x}$. If $p$ is
nonzero then we can choose $w\in E|_{x}$ such that $u,v,w$ is a basis of
$E|_{x}$. Let $u^{*},v^{*},w^{*}$ be the dual basis of $E^{*}|_{x}$. Set
$m:=u\wedge v\wedge w\in M|_{x}$. Then $u^{*}\wedge v^{*}\wedge w^{*}=m^{*}$
where $m^{*}\in M^{*}|_{x}$ is dual to $m$ (that is, $\langle
m^{*},m\rangle=1$). Then via (6.3), we have identifications

(6.5) $u\wedge v\ \leftrightarrow\ w^{*}\otimes m,\quad u\wedge w\
\leftrightarrow\ (-v)^{*}\otimes m,\quad\hbox{and}\quad v\wedge w\
\leftrightarrow\ u^{*}\otimes m.$

Via these identifications, and using the contraction $m^{*}\otimes m\mapsto
1$, the map $\mathrm{Id}_{\wedge^{2}E}|_{x}$ can be identified with

$(u\wedge v)\otimes m^{*}\otimes w+(u\wedge w)\otimes
m^{*}\otimes(-v)+(v\wedge w)\otimes m^{*}\otimes u.$

Thus $p\otimes\mathrm{Id}_{\wedge^{2}E}$ is expressed as

$(u\wedge v)\otimes(u\wedge v)\otimes m^{*}\otimes w+(u\wedge
v)\otimes(u\wedge w)\otimes m^{*}\otimes(-v)+(u\wedge v)\otimes(v\wedge
w)\otimes m^{*}\otimes u.$

Substituting from (6.5) (thereby using the first identification in (6.4)), we
obtain

$a_{*}(p\otimes\mathrm{Id}_{\wedge^{2}E})=(w^{*}\otimes
m)\wedge((-v^{*})\otimes m)\otimes m^{*}\otimes(-v)+(w^{*}\otimes
m)\wedge(u^{*}\otimes m)\otimes m^{*}\otimes u.$

Applying the contraction $m\otimes m^{*}\mapsto 1$, this becomes

(6.6) $(w^{*}\wedge v^{*})\otimes v\otimes m+(w^{*}\wedge u^{*})\otimes
u\otimes m.$

Now since $u^{*}\wedge v^{*}\wedge w^{*}=m^{*}$, in particular $w^{*}\wedge
v^{*}$ is the dual basis element to $(-u)^{*}\otimes m$. Thus via the
identification
$\wedge^{2}E^{*}\xrightarrow{\sim}\mathrm{Hom}(E^{*},M^{-1})\cong E\otimes
M^{-1}$, we see that $w^{*}\wedge v^{*}$ corresponds to $(-u)\otimes m^{*}$.
Similarly, $w^{*}\wedge u^{*}$ corresponds to $v\otimes m^{*}$. Thus (6.6) is
identified with

$(-u)\otimes m^{*}\otimes v\otimes m+v\otimes m^{*}\otimes u\otimes m,$

corresponding to $-(u\wedge
v)\otimes\mathrm{Id}_{M}=-p\otimes\mathrm{Id}_{M}$, as desired. ∎

We can now give a structure theorem for orthogonal bundles of rank six.

######  Theorem 6.4.

Let $V$ be an $L$-valued orthogonal bundle of rank six.

  1. (a)

The bundle $V$ is isomorphic to $\wedge^{2}W$ for some $W$ of rank four and
determinant $L$ if and only if $V$ admits a rank three isotropic subbundle of
even degree.

  2. (b)

Let $x$ be any point of $C$. Then $V$ is isomorphic to
$\wedge^{2}W_{1}\otimes{{{\mathcal{O}}}_{C}}(x)$ for some $W_{1}$ of rank four
and determinant $L(-2x)$ if and only if $V$ admits a rank three isotropic
subbundle of odd degree.

Note that (a) and (b) apply simultaneously when $\deg(L)$ is odd.

###### Proof.

As the proofs of (a) and (b) are almost identical, we prove (b) only. Let $E$
be a rank three subbundle of $V$ which is isotropic and of odd degree. By
Proposition 6.2, we have $V\cong\wedge^{2}W\otimes\det E^{*}$ where
$0\to\wedge^{2}E\to W\to L\to 0$ is constructed from $V$ and $E$ as
previously. By hypothesis, we can find $N$ such that $\det E\cong N^{2}(-x)$.
Set $W_{1}:=W\otimes N^{-1}$. Then

$\det W_{1}\ \cong\ (\det E)^{2}\otimes L\otimes N^{-4}\ \cong\
N^{4}(-2x)\otimes L\otimes N^{-4}\ \cong\ L(-2x),$

and $V\cong\wedge^{2}W\otimes
N^{-2}\otimes{{{\mathcal{O}}}_{C}}(x)=(\wedge^{2}W_{1})\otimes{{{\mathcal{O}}}_{C}}(x)$.

Conversely, if $V=(\wedge^{2}W_{1})\otimes{{{\mathcal{O}}}_{C}}(x)$ then any
rank three subbundle $F\subset W_{1}$ yields a subbundle
$\wedge^{2}F\otimes{{{\mathcal{O}}}_{C}}(x)\subset V$ which is isotropic since
$\wedge^{4}F=0$, and has odd degree $2\cdot\deg F+3$. ∎

###  6.2. Enveloping bundle structure and isotropic subbundles

Let $W$ be a bundle of rank four and determinant $L$, and let $V:=\wedge^{2}W$
be the associated $L$-valued orthogonal bundle of rank six. By [GH78, pp.
209–211], an element $\omega\in\wedge^{2}W$ belongs to the cone over the
Plücker image of $\mathrm{Gr}(2,W)$ in ${\mathbb{P}}(\wedge^{2}W)$ if and only
if $\omega$ is decomposable; equivalently $\omega\wedge\omega=0$ in
$\wedge^{4}W$. In view of (6.1), this is equivalent to $\omega$ being
isotropic with respect to the orthogonal structure on $\wedge^{2}W$. It
follows that:

######  Proposition 6.5.

The relative Plücker map embeds $\mathrm{Gr}(2,W)$ in
${\mathbb{P}}(\wedge^{2}W)$ as the projectivization $Q_{\wedge^{2}W}$ of the
locus of vectors isotropic with respect to the orthogonal structure on
$\wedge^{2}W$. In particular, ${\mathbb{P}}(\wedge^{2}W)$ is an enveloping
bundle for $Q_{\wedge^{2}W}=\mathrm{Gr}(2,W)\to C$.

We shall also require the following lemma on isotropic subbundles of
orthogonal bundles.

######  Lemma 6.6.

Let $V$ be an $L$-valued orthogonal bundle (of arbitrary rank) and $E\subset
V$ any subbundle. Then $E$ is isotropic if and only if ${\mathbb{P}}E\subset
Q_{V}$.

###### Proof.

One direction is clear from the definition of $Q_{V}$. Conversely, suppose
${\mathbb{P}}E\subset Q_{V}$. Let $v,w$ be elements of a fiber $E|_{x}$. By
symmetry of $\sigma$, we have

$2\cdot\sigma(v\otimes w)\ =\
\sigma\left((v+w)\otimes(v+w)\right)-\sigma(v\otimes v)-\sigma(w\otimes w).$

By hypothesis, the right hand side is zero, so $\sigma(v\otimes w)=0$. ∎

######  Remark 6.7.

Note that the above proof depends on the symmetry of the bilinear form, and
may fail in characteristic $2$.

We shall use these observations to classify isotropic subbundles of $V$,
beginning with rank three. By Proposition 6.5 and Lemma 6.6, a rank three
subbundle $E\subset V$ is isotropic if and only if the
${\mathbb{P}}^{2}$-subbundle ${\mathbb{P}}E\subset{\mathbb{P}}V$ is contained
in $\mathrm{Gr}(2,W)$. For such an $E$, by [GH78, p. 757], one of the
following two situations must arise:

  1. (i)

For each $p\in C$, there exists a three-dimensional subspace $F_{p}\subset
V|_{p}$ such that

${\mathbb{P}}E|_{p}\ =\ \\{\Lambda\in\mathrm{Gr}(2,W|_{p}):\Lambda\subset
F_{p}\\}.$

  2. (ii)

For each $p\in C$, there exists a line $N_{p}\subset W|_{p}$ such that

${\mathbb{P}}E|_{p}\ =\
\\{\Lambda\in\mathrm{Gr}(2,W|_{p}):N_{p}\subset\Lambda\\}.$

If situation (i) arises, then, using local triviality, there exists a rank
three subbundle $F\subset W$ such that ${\mathbb{P}}E=\mathrm{Gr}(2,F)$. By
definition of the Plücker embedding
$\mathrm{Gr}(2,F)\to{\mathbb{P}}(\wedge^{2}F)\subset{\mathbb{P}}(\wedge^{2}W)$
it follows that ${\mathbb{P}}E$ is the projectivization of the set of
bivectors

$\\{v\wedge w:v,w\in F\\}.$

As $F$ has rank three, every element of $\wedge^{2}F$ is of this form. Thus
$E=\wedge^{2}F$. In particular, $\deg E=2\cdot\deg F$ is even.

On the other hand, suppose situation (ii) arises. Then there is a line
subbundle $N\subset W$ such that ${\mathbb{P}}E$ is the projectivization of
$\\{v\wedge w:v\in N,w\in W\\}$. We define a map $\tau_{N}\colon N\otimes
W\to\wedge^{2}W$ by

(6.7) $\tau_{N}(v\otimes w)\ =\ v\wedge w.$

Clearly $\tau_{N}$ has kernel $N\otimes N$ and image exactly $E$. Thus $E\cong
N\otimes\frac{W}{N}$, and

$\deg E\ =\ 3\cdot\deg N+\deg W-\deg N\ =\ 2\cdot\deg N+\deg L.$

In particular, $\deg E\equiv\deg L\mod 2$. Now we can summarize the above
discussion as follows.

######  Theorem 6.8.

Let $V=\wedge^{2}W$ be a rank six orthogonal bundle for a rank four bundle $W$
of determinant $L$. Set $\ell:=\deg L$ and let $d$ be any integer.

  1. (a)

Suppose $\ell$ is even. Then there is an isomorphism

$\mathrm{Quot}^{\circ}_{3,d}(W)\ \sqcup\
\mathrm{Quot}^{\circ}_{1,d-\frac{\ell}{2}}(W)\ \xrightarrow{\sim}\
\mathrm{IQ}^{\circ}_{3,2d}(V).$

In particular, $\mathrm{IQ}^{\circ}_{3,2d}(V)$ is disconnected for $d\ll 0$.

  2. (b)

Suppose $\ell$ is odd. Then there are isomorphisms

$\mathrm{Quot}^{\circ}_{3,d}(W)\ \xrightarrow{\sim}\
\mathrm{IQ}^{\circ}_{3,2d}(V)\quad\hbox{and}\quad\mathrm{Quot}^{\circ}_{1,d-\frac{\ell+1}{2}}(W)\
\xrightarrow{\sim}\ \mathrm{IQ}^{\circ}_{3,2d-1}(V).$

In both cases, the isomorphisms are given respectively by

$E\ \mapsto\ \wedge^{2}E\quad\hbox{and}\quad N\ \mapsto\ \tau_{N}(N\otimes W)\
\cong\ N\otimes\frac{W}{N}.$

######  Remark 6.9.

A Riemann–Roch computation shows that $\mathrm{Quot}_{3,d}(W)$ and
$\mathrm{Quot}_{1,d-\frac{\ell+\varepsilon}{2}}(W)$ have the same expected
dimensions respectively as those given in (2.7) for
$\mathrm{IQ}^{\circ}_{3,2d}(\wedge^{2}W)$ and
$\mathrm{IQ}^{\circ}_{3,2d-\varepsilon}(\wedge^{2}W)$, in agreement with
Theorem 6.8.

Lastly, we treat the case where all rank three isotropic subbundles have odd
degree, so $V$ is of the form $\wedge^{2}W\otimes{{{\mathcal{O}}}_{C}}(x)$.

######  Corollary 6.10.

Suppose $\ell$ is even. Let $W$ be a rank four bundle of determinant $L$, and
let $V$ be the orthogonal bundle $\wedge^{2}W\otimes{\mathcal{O}}_{C}(x)$.
Then for any $d$, there are isomorphisms

$\mathrm{Quot}^{\circ}_{3,d-1}(W)\ \sqcup\
\mathrm{Quot}^{\circ}_{1,d-1-\frac{\ell}{2}}(W)\ \xrightarrow{\sim}\
\mathrm{IQ}^{\circ}_{3,2d+1}(V),$

given by

$\begin{cases}E\ \mapsto\ \wedge^{2}E\otimes{{{\mathcal{O}}}_{C}}(x)\hbox{ if
}E\in\mathrm{Quot}^{\circ}_{3,d-1}(W);\\\ M\ \mapsto\ \tau_{N}(N\otimes
W)\otimes{{{\mathcal{O}}}_{C}}(x)\ \cong\
N\otimes\frac{W}{N}\otimes{{{\mathcal{O}}}_{C}}(x)\hbox{ if
}E\in\mathrm{Quot}^{\circ}_{1,d-1-\frac{\ell}{2}}(W).\end{cases}$

In particular, $\mathrm{IQ}^{\circ}_{3,2d+1}(V)$ is disconnected for $d\ll 0$.

###### Proof.

For any rank six $L$-valued orthogonal $V$, tensor product by
${{{\mathcal{O}}}_{C}}(x)$ defines a canonical isomorphism
$\mathrm{IQ}^{\circ}_{3,2d-2}(V)\xrightarrow{\sim}\mathrm{IQ}^{\circ}_{3,2d+1}(V\otimes{{{\mathcal{O}}}_{C}}(x))$.
Thus the statement follows from Theorem 6.8 (a). ∎

######  Remark 6.11.

The fact that the isotropic Quot schemes of $W$ may have two connected
components reflects that fact that the isotropic Grassmann bundle
$\mathrm{OG}(3,\wedge^{2}W)$ has two components. This is studied in more
generality in [CCH21b]. Further detail for rank six is given in Remark 7.6.

Let us now consider isotropic subbundles of rank two or one. For brevity, we
shall suppose throughout that $V$ is of the form $\wedge^{2}W$; the case
$V=\wedge^{2}W\otimes{{{\mathcal{O}}}_{C}}(x)$ is left to the reader.

Let $F\subset V$ be a rank two isotropic subbundle. This corresponds to a
${\mathbb{P}}^{1}$-subbundle of the Grassmannian bundle $\mathrm{Gr}(2,W)\to
C$. By [GH78, pp. 756–757], any ${\mathbb{P}}^{1}$ lying inside
$\mathrm{Gr}(2,4)$ is of the form

$\\{\Lambda\in\mathrm{Gr}(2,4)\>:\>p\subset{\mathbb{P}}\Lambda\subset h\\}$

for some fixed point $p$ and hyperplane $h$ in ${\mathbb{P}}^{3}$. Using local
triviality as above, there exists a flag of subbundles $N\subset H\subset W$
with $\mathrm{rk}\,(N)=1$ and $\mathrm{rk}\,(H)=3$ such that for each $x\in C$
we have

${\mathbb{P}}F|_{x}\>=\>\\{\Lambda\in\mathrm{Gr}(2,W|_{x})\>:\>N|_{x}\subset\Lambda\subset
H|_{x}\\}.$

It follows that

$F\>=\>\\{v\wedge w\>:\>v\in N,\ w\in H\\}\ =\ \tau_{N}(N\otimes H),$

where $\tau_{N}$ is the map defined in (6.7). Hence $F\cong(N\otimes
H)/(N\otimes N)$ and $\deg(F)=\deg(H)+\deg(N)$. As a consequence, we obtain:

######  Theorem 6.12.

For any rank four bundle $W$, there is an identification

$\mathrm{IQ}^{\circ}_{2,d}(\wedge^{2}W)\cong\bigcup_{d_{1}+d_{2}=d}{\mathcal{Q}}uot^{\circ}_{1,d_{1}}({\mathcal{H}}),$

where
$\pi\colon{\mathcal{Q}}uot_{1,d_{1}}^{\circ}({\mathcal{H}})\to\mathrm{Quot}^{\circ}_{3,d_{2}}(W)$
is the relative Quot scheme with
$\pi^{-1}(H)=\mathrm{Quot}_{1,d_{1}}^{\circ}(H)$ at
$H\in\mathrm{Quot}^{\circ}_{3,d_{2}}(W)$, which parameterizes the flags
$N\subset H\subset W$ given as above.

######  Remark 6.13.

Notice that the rank two isotropic subbundle $\tau_{N}(N\otimes H)$ is exactly
the intersection of the rank three isotropic subbundles $\tau_{N}(N\otimes W)$
and $\wedge^{2}H$, which belong to opposite components of $\mathrm{OG}(V)$.
Compare with [GH78, Proposition, p. 735]. We discuss another aspect of this in
Remark 7.6.

Lastly: By Theorem 6.5, an isotropic line subbundle of $\wedge^{2}W$ is simply
a section of $\mathrm{Gr}(2,W)\to C$. But this is equivalent to a choice of
rank two subbundle $E\subset W$, and then $N=\wedge^{2}E$. Thus we have

######  Theorem 6.14.

Let $W$ be any rank four bundle and $\wedge^{2}W$ the associated orthogonal
bundle. Then
$\mathrm{IQ}^{\circ}_{1,d}(\wedge^{2}W)\cong\mathrm{Quot}^{\circ}_{2,d}(W)$.

###  6.3. Enumeration of maximal isotropic subbundles

We now apply the previous results to the problem of enumerating isotropic
subbundles of maximal degree of a general rank six $L$-valued orthogonal
bundle $V$, when there are finitely many of these. For ranks one and two, for
brevity we restrict to the case $V\cong\wedge^{2}W$ with
$\det(W)\cong{{{\mathcal{O}}}_{C}}$.

######  Theorem 6.15.

Let $V$ be a rank six ${{{\mathcal{O}}}_{C}}$-valued orthogonal bundle which
is general in its component of moduli.

  1. (a)

Suppose $g\equiv 1\mod 4$ and $w_{2}(V)=0$. Then $V$ has $2\cdot 4^{g}$ rank
three isotropic subbundles of maximal degree $-\frac{3}{2}(g-1)$.

  2. (b)

Suppose $g\equiv 3\mod 4$ and $w_{2}(V)=1$. Then $V$ has $2\cdot 4^{g}$ rank
three isotropic subbundles of maximal degree $-\frac{3}{2}(g-1)$.

  3. (c)

Suppose that $g\equiv 1\mod 12$ and $w_{2}(V)=0$. Then $V$ has $12^{g}$ rank
two isotropic subbundles of maximal degree $-\frac{5}{3}(g-1)$.

  4. (d)

Suppose $w_{2}(V)=0$. Then $V$ has

$2^{3g-1}-(-1)^{g}\cdot 2^{2g-1}$

isotropic line subbundles of maximal degree $1-g$.

###### Proof.

(a) By Theorem 6.1 (a), we have $V\cong\wedge^{2}W$ for some
$W\in{\mathcal{S}}{\mathcal{U}}_{C}(4,{{{\mathcal{O}}}_{C}})$, which may
assumed to be general in moduli. Note that $W^{*}$ may also be assumed to be
general in moduli, and that
$\mathrm{Quot}_{3,d}(W)\cong\mathrm{Quot}_{1,d}(W^{*})$. Thus, as
$\deg(W)\equiv 1-g\mod 4$, by Theorem 2.5 the bundles $W$ and $W^{*}$ both
have $4^{g}$ maximal line subbundles of degree $\frac{3}{4}(g-1)$. By Theorem
6.8 (a), the bundle $V$ contains $2\cdot 4^{g}$ rank three isotropic
subbundles of maximal degree $-\frac{3}{2}(g-1)$.

(b) By Theorem 6.1 (b), we have
$V\cong\wedge^{2}W_{1}\otimes{{{\mathcal{O}}}_{C}}(x)$ for some
$W_{1}\in{\mathcal{S}}{\mathcal{U}}_{C}(4,{{{\mathcal{O}}}_{C}}(-2x))$, which
may be assumed to be general in moduli. As $-2\equiv 1-g\mod 4$, by Theorem
2.5 both the bundles $W_{1}$ and $W_{1}^{*}$ have $4^{g}$ maximal line
subbundles of degree $d$ and $d+1$ respectively, for $d=\frac{1}{4}(1-3g)$. By
Corollary 6.10, we have

$\mathrm{IQ}^{\circ}_{3,2d+1}(V)\ \cong\
\mathrm{Quot}^{\circ}_{3,d-1}(W_{1})\sqcup\mathrm{Quot}^{\circ}_{1,d}(W_{1}).$

Since
$\mathrm{Quot}^{\circ}_{3,d-1}(W_{1})\cong\mathrm{Quot}^{\circ}_{1,d+1}(W_{1}^{*})$,
we get the desired number.

(c) Here $V=\wedge^{2}W$ is as in part (a). By hypothesis, in particular
$g\equiv 1\mod 4$, so a general
$W\in{\mathcal{S}}{\mathcal{U}}_{C}(4,{\mathcal{O}}_{C})$ has $4^{g}$ maximal
rank three subbundles of degree $-\frac{3}{4}(g-1)$. By [LN03, Lemma 2.1],
these maximal subbundles are general in moduli. As by hypothesis we also have
$g\equiv 1\mod 3$, each maximal rank three $H\subset W$ has $3^{g}$ maximal
line subbundles, each of degree $-\frac{11}{12}(g-1)$. Thus there are
$4^{g}\cdot 3^{g}=12^{g}$ flags $N\subset H\subset W$ where both $N\subset H$
and $H\subset W$ are maximal subbundles. By Theorem 6.12, then, $V$ has
$12^{g}$ isotropic subbundles of rank two and maximal degree

$-\frac{3}{4}(g-1)-\frac{11}{12}(g-1)\ =\ -\frac{5}{3}(g-1).$

(d) Using the formulas in [Hol04, § 4], one finds that a general bundle of
rank four and degree zero has a finite number of rank two subbundles of
maximal degree $1-g$, the number being given as follows:

(6.8)
$2^{4g-3}\cdot\sum_{z\in\\{-1,\pm\sqrt{-1}\\}}\left(z(1-z)^{2}\right)^{1-g}\
=\ 2^{3g-1}+(-1)^{g-1}\cdot 2^{2g-1}.$

The statement now follows from Theorem 6.14. ∎

######  Remark 6.16.

By the Hirschowitz bound [Hir96], every rank six vector bundle of degree zero
has a rank two subbundle of degree at least $-\frac{4}{3}(g-1)$. This means
that a general $V\in{\mathcal{M}}{\mathcal{O}}_{C}^{0}(6,{\mathcal{O}}_{C})$
does have rank two subbundles of degree greater than $-\frac{5}{3}(g-1)$, all
of which are nonisotropic.

######  Theorem 6.17.

Let $V$ be a rank six ${{{\mathcal{O}}}_{C}}(x)$-valued orthogonal bundle of
determinant ${{{\mathcal{O}}}_{C}}(3x)$ which is general in moduli. Suppose
$g$ is even. Then $V$ has $4^{g}$ isotropic rank three subbundles of maximal
degree $3-\frac{3}{2}g$.

###### Proof.

By Theorem 6.1 (c), we have $V\cong\wedge^{2}W_{2}$ for some
$W_{2}\in{\mathcal{S}}{\mathcal{U}}_{C}(4,{{{\mathcal{O}}}_{C}}(x))$, which
may be assumed to be general in moduli. There are two cases.

If $g\equiv 0\mod 4$, put $d:=2-\frac{3}{4}g$, so that $2d-1=3-\frac{3}{2}g$.
Then by Theorem 6.8 (b), we have
$\mathrm{IQ}^{\circ}_{3,2d-1}(V)\cong\mathrm{Quot}^{\circ}_{1,d-1}(W_{2})$.

If $g\equiv 2\mod 4$, put $d:=\frac{3}{2}-\frac{3}{4}g$, so that
$2d=3-\frac{3}{2}g$. Then by Theorem 6.8 (b), we have
$\mathrm{IQ}^{\circ}_{3,2d}(V)\cong\mathrm{Quot}^{\circ}_{3,d}(W_{2})\cong\mathrm{Quot}^{\circ}_{1,d-1}(W_{2}^{*})$.

Applying Theorem 2.5 to $\mathrm{Quot}^{\circ}_{1,d-1}(W_{2})$ and
$\mathrm{Quot}^{\circ}_{1,d-1}(W_{2}^{*})$, we get the desired number. ∎

##  7\. Rank five

Let $L$ be a line bundle, and $V$ an $L$-valued orthogonal bundle of rank
five. By (2.1), necessarily $L$ has even degree $2m$, and $\det W\cong M^{5}$
for some $M$ satisfying $M^{2}\cong L$. We shall use the results in § 6 to
analyze the structure of $V$, in particular the relation with rank four
symplectic bundles.

###  7.1. Structure of rank five orthogonal bundles

Let $M$ be a line bundle and $W$ a rank four bundle with a symplectic form
$\alpha\colon\wedge^{2}W\to M$. Then $\deg(W)=2\cdot\deg(M)$, and
$S_{W}:=\mathrm{Ker}\left(\alpha\colon\wedge^{2}W\to M\right)$ is a rank five
subbundle of determinant $M^{5}$. Note that $S_{W}$ depends on the choice of
$\alpha$, but if for example $W$ is stable then $\alpha$ is unique up to
nonzero scalar multiple.

######  Proposition 7.1.

The bundle $S_{W}$ is $M^{2}$-valued orthogonal, and there is an orthogonal
direct sum decomposition $\wedge^{2}W=S_{W}\perp M$.

###### Proof.

As $W$ is $M$-valued symplectic, $\det W\cong M^{2}$. Thus $S_{W}$ inherits
the symmetric bilinear form $\sigma\colon(\wedge^{2}W)^{\otimes 2}\to M^{2}$
given by the determinant, as discussed in the previous section. Let us show
that $\sigma|_{\left(S_{W}\right)^{\otimes 2}}$ is nondegenerate. Suppose
$\eta\in S_{W}|_{p}$ is such that $\sigma(\eta\wedge-)$ is the zero functional
on $S_{W}|_{p}$. Then in particular $\eta\wedge\eta=0$, so $\eta$ is
decomposable. Furthermore, as

$\eta\ \in\ \mathrm{Ker}\left(\alpha|_{p}\colon\wedge^{2}W|_{p}\ \to\
M|_{p}\right),$

if $\eta$ is nonzero then $\eta=v\wedge w$ where $v$ and $w$ span a subspace
$\Lambda\subset W|_{p}$ isotropic with respect to $\alpha$. But then for any
$v^{\prime},w^{\prime}\in W|_{p}$ spanning an $\alpha$-isotropic subspace
complementary to $\Lambda$, we have $v^{\prime}\wedge w^{\prime}\in
S_{W}|_{p}$ and $v\wedge w\wedge v^{\prime}\wedge w^{\prime}\neq 0$ in
$M^{2}|_{p}$. Thus $\eta$ must be zero, and
$\sigma|_{\left(S_{W}\right)^{\otimes 2}}$ is nondegenerate. Hence $S_{W}$ is
$M^{2}$-valued orthogonal.

For the rest: By the nondegeneracy, $S_{W}\cap(S_{W})^{\perp}=0$ in
$\wedge^{2}W$. Comparing determinants, we see that $(S_{W})^{\perp}\cong M$.
Therefore, $\wedge^{2}W=S_{W}\perp M$ as desired. ∎

Again, we get morphisms of moduli spaces from this observation. For any line
bundle $M$, we denote by ${\mathcal{M}}{\mathcal{S}}_{C}(4,M)$ the moduli
space of semistable $M$-valued symplectic bundles of rank $4$ over $C$. This
is a closed irreducible subvariety of
${\mathcal{S}}{\mathcal{U}}_{C}(2n,M^{n})$ of dimension $10(g-1)$.

######  Theorem 7.2.

Let $x$ be any point of $C$.

  1. (a)

The association $W\mapsto S_{W}$ defines a finite surjective morphism

$\Theta_{5}^{0}\colon{\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}})\
\to\ {\mathcal{M}}{\mathcal{O}}_{C}^{0}(5,{{{\mathcal{O}}}_{C}}).$

  2. (b)

The association $W_{1}\mapsto S_{W_{1}}\otimes{{{\mathcal{O}}}_{C}}(x)$
defines a finite surjective morphism

$\Theta_{5}^{1}\colon{\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}}(-x))\to\
{\mathcal{M}}{\mathcal{O}}_{C}^{1}(5,{{{\mathcal{O}}}_{C}}).$

###### Proof.

Let $\widetilde{M}$ be an étale cover of
${\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}$
admitting a Poincaré family ${\mathcal{W}}\to\widetilde{M}\times C$ together
with a symplectic form
$\tilde{\alpha}\colon\wedge^{2}{\mathcal{W}}\to{\mathcal{O}}_{\widetilde{M}\times
C}$. Then by Proposition 7.1 we see that
$S_{{\mathcal{W}}}:=\mathrm{Ker}(\tilde{\alpha})$ is a family of rank five
orthogonal bundles. Moreover, since ${\mathbb{C}}$ has characteristic zero, if
a bundle $W$ is stable then $\wedge^{2}W$ is polystable. Hence the family
$S_{{\mathcal{W}}}$ defines a morphism
$\widetilde{M}\to{\mathcal{M}}{\mathcal{O}}_{C}^{0}(5,{{{\mathcal{O}}}_{C}})$
which factorizes via a morphism
$\Theta_{5}^{0}\colon{\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}})_{\mathrm{st}}\to{\mathcal{M}}{\mathcal{O}}_{C}^{0}(5,{{{\mathcal{O}}}_{C}})$.

Next, note that if $W$ and $W^{\prime}$ are S-equivalent semistable
${{{\mathcal{O}}}_{C}}$-valued symplectic bundles, then

$\mathrm{gr}(S_{W})\oplus{{{\mathcal{O}}}_{C}}\ =\ \mathrm{gr}(\wedge^{2}W)\
\cong\ \mathrm{gr}(\wedge^{2}W^{\prime})\ =\
\mathrm{gr}(S_{W^{\prime}})\oplus{{{\mathcal{O}}}_{C}}.$

Therefore, if $W\sim W^{\prime}$ then $S_{W}\sim S_{W^{\prime}}$. Now by
[BLS89], the moduli space
${\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}})$ has Picard number
one. Thus part (a) can be proven similarly as before. The proof of (b) is
similar.

As before, the target component is determined by the parity of degree of rank
two isotropic subbundles. The details are described in Theorem 7.3. ∎

As in previous cases, we now describe how to construct an “inverse” for the
operation $W\mapsto S_{W}$, which is valid for an arbitrary $L$-valued
orthogonal bundle of rank five.

######  Theorem 7.3.

Let $L$ be a line bundle of degree $2m$, and $V$ a rank five $L$-valued
orthogonal bundle of determinant $M^{5}$, where $M^{2}\cong L$.

  1. (a)

The bundle $V$ admits an isotropic subbundle $E$ of rank two.

  2. (b)

If $\deg(E)\equiv m\mod 2$, then there is an $M$-valued symplectic bundle
$(W,\alpha)$ of rank four (and determinant $L$) such that $V\cong S_{W}$.

  3. (c)

If $\deg(E)\not\equiv m\mod 2$, then there is an $M(-x)$-valued symplectic
bundle $(W_{1},\alpha_{1})$ of rank four (and determinant $L(-2x)$) such that
$V\cong S_{W_{1}}\otimes{{{\mathcal{O}}}_{C}}(x)$.

###### Proof.

As $M^{2}\cong L$, in fact $M$ itself is an $L$-valued orthogonal bundle. The
orthogonal direct sum $V\perp M$ is a rank six orthogonal bundle of
determinant $L^{3}$. Then $V\perp M$ admits a rank three isotropic subbundle
$\overline{E}$ by Proposition 3.2. It is easy to see that

$E\ :=\ \overline{E}\cap V\ =\ \mathrm{Ker}\left(\overline{E}\ \to\ (V\perp
M)\ \to\ M\right)$

is an isotropic subbundle of rank two in $V$. Thus we obtain (a).

Note moreover that $\deg(E)+m=\deg(\overline{E})$. Suppose $\deg(E)\equiv
m\mod 2$. Then $\deg(\overline{E})$ is even. By Theorem 6.4 (a), there exists
a bundle $W$ of rank four and determinant $L$ such that $V\perp
M\cong\wedge^{2}W$.

Now the subbundle $M\subset\wedge^{2}W$ is nowhere isotropic with respect to
the orthogonal structure, being an orthogonal direct summand. By Proposition
6.5, vectors in $\wedge^{2}W$ isotropic with respect to the orthogonal
structure correspond precisely to the decomposable tensors. Therefore, the
image of $M\to\wedge^{2}W$ is indecomposable at all points. Tensoring by
$W^{*}$ and contracting, we obtain an antisymmetric isomorphism

$M\otimes W^{*}\ \to\ \left(\wedge^{2}W\right)\otimes W^{*}\ \to\ W,$

equivalently, an $M$-valued symplectic structure on $W$. By Proposition 7.1,
we have

$V\perp M\ =\ \wedge^{2}W\ =\ S_{W}\perp M.$

By uniqueness of direct sum decompositions, we have $V\cong S_{W}$ as vector
bundles; and by [Gro57, Proposition 3.1] they are isomorphic as orthogonal
bundles. This proves (b).

If $\deg(E)\not\equiv m\mod 2$, then $\deg(\overline{E})$ is odd. Then
$V(-x)\perp M(-x)$ is $L(-2x)$-valued orthogonal and admits the isotropic
subbundle $\overline{E}(-x)$ which is of rank three and even degree. By part
(a), there exists a rank four $M(-x)$-valued symplectic bundle $W_{1}$ such
that $V(-x)\cong S_{W_{1}}$, as desired. ∎

###  7.2. Enveloping bundle structure

Let $W$ be a rank four $M$-valued symplectic bundle as above, and $V=S_{W}$
the associated rank five $M^{2}$-valued orthogonal bundle. We now determine
the quadric fibration $Q_{V}\subset{\mathbb{P}}V$.

By Proposition 6.5, the projective bundle ${\mathbb{P}}(\wedge^{2}W)$ is an
enveloping bundle for $Q_{\wedge^{2}W}=\mathrm{Gr}(2,W)$. As the orthogonal
structure on $V$ is inherited from $\wedge^{2}W$, we have

$Q_{S_{W}}\ =\ Q_{\wedge^{2}W}\cap{\mathbb{P}}V.$

Now recall from [GH78, p. 759 ff.] that a linear line complex in
${\mathbb{P}}^{3}$ is a three-dimensional family of lines defined by a
hyperplane section of the Plücker image of $\mathrm{Gr}(2,{\mathbb{C}}^{4})$
in ${\mathbb{P}}^{5}$. The datum of a linear line complex is equivalent to a
choice of bilinear antisymmetric form on ${\mathbb{C}}^{4}$ up to scalar,
which is nondegenerate if and only if the intersection is smooth;
equivalently, the hyperplane is not tangent to
$\mathrm{Gr}(2,{\mathbb{C}}^{4})$ at any point. By the above discussion, for
each $p\in C$, the hyperplane ${\mathbb{P}}V\subset{\mathbb{P}}(\wedge^{2}W)$
determines a smooth linear line complex in ${\mathbb{P}}W|_{p}$. Clearly this
is isomorphic to the Lagrangian Grassmannian $\mathrm{LG}(W|_{p})$
parameterizing two-dimensional $\alpha$-isotropic subspaces of $W|_{p}$.
Summarizing, we have:

######  Proposition 7.4.

Let $V=S_{W}$ be as above. Then the projectivization of the locus of isotropic
vectors in ${\mathbb{P}}V$ is the Lagrangian Grassmann bundle
$\mathrm{LG}(W)$.

###  7.3. Isotropic subbundles of $S_{W}$

Let $W$ be a rank four $M$-valued symplectic bundle. To classify rank two
isotropic subbundles of the rank five $M^{2}$-valued orthogonal bundle
$S_{W}$, we could use a strategy similar to that in [CCH21b, § 8]. However, we
offer instead an approach exploiting the geometry of the enveloping bundle,
which also gives further insight into the rank six case.

Let $E\subset S_{W}$ be a rank two isotropic subbundle. Then ${\mathbb{P}}E$
is a ${\mathbb{P}}^{1}$-bundle contained in $\mathrm{LG}(W)$. As above, by
nondegeneracy of the symplectic form, for each $p\in C$ the intersection
${\mathbb{P}}(S_{W}|_{p})\cap\mathrm{Gr}(2,W|_{p})$ is smooth. Then by [GH78,
p. 759], there exist a line $N_{p}$ and a hyperplane $H_{p}$ in $W|_{p}$ such
that ${\mathbb{P}}W$ parameterizes the pencil of two-dimensional (isotropic)
subspaces of $V|_{p}$ forming a complete flag

$0\ \subset\ \ N_{p}\ \subset\ \Lambda\ \subset\ H_{p}\ \subset\ W|_{p}.$

It follows that

(7.1) $E|_{p}\ =\ \\{v\wedge w\in\wedge^{2}W:v\in N_{p},w\in H_{p}\\}\ \cong\
N_{p}\otimes\frac{H_{p}}{N_{p}}.$

We claim moreover that $H_{p}=N_{p}^{\perp}$, the orthogonal complement of
$N_{p}$ with respect to the symplectic form $\alpha$. To see this: $H_{p}$
contains a pencil of two-dimensional subspaces containing $N_{p}$ which are
moreover isotropic, so in particular annihilate $N_{p}$. As $H_{p}$ has
dimension three, it is spanned by elements of $N_{p}^{\perp}$. As
$\dim(N_{p}^{\perp})=\dim H_{p}$, we have equality. Using local triviality, we
see that the union of the subspaces $N_{p}$ is a (uniquely determined) line
subbundle $N\subset W$.

We now define a map $\lambda_{N}\colon N\otimes N^{\perp}\to\wedge^{2}W$ by

$\lambda_{N}(v\otimes w)\ =\ v\wedge w.$

Clearly $\lambda_{N}$ is the restriction to $N\otimes N^{\perp}$ of the map
$\tau_{N}$ defined in (6.7), and

$\lambda_{N}(N\otimes N^{\perp})\ =\ \tau_{N}(N\otimes
V)\cap\left(\wedge^{2}N^{\perp}\right)\ \cong\ N\otimes\frac{N^{\perp}}{N}.$

Now as $N^{\perp}/N$ is $M$-valued symplectic of rank two, it has determinant
$M$ and degree $m$. Thus $\deg\mathrm{Im}\,(\lambda_{N})=2\cdot\deg N+m$.

Conversely, given any line subbundle $N\subset W$, the subbundle
$\lambda_{N}(N\otimes N^{\perp})$ is a rank two subbundle of $\wedge^{2}W$. As
$\alpha(v\wedge w)=0$ for any $v\in N$ and $w\in N^{\perp}$, we have
$\mathrm{Im}\,\left(\lambda_{N}\right)\subseteq\mathrm{Ker}\left(\alpha\colon\wedge^{2}W\to
M\right)=S_{W}$. As moreover every element of $\mathrm{Im}\,(\lambda_{N})$ is
decomposable, ${\mathbb{P}}\mathrm{Im}\,(\Lambda_{N})$ is contained in
$Q_{\wedge^{2}W}\cap{\mathbb{P}}S_{W}=\mathrm{LG}(W)$. Hence
$\mathrm{Im}\,(\Lambda_{N})$ is isotropic by Lemma 6.6. Thus we may summarize
as follows.

######  Theorem 7.5.

The association $N\mapsto\lambda_{N}(N\otimes N^{\perp})$ defines isomorphisms
$\mathrm{Quot}^{\circ}_{1,d}(W)\xrightarrow{\sim}\mathrm{IQ}^{\circ}_{2,2d+m}(S_{W})$.
Note also that the association
$N\mapsto\mathrm{Im}\,\left(\lambda_{N}\right)\otimes{{{\mathcal{O}}}_{C}}(x)$
defines an isomorphism

$\mathrm{Quot}^{\circ}_{1,d}(W)\cong\mathrm{IQ}^{\circ}_{2,2d+m+2}(S_{W}\otimes{{{\mathcal{O}}}_{C}}(x)).$

######  Remark 7.6.

Let $E$ be a rank two isotropic subbundle of $S_{W}$. We observe from the
description (7.1) that

$E\ =\ \tau_{N}(N\otimes V)\cap\left(\wedge^{2}N^{\perp}\right);$

that is, $E$ is the intersection of two isotropic rank three subbundles
$\wedge^{2}N^{\perp}$ and $N\otimes\frac{W}{N}$ of the rank six orthogonal
bundle $\wedge^{2}W$, as described in Theorem 6.8. Moreover, in view of [GH78,
Proposition, p. 735], the fact that

$\dim\left(\wedge^{2}\left(N^{\perp}|_{p}\right)\cap\lambda_{N}(N\otimes
N^{\perp})\right)\ =\ 2\ \not\equiv 3\mod 2$

reflects the fact that these two subbundles define elements of opposite
components of the orthogonal Grassmannian $\mathrm{OG}(3,\wedge^{2}V|_{p})$ at
each point.

Next, by Theorem 7.4, an isotropic line subbundle of $S_{W}$ is nothing but a
section of the Lagrangian Grassmannian bundle $\mathrm{LG}(W)\to C$. This
equivalent to a choice of rank two isotropic subbundle $E\subset W$, and then
$\wedge^{2}E$ is the isotropic line subbundle of $S_{W}$. Thus we have:

######  Theorem 7.7.

Let $W$ be an $M$-valued symplectic bundle and $S_{W}$ the associated rank
five $M^{2}$-valued orthogonal bundle. Then for each $d$, the assignment
$E\mapsto\wedge^{2}E$ defines an isomorphism

$\mathrm{LQ}_{d}^{\circ}(W)\ \xrightarrow{\sim}\
\mathrm{IQ}^{\circ}_{1,d}(S_{W}),$

where $\mathrm{LQ}_{d}^{\circ}(W)$ is the subscheme of
$\mathrm{Quot}^{\circ}_{2,d}(W)$ consisting of Lagrangian (maximal rank
isotropic) subbundles of the symplectic bundle $W$.

####  7.3.1. Enumeration of maximal isotropic subbundles

By Theorem 7.5, the number of rank two isotropic subbundles of maximal degree
of $V$ equals the number of maximal line subbundles of the rank four
symplectic bundle $W$. However, the counting formula in Theorem 2.5 is valid
only for a general vector bundle of rank four, while the symplectic bundles of
rank four are contained in a proper closed subset of the moduli of vector
bundles, so we cannot apply Theorem 2.5 directly. Therefore, we leave the
following as a conjecture.

######  Conjecture 7.8.

Let $V$ be a stable rank five ${{{\mathcal{O}}}_{C}}$-valued orthogonal bundle
which is general in its component of moduli.

  1. (a)

Suppose $g\equiv 1\mod 4$. Suppose $w_{2}(V)=0$, so that $V\cong S_{W}$ for a
$W\in{\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}})$ which may be
assumed to be general. Then $V$ has $4^{g}$ maximal rank two isotropic
subbundles, each of degree $-\frac{3}{2}(g-1)$.

  2. (b)

Suppose $g\equiv 3\mod 4$. Suppose $w_{2}(V)=1$, so that $V\cong
S_{W_{1}}\otimes{{{\mathcal{O}}}_{C}}(x)$ for some
$W_{1}\in{\mathcal{M}}{\mathcal{S}}_{C}(4,{{{\mathcal{O}}}_{C}}(-x))$ which
may be assumed to be general. Then $V$ has $4^{g}$ rank two isotropic
subbundles of maximal degree $-\frac{3}{2}(g-1)$.

On the other hand, for isotropic line subbundles we have the following.

######  Theorem 7.9.

Let $V$ be a rank five  ${\mathcal{O}}_{C}$-valued orthogonal bundle which is
general in its component of moduli. Then each isotropic line subbundle has
degree $1-g$, and the number of such subbundles is given by

$\begin{array}[]{|c|c|c|}&g\hbox{ even}&g\hbox{ odd}\\\ \hline\cr
w_{2}(V)=0&2^{g-1}\cdot(3^{g}-1)&2^{g-1}\cdot(3^{g}+1)\\\ \hline\cr
w_{2}(V)=1&2^{g-1}\cdot(3^{g}+1)&2^{g-1}\cdot(3^{g}-1)\end{array}$

###### Proof.

Suppose $w_{2}(V)=1$. By Theorem 7.2 (b), we may assume that
$V=S_{W}\otimes{{{\mathcal{O}}}_{C}}(x)$ for a rank four
${{{\mathcal{O}}}_{C}}(-x)$-valued symplectic $W$ which is general in moduli.
By Theorem 7.7, each maximal isotropic line subbundle is of the form
$(\wedge^{2}E)\otimes{{{\mathcal{O}}}_{C}}(x)$ for a maximal Lagrangian
subbundle $E\subset W$. By [CCH21a, Proposition 3.2] we have $\deg(E)=-g$,
whence $\deg(\wedge^{2}E\otimes{{{\mathcal{O}}}_{C}}(x))=1-g$; and the number
of such $E$ is given by [CCH21a, Corollary 6.3 (2) and (4)]. The case where
$w_{1}(V)=0$ is similar. ∎

##  8\. Higher rank

We shall now conclude by formulating some conjectures on orthogonal bundles of
higher rank and their isotropic Quot schemes in general, based on the extra
geometric information for low rank gathered in the previous sections. We shall
use results from the literature to a greater extent than before.

Let $i\in\\{0,1\\}$. By [CH14, § 5.4] and [CH15, Theorem 6.1], a general
$V\in{\mathcal{M}}{\mathcal{O}}_{C}^{i}(2n,{{{\mathcal{O}}}_{C}})$ or
${\mathcal{M}}{\mathcal{O}}_{C}^{i}(2n-1,{{{\mathcal{O}}}_{C}})$ has a finite
number of isotropic subbundles of rank $n$ and maximal degree
$-\frac{1}{2}n(g-1)$ under the following congruence condition:

$n(g-1)\equiv 2i\mod 4$

We denote this number by $N^{0}(g,r)$, where $r=2n$ or $2n-1$.

Theorems 4.12, 5.6 and 6.15 and Conjecture 7.8 suggest the following
equalities for arbitrary rank:

######  Conjecture 8.1.

Let $g,r,n,i$ be as above. For all $r\geq 3$, we have

$N^{0}(g,2n)\ =\ 2\cdot N^{0}(g,2n-1).$

An intuition behind the conjecture is as follows. We have briefly mentioned
the orthogonal Grassmannian bundle $\mathrm{OG}(V)$ of an orthogonal bundle
$V$, which parameterizes isotropic subspaces of maximal dimension in fibers of
$V$; equivalently, projective linear subspaces of maximal dimension contained
in the quadrics $Q_{V}|_{x}$. If $V$ has rank $2n$ then by [CCH21b,
Proposition 2.12], the fibration $\mathrm{OG}(V)$ has two connected components
$\mathrm{OG}(V)_{0}$ and $\mathrm{OG}(V)_{1}$. (This has a particularly clear
geometric interpretation in rank four; see Remark 4.9.) Following [CCH21b,
Definition 3.1], we define $\mathrm{IQ}^{\circ}_{n,e}(V)_{\delta}$ to be the
component of $\mathrm{IQ}^{\circ}_{n,e}(V)$ parameterizing subbundles
belonging to $\mathrm{OG}(V)_{\delta}$. By [CH15, Proposition 3.4 (3)] and
[CCH21b], for all $e$, there are canonical isomorphisms

$\mathrm{IQ}^{\circ}_{n,e}\left(V\perp{{{\mathcal{O}}}_{C}}\right)_{\delta}\
\xrightarrow{\sim}\ \mathrm{IQ}^{\circ}_{n-1,e}(V)$

for $\delta\in\\{0,1\\}$. (See Remark 7.6 for related discussion for rank
five.) In particular, the number of maximal isotropic subbundles of
$V\perp{{{\mathcal{O}}}_{C}}$ is exactly twice that of $V$. However, since a
bundle of the form $V\perp{{{\mathcal{O}}}_{C}}$ is not general in
${\mathcal{M}}{\mathcal{O}}_{C}^{i}(2n,{{{\mathcal{O}}}_{C}})$, we need an
argument on the conservation of number.

The same argument suggests:

######  Conjecture 8.2.

If a general $V\in{\mathcal{M}}{\mathcal{O}}_{C}(2n,{{{\mathcal{O}}}_{C}})$
has a finite number $N^{0}(g,2n)$ of rank $n$ maximal isotropic subbundles,
then exactly half of them define sections of $\mathrm{OG}(V)_{0}$ and another
half those of $\mathrm{OG}(V)_{1}$.

In a forthcoming paper with D. Cheong, we propose to compute the numbers
$N^{i}(r,g)$ more generally and to investigate the validity of the above
conjectures in general.

## References

  * [BS02] V. Balaji and C. S. Seshadri: Semistable principal bundles - I (characteristic zero). J. of Algebra 258 (2002), 321–47.
  * [Bea08] A. Beauville: Orthogonal bundles on curves and theta functions. Ann. Inst. Fourier 56, no. 5 (2008), 1405–1418.
  * [BLS89] A. Beauville, Y. Laszlo and C. Sorger: The Picard group of the moduli of $G$-bundles on a curve. Compos. Math. 112, no. 2 (1998), 183–216.
  * [BG10] I. Biswas and T. L. Gomez: Hecke transformation for orthogonal bundles and stability of Picard bundles. Commun. Analytic Geom. 18, no. 5 (2010), 857–890.
  * [CH14] I. Choe and G. H. Hitching: A stratification on the moduli spaces of symplectic and orthogonal bundles over a curve. International J. of Math., 25, no. 5 (2014), 27 pp.
  * [CH15] I. Choe and G. H. Hitching: Maximal isotropic subbundles of orthogonal bundles of odd rank over a curve. International J. of Math., 26, no. 13 (2015), 23 pp.
  * [CCH21a] D. Cheong, I. Choe and G. H. Hitching: Counting Lagrangian subbundles over an algebraic curve. J. of Geometry and Physics (2021); DOI: 10.1016/j.geomphys.2021.104288
  * [CCH21b] D. Cheong, I. Choe and G. H. Hitching: Isotropic Quot schemes of orthogonal bundles over a curve. International J. of Math. (2021); DOI: 10.1142/S0129167X21500476.
  * [DN89] J.-M. Drezet and M. S. Narasimhan: Groupe de Picard des variétés de modules de fibrés semi-stables sur les courbes algébriques. Invent. Math. 97, no. 1 (1989), 53–94.
  * [FH91] W. Fulton and J. Harris: Representation Theory – A First Course. GTM 129, Springer-Verlag, New York, 1991.
  * [GHS03] T. Graber, J. Harris, and J. Starr: Families of rationally connected varieties. J. Amer. Math. Soc. 16 (2003), no. 1, 57-67.
  * [Gro57] A. Grothendieck: Sur la classification des fibrés holomorphes sur la sphère de Riemann. Amer. J. Math. 79 (1957), 121–138.
  * [Gro58] A. Grothendieck: A General Theory of Fibre Spaces with Structure Sheaf, second ed. NSF report, Univ. of Kansas, 1958. Available at https://webusers.imj-prg.fr/~leila.schneps/grothendieckcircle/Kansasnotes.pdf .
  * [GH78] P. Griffiths; J. Harris: Principles of Algebraic Geometry. Wiley, USA, 1994.
  * [Har77] R. Hartshorne: Algebraic geometry Graduate Texts in Mathematics 52. New York-Heidelberg-Berlin: Springer-Verlag (1983).
  * [Hit07] G. H. Hitching: Subbundles of symplectic and orthogonal vector bundles over curves. Math. Nachr. 280, no. 13-14 (2007), 1510–1517.
  * [Hir96] Hirschowitz, A.: Problèmes de Brill–Noether en rang supérieur. Prépublications Mathématiques n. 91, Nice (1986).
  * [Hol04] Y. I. Holla: Counting maximal subbundles via Gromov–Witten invariants. Math. Ann. 328, no. 1-2 (2004), 121–133.
  * [LN03] H. Lange and P. E. Newstead: Maximal subbundles and Gromov–Witten invariants. V. Lakshmibai (ed.) et al., A tribute to C. S. Seshadri. A collection of articles on geometry and representation theory. Basel: Birkhäuser. Trends in Mathematics (2003), 310–322.
  * [LM20] A. Lanteri and R. Mallavibarrena: Projective bundles enveloping rational conic fibrations and osculation. J. Pure Appl. Algebra 224, no. 12 (2020), 20 pp.
  * [LMP15] A. Lanteri, R. Mallavibarrena, and R. Piene: Inflectional loci of quadric fibrations. J. Algebra 441 (2015), 363–397.
  * [Mum71] D. Mumford: Theta characteristics of an algebraic curve. Ann. Sci. Éc. Norm. Supér. (4) 4 (1971), 181–192.
  * [Oxb00] W. M. Oxbury: Varieties of maximal line subbundles. Math. Proc. of the Cambridge Phil. Soc. 129 (2000), 9–18.
  * [Ram81] S. Ramanan: Orthogonal and spin bundles over hyperelliptic curves. Proc. Indian Acad. Sci., Math. Sci. 90 (1981), 151–166.
  * [Rth96] A. Ramanathan: Moduli for principal bundles over algebraic curves: I & II. Proc. Indian Acad. Sci., Math. Sci. 106 (1996), no. 3, 301–328 and no. 4, 421–449.
  * [Serm12] O. Serman: Orthogonal and symplectic bundles on curves and quiver representations. Brion, M. (ed.), “Geometric methods in representation theory” II. Selected papers based on the presentations at the summer school, Grenoble, France, June 16 -– July 4, 2008. Paris: Soc. Math. de France. Séminaires et Congrès 24, pt. 2 (2012), 393–418.
  * [Serre90] J.-P. Serre, Revêtements à ramification impaire et thêta-caractéristiques. C. R. Acad. Sci. Paris Sér. I Math. 311, no. 9 (1990), 547–552.