# Refined Verlinde formulas for Hilbert schemes of points and moduli spaces of
sheaves on K3 surfaces

Lothar Göttsche  International Centre for Theoretical Physics, Strada Costiera
11, 34100 Trieste, Italy<EMAIL_ADDRESS>
  *     * scAbstract. We compute generating functions for elliptic genera with values in line bundles on Hilbert schemes of points on surfaces. As an application we also compute generating functions for elliptic genera with values in determinant line bundles on moduli spaces of sheaves on K3 surfaces.

scKeywords. Hilbert schemes; moduli spaces of sheaves; elliptic genus;
Verlinde formula

sc2020 Mathematics Subject Classification. 14C05; 14J60; 14J42; 58J26

sc[Français]

scFormules de Verlinde raffinées pour les schémas de Hilbert de points et
espaces de modules sur les surfaces K3

scRésumé. Nous calculons les fonctions génératrices pour les genres
elliptiques à valeur dans les fibrés en droites sur les schémas de Hilbert des
points sur les surfaces. En guise d’application, nous calculons également les
fonctions génératrices pour les genres elliptiques à valeurs dans les fibrés
en droites sur les espaces de modules de faisceaux sur les surfaces K3.

  *
cJuly 17, 2020Received by the Editors on March 13, 2019.
Accepted on August 19, 2020.

International Centre for Theoretical Physics, Strada Costiera 11, 34100
Trieste, Italy

sce-mail<EMAIL_ADDRESS>

© by the author(s) This work is licensed under
http://creativecommons.org/licenses/by-sa/4.0/

###### Contents

  1. 1 Introduction
  2. 2 Background material
    1. 2.1 Hilbert schemes of points
    2. 2.2 Elliptic genus.
    3. 2.3 Beauville-Bogomolov quadratic form
  3. 3 The case $r=0$.
  4. 4 The case of Hilbert schemes of points on K3 surfaces
  5. 5 Moduli of sheaves on K3 surfaces

## 1\. Introduction

The celebrated Verlinde formula (see [Ver88, NR93, BL94, Fal94]) is a formula
for the generating function for dimensions of spaces of sections of line
bundles on moduli spaces of vector bundles on algebraic curves.

Now let $S$ be a smooth projective surface over $\mathbb C$ and $S^{[n]}$ the
Hilbert scheme of $n$ points on $S$. For every vector bundle $V$ on $S$ there
is a corresponding tautological bundle $V^{[n]}$ of rank
$\operatorname{rk}(V^{[n]})=n\operatorname{rk}(V)$, whose fibre over $Z\in
S^{[n]}$ is $H^{0}(Z,V|_{Z})$. The map $V\mapsto V^{[n]}$ extends to a
homomorphism from the Grothendieck group $K^{0}(S)$ of vector bundles on $S$
to $K^{0}(S^{[n]})$. For $L\in S^{[n]}$ denote
$\mu(L):=\det((L-\mathcal{O}_{S})^{[n]})$ and
$E:=\det(\mathcal{O}^{[n]}_{S})$. Then it is well known that
$\operatorname{Pic}(S^{[n]})=\mu(\operatorname{Pic}(S))\oplus\mathbb ZE$. The
analogue of the Verlinde formula for Hilbert schemes of points is a formula
for the generating function for holomorphic Euler characteristics
$\chi(S^{[n]},\mu(L)\otimes E^{r})$. In [EGL01] such a formula was proven in
the cases $r=-1,0,1$ or $K_{S}^{2}=K_{S}L=0$. On the other hand the celebrated
Dijkgraaf-Moore-Verlinde-Verlinde formula [DMVV97], shown in [BL00, BL03,
BL05], relates the generating function of the elliptic genera
$\operatorname{Ell}(S^{[n]})$ of Hilbert schemes of points to Siegel modular
forms.

In this short note we interpolate between these two results, by proving a
formula for $\operatorname{Ell}(S^{[n]},\mu(L)\otimes E^{r})$, the elliptic
genus with values in the line bundle $\mu(L)\otimes E^{r}$. To state these
results we introduce the following power series.

$\displaystyle\phi_{-2,1}(q,y)$
$\displaystyle:=(y^{1/2}-y^{-1/2})^{2}\prodop\displaylimits_{n>0}\frac{(1-q^{n}y)^{2}(1-q^{n}/y)^{2}}{(1-q^{n})^{4}},$
$\displaystyle\wp(q,y)$
$\displaystyle:=\frac{1}{12}+\frac{1}{(y^{1/2}-y^{-1/2})^{2}}+\sumop\displaylimits_{n>0}\sumop\displaylimits_{d|n}d(y^{d}-2+y^{-d})q^{n},$
$\displaystyle\phi_{0,1}(q,y)$ $\displaystyle:=12\wp(x,y)\phi_{-2,1}(x,y),$
$\displaystyle h(q,y)$
$\displaystyle:=-\frac{1}{12}+\sumop\displaylimits_{n>0}\sumop\displaylimits_{d|n}\frac{n}{d}(y^{d}+y^{-d})x^{n}.$

$\wp(q,y)$ is the Weierstrass $\wp$-function. It is standard that

$\operatorname{Ell}(S)=-(\wp+h)\phi_{-2,1}K_{S}^{2}+\phi_{0,1}\chi(\mathcal{O}_{S}).$

We also introduce the following Borcherds type lifts. For
$f:=\sumop\displaylimits_{m,n}c_{m,n}q^{m}y^{n}\in\mathbb Q[[y,q]]$, we put

$\displaystyle\mathbf{L}(f,p)$
$\displaystyle:=\prodop\displaylimits_{l>0,m\geq 0,n\in\mathbb
Z}(1-p^{l}q^{m}y^{n})^{c_{lm,n}},$ $\displaystyle\mathbf{L}^{(a,b)}(f,p)$
$\displaystyle:=\prodop\displaylimits_{l>0,m\geq 0,n\in\mathbb
Z}(1-p^{l}q^{m}y^{n})^{l^{a}n^{b}c_{lm,n}},\quad a,b\in\mathbb Z.$

Then the DMVV formula says

$\sumop\displaylimits_{n\geq
0}\operatorname{Ell}(S^{[n]})p^{n}=\frac{1}{\mathbf{L}(\operatorname{Ell}(S),p)}=\frac{\mathbf{L}((\wp+h)\phi_{-2,1},p)^{K_{S}^{2}}}{\mathbf{L}(\phi_{0,1},p)^{\chi(\mathcal{O}_{S})}}.$

The first theorem deals with the case $r=0$.

######  Theorem 1.1.

Let $S$ be an algebraic surface, $L\in\operatorname{Pic}(S)$. Then

$\displaystyle\sumop\displaylimits_{n\geq
0}\operatorname{Ell}(S^{[n]},\mu(L))p^{n}=$
$\displaystyle\frac{\big{(}\mathbf{L}^{(2,0)}(-\phi_{-2,1},p)\big{)}^{\frac{L^{2}}{2}}\big{(}\mathbf{L}^{(1,1)}(-\phi_{-2,1},p)\big{)}^{\frac{LK_{S}}{2}}}{\mathbf{L}(\operatorname{Ell}(S),p)}.$

Specializing to $L=\mathcal{O}_{S}$, we recover the DMVV formula. Specializing
to $q=0$ yields an infinite product formula for the $\chi_{y}$-genera with
values in a $\mu(L)$, which in turn recovers for $L=\mathcal{O}_{S}$ the
formula for the $\chi_{-y}$-genera of Hilbert schemes from [GS93]. We write

$\widetilde{\Delta}(p,y):=\prodop\displaylimits_{n>0}(1-p^{n})^{20}(1-p^{n}y)^{2}(1-p^{n}/y)^{2},\quad\overline{\eta}(p)=\prodop\displaylimits_{n>0}(1-p^{n}).$

######  Corollary 1.2.

$\displaystyle\sumop\displaylimits_{n\geq
0}\overline{\chi}_{-y}(S^{[n]},\mu(L))p^{n}$
$\displaystyle=\prodop\displaylimits_{n>0}\left(\frac{(1-p^{n})^{2}}{(1-p^{n}y)(1-\frac{p^{n}}{y})}\right)^{\frac{n^{2}}{2}L^{2}}\prodop\displaylimits_{n>0}\left(\frac{1-\frac{p^{n}}{y}}{1-p^{n}y}\right)^{\frac{n}{2}LK_{S}}\frac{\overline{\eta}(p)^{K_{S}^{2}}}{\widetilde{\Delta}(p,y)^{\frac{\chi(\mathcal{O}_{S})}{2}}}.$

For general line bundles on Hilbert schemes of points we can partially
determine the generating function.

######  Theorem 1.3.

For every $r\in\mathbb Z$ there are universal power series
$A_{r},B_{r}\in\mathbb Q[y^{\pm 1}][[z,q]]$ such that for every smooth
projective surface $S$ and every $L\in\operatorname{Pic}(S)$ we have

$\displaystyle\sumop\displaylimits_{n\geq
0}\operatorname{Ell}(S^{[n]},\mu(L)\otimes E^{r})p^{n}$
$\displaystyle=\frac{\mathbf{L}^{(2,0)}(-\phi_{-2,1},z)^{\frac{1}{2}(L^{2}+r^{2}\chi(\mathcal{O}_{S}))}A_{r}^{LK_{S}}B_{r}^{K_{S}^{2}}}{\left(\mathbf{L}(2\phi_{0,1},z)\left(1-r^{2}z\frac{\partial}{\partial
z}\log\mathbf{L}^{(2,0)}(\phi_{-2,1},z)\right)\right)^{\frac{\chi(\mathcal{O}_{S})}{2}}},$

with the change of variables $p=z\mathbf{L}^{(2,0)}(-\phi_{-2,1},z)^{r^{2}}$.

Specialising to $q=0$ again yields a formula for the $\chi_{-y}$-genus with
values in $\mu(L)\otimes E^{r}$.

######  Corollary 1.4.

For every smooth projective surface $S$ and every $L\in\operatorname{Pic}(S)$
we have

$\displaystyle\sumop\displaylimits_{n\geq
0}\overline{\chi}_{-y}(S^{[n]},\mu(L)\otimes E^{r})p^{n}$
$\displaystyle=\prodop\displaylimits_{n>0}\left(\frac{(1-z^{n})^{2}}{(1-z^{n}y)(1-z^{n}/y)}\right)^{\frac{n^{2}}{2}(L^{2}+r^{2}\chi(\mathcal{O}_{S}))}$
$\displaystyle\times\frac{A_{r}(y,z,0)^{LK_{S}}B_{r}(y,z,0)^{K_{S}^{2}}}{\left(\widetilde{\Delta}(z,y)\left(1+r^{2}\sumop\displaylimits_{n\geq
1}(y^{d}-2+y^{-d})\frac{n^{3}}{d^{3}}z^{n}\right)\right)^{\frac{\chi(\mathcal{O}_{S})}{2}}}.$

with the change of variables
$p=z\prodop\displaylimits_{n>0}\left(\frac{(1-z^{n})^{2}}{(1-z^{n}y)(1-z^{n}/y)}\right)^{n^{2}r^{2}}$.

Finally we show an analogue of Theorem 1.3 for moduli spaces of sheaves on K3
surfaces $S$. Fix $s\in\mathbb Z_{>0}$. Let $H$ be an ample line bundle on $S$
and $M_{S}^{H}(s,c_{1},c_{2})$ the moduli space of $H$-semistable sheaves on
$S$ of rank $s$ with Chern classes $c_{1}$, $c_{2}$. We assume that
$M_{S}^{H}(s,c_{1},c_{2})$ consists only of stable sheaves. For any choice of
$s,c_{1},c_{2}$, we denote

$\operatorname{vd}=\operatorname{vd}(s,c_{1},c_{2}):=2sc_{2}-(s-1)c_{1}^{2}-2(s^{2}-1).$

Let $r\in\mathbb Z$. Let $L\in\operatorname{Pic}(S)\otimes\mathbb Q$ with
$c_{1}(L)-\frac{r}{s}c_{1}\in H^{2}(S,\mathbb Z)$. If $s$ divides
$(c_{1}(L)-\frac{r}{s}c_{1})c_{1}+r(\frac{c_{1}^{2}}{2}-c_{2})$, we define a
determinant line bundle $\mu(L)\otimes E^{r}\in
Pic(M^{H}_{S}(s,c_{1},c_{2}))$. This is the generalization of the line bundle
with the same name on $S^{[n]}=M^{H}_{S}(1,0,n)$. We obtain the following
result.

######  Theorem 1.5.

Let $S$ be a K3 surface, $s\in\mathbb Z_{>0}$. Under the assumptions above we
have

$\displaystyle\operatorname{Ell}(M^{H}_{S}(s,c_{1},c_{2}),\mu(L)\otimes
E^{r})$
$\displaystyle=\mathrm{Coeff}_{p^{\operatorname{vd}/2}}\left[\frac{\mathbf{L}^{(2,0)}(-\phi_{-2,1},z)^{\frac{L^{2}}{2}+\frac{r^{2}}{s^{2}}}}{\mathbf{L}(2\phi_{0,1},z)\left(1-\frac{r^{2}}{s^{2}}z\frac{\partial}{\partial
z}\log\mathbf{L}^{(2,0)}(\phi_{-2,1},z)\right)}\right],$

with the change of variables
$p=z\mathbf{L}^{(2,0)}(-\phi_{-2,1},z)^{\frac{r^{2}}{s^{2}}}$, and

$\displaystyle\overline{\chi}_{-y}(M^{H}_{S}(s,c_{1},n),\mu(L)\otimes E^{r})$
$\displaystyle=\mathrm{Coeff}_{p^{\operatorname{vd}/2}}\left[\frac{\prodop\displaylimits_{n>0}\left(\frac{(1-z^{n})^{2}}{(1-z^{n}y)(1-z^{n}/y)}\right)^{n^{2}(\frac{L^{2}}{2}+\frac{r^{2}}{s^{2}})}}{\left(\widetilde{\Delta}(z,y)\left(1+\frac{r^{2}}{s^{2}}\sumop\displaylimits_{n\geq
1}\sumop\displaylimits_{d|n}(y^{d/2}-y^{-d/2})^{2}\frac{n^{3}}{d^{3}}z^{n}\right)\right)}\right],$

with the change of variables
$p=z\prodop\displaylimits_{n>0}\left(\frac{(1-z^{n})^{2}}{(1-z^{n}y)(1-z^{n}/y)}\right)^{\frac{n^{2}r^{2}}{s^{2}}}$.

In the special case of K3 surfaces Theorem 1.5 in particular confirms the
conjectures of [GKW] about refinements of Verlinde formulas for moduli spaces
of rank 2 sheaves on surfaces in the case of K3 surfaces. The specialization
$L=\mathcal{O}_{X}$ reproduces in the case of K3 surfaces the formulas of [GK]
on the elliptic genus of moduli spaces of sheaves on surfaces.

######  Acknowledgements.

I thank Don Zagier for helping me with the proof of Lemma 4.3. This work grew
out of collaboration with Martijn Kool. I thank him for many useful
discussions.

## 2\. Background material

### 2.1. Hilbert schemes of points

Let $S$ be a smooth projective surface. We denote $S^{[n]}$ the Hilbert scheme
of $n$ points on $S$. It is a smooth projective variety of dimension $2n$. Let
$S^{(n)}$ be the $n$-th symmetric power of $S$. The Hilbert-Chow morphism
$\pi:S^{[n]}\to S^{(n)},Z\mapsto supp(Z)$, sending a zero dimensional scheme
to its support with multiplicities is a crepant resolution of $S^{(n)}$, i.e.
it is birational and $\pi^{*}K_{S^{(n)}}=K_{S^{[n]}}$. Let $Z_{n}(S)\subset
S\times S^{[n]}$ be the universal subscheme, with projections $p:Z_{n}(S)\to
S^{[n]}$, $q:Z_{n}(S)\to S$. For a vector bundle $V$ of rank $r$ on $S$ the
corresponding tautological vector bundle is $V^{[n]}:=p_{*}q^{*}V$, a vector
bundle of rank $rn$ on $S^{[n]}$. This extends to a homomorphism
$\vphantom{S}{}^{[n]}:K^{0}(S)\to K^{0}(S^{[n]})$ between the Grothendieck
groups of vector bundles. We put $E:=\det(\mathcal{O}_{S}^{[n]})$, and for a
line bundle $L$ on $S$ we put $\mu(L):=\det((L-\mathcal{O}_{S})^{[n]})$. Let
$\eta:S^{n}\to S^{(n)}$ be the natural projection. Let
$L_{n}:=\eta_{*}(\otimes_{i=1}^{n}pr_{i}^{*}L)^{\mathfrak S_{n}}$ be the
$\mathfrak S_{n}$-equivariant pushforward, where $pr_{i}:S^{n}\to S$ is the
$i$-th projection. Then it is well-known that $\mu(L)=\eta^{*}(L_{n})$, and
from the definitions it follows that $\det(V^{[n]})=\mu(\det(V))\otimes
E^{\operatorname{rk}(V)}$.

### 2.2. Elliptic genus.

For a compact complex manifold $M$, the $\chi_{-y}$-genus is

$\chi_{-y}(M):=\sumop\displaylimits_{p}(-y)^{p}\chi(M,{}^{p}_{M}).$

Usually we consider the normalized version
$\overline{\chi}_{-y}(M):=(-y)^{-\frac{\dim(M)}{2}}\chi_{-y}(M)$. For $V\in
K^{0}(M)$ the $\chi_{-y}$-genus with values in $V$ is

$\overline{\chi}_{-y}(M,V):=y^{-\frac{\dim(M)}{2}}\sumop\displaylimits_{p}(-y)^{p}\chi(M,{}^{p}_{M}\otimes
V).$

For a rank $r$ vector bundle $V$ on $M$ put

$\displaystyle{}_{t}V:=\sumop\displaylimits_{n=0}^{r}[{}^{n}V]\,t^{n},\quad\operatorname{Sym}_{t}V:=\sumop\displaylimits_{n=0}^{\infty}[\operatorname{Sym}^{n}V]\,t^{n}.$

Write $y=e^{2\pi iz}$, $q=e^{2\pi i\tau}$. Then for $W\in K^{0}(M)$, the
elliptic genus and the elliptic genus with values in $W$ are defined by

$\displaystyle\operatorname{Ell}(M)$
$\displaystyle:=\operatorname{Ell}(M,z,\tau):=\overline{\chi}_{-y}(M,\mathcal{E}(T_{M})),$
$\displaystyle\operatorname{Ell}(M,W)$
$\displaystyle:=\operatorname{Ell}(M,W,z,\tau):=\overline{\chi}_{-y}(M,\mathcal{E}(T_{M})\otimes
W),$

with

$\displaystyle\mathcal{E}(V)$
$\displaystyle:=\bigotimesop\displaylimits_{n=1}^{\infty}{}_{-yq^{n}}V^{\vee}\otimes{}_{-y^{-1}q^{n}}V\otimes\operatorname{Sym}_{q^{n}}(V\oplus
V^{\vee}).$

Let

$\theta(z,\tau):=-iq^{\frac{1}{8}}(y^{\frac{1}{2}}-y^{-\frac{1}{2}})\prodop\displaylimits_{n>0}(1-q^{n})(1-q^{n}y)(1-q^{n}/y)$

be the classical Jacobi theta function, where we write $y=e^{2\pi iz}$,
$q=e^{2\pi i\tau}$. Let $c(M)=\prodop\displaylimits_{j=1}^{n}(1+x_{j})$ be a
formal splitting of the total Chern class of $M$. Putting

$\operatorname{ELL}(M):=\operatorname{ELL}(M,z,\tau)=\prodop\displaylimits_{j}x_{j}\frac{\theta(\frac{x_{j}}{2\pi
i}-z,\tau)}{\theta(\frac{x_{j}}{2\pi i},\tau)}\in H^{*}(X,\mathbb
Q)[y^{\pm\frac{1}{2}}][[q]],$

it follows from the definitions and the Riemann-Roch theorem that

$\operatorname{Ell}(M)=\intop\nolimits_{M}\operatorname{ELL}(M),\quad\operatorname{Ell}(M,W)=\intop\nolimits_{M}\operatorname{ELL}(M)\operatorname{ch}(W).$

### 2.3. Beauville-Bogomolov quadratic form

Let $X$ be a compact holomorphic symplectic manifold of dimension
$\dim(X)=2m$. We briefly review some properties of the Beauville-Bogomolov
quadratic form $q_{X}:H^{2}(X)\to\mathbb Q$ on $X$ from [Huy99, Sections
1.9–1.11]. Note that the odd Chern classes of $X$ vanish.

######  Theorem 2.1.

For any $\beta\in H^{4k}(X,\mathbb Q)$ in the sub-algebra generated by the
Chern classes of $X$, there is a constant $c(\beta)\in\mathbb Q$, such that
for all $\alpha\in H^{2}(X,\mathbb Q)$

$\intop\nolimits_{X}\beta\alpha^{\dim(X)-2k}=c(\beta)q_{X}(\alpha)^{m-k}.$

The quantity $c(\beta)$ is invariant under deformation of $X$.

######  Corollary 2.2.

There exists a polynomial $h_{X}=h_{X}(z)$ with coefficients in $\mathbb
Q[y^{\pm 1}][[q]]$ such that

$\intop\nolimits_{X}\operatorname{ELL}(X)\exp(\alpha)=h_{X}(q_{X}(\alpha)),\quad\hbox{for
all }\alpha\in H^{2}(X,\mathbb Z).$

The polynomial $h$ is invariant under deformation of $X$.

## 3\. The case $r=0$.

In this section we will prove Theorem 1.1. We start by reviewing some of the
ideas and definitions of [BL05]. For a Kawamata log-terminal pair $(Z,D)$ of a
projective variety and a divisor $D$ in $Z$ with an action of a finite group
$G$, Borisov and Libgober define in [BL05, Definition 3.6] the orbifold
elliptic class
$\operatorname{Ell}_{\operatorname{orb}}(Z,D,G)=\operatorname{Ell}_{\operatorname{orb}}(Z,D,G,z,\tau)$.
In fact they first define it in [BL05, Definition 3.2] in case $X$ is
nonsingular and the pair $(X,E)$ is also $G$-normal (see [BL05, Definition
3.1] for the definitions) by an explicit formula. In the general case, they
define in [BL05, Definition 3.6]
$\operatorname{Ell}_{\operatorname{orb}}(Z,D,G):=\rho_{*}\operatorname{Ell}_{\operatorname{orb}}(X,E,G)$
for $\rho:(X,E)\to(Z,D)$ a $G$-normal equivariant resolution. We will write
$\operatorname{Ell}_{\operatorname{orb}}(Z,G)$,
$\operatorname{Ell}_{\operatorname{orb}}(Z,D)$,
$\operatorname{Ell}_{\operatorname{orb}}(Z)$ in case $D=0$ and/or $G$ is the
trivial group.

If $X$ is a nonsingular projective variety, with an action of a finite group
their formula specializes to

(3.1)
$\displaystyle\operatorname{Ell}_{\operatorname{orb}}(X,G):=\frac{1}{|G|}\sumop\displaylimits_{gh=hg}\sumop\displaylimits_{Z}[Z]\Big{(}\prodop\displaylimits_{\lambda(g)=\lambda(h)=0}\prodop\displaylimits_{j}x_{\lambda,j}\Big{)}\prodop\displaylimits_{\lambda,j}\frac{\theta(\frac{x_{\lambda,j}}{2\pi
i}+\lambda(g)-\tau\lambda(h)-z)}{\theta(\frac{x_{\lambda,j}}{2\pi
i}+\lambda(g)-\tau\lambda(h))}e^{2\pi i\lambda(h)z}.$

Here $Z$ runs over the irreducible components of the common fixpoint set of
$g$ and $h$, and $[Z]$ is the class of $Z$ in the Chow group of $X$. The
restriction of $T_{X}$ to Z splits into linearized bundles according to the
$[0,1)$-valued characters $\lambda$ of $\langle g,h\rangle$. We denote by
$x_{\lambda,j}$ the elements of a formal splitting of the total Chern class of
the bundle with character $\lambda$. If $G$ acts effectively on $(Z,D)$ and
$(Z/G,D/G)$ is the quotient pair in the sense of [BL05, Definition 2.7], they
show in [BL05, Theorem. 5.3] that
$\psi_{*}\operatorname{Ell}_{\operatorname{orb}}(Z,D)=\operatorname{Ell}_{\operatorname{orb}}(Z/G,D/G)$
for the quotient morphism $\psi:Z\to Z/G$. This is in particular true for the
pairs $(Z,0)$, $(Z/G,0)$, if $Z$ is nonsingular and $G$ is acting freely in
codimension $1$.

Now consider the action of $\mathfrak S_{n}$ on $S^{n}$, and recall the
quotient morphism $\eta:S^{n}\to S^{(n)}$ and the Hilbert Chow morphism
$\pi:S^{[n]}\to S^{(n)}$. As $S^{[n]}$ is nonsingular we have
$\operatorname{ELL}(S^{[n]})=\operatorname{Ell}_{\operatorname{orb}}(S^{[n]})$.
As $\pi$ is a crepant resolution, [BL05, Theorem 3.5] implies that
$\pi_{*}\operatorname{ELL}(S^{[n]})=\operatorname{Ell}_{\operatorname{orb}}(S^{(n)})$.
Thus we find by the above

(3.2)
$\pi_{*}\operatorname{ELL}(S^{[n]})=\operatorname{Ell}_{\operatorname{orb}}(S^{(n)})=\eta_{*}\operatorname{Ell}_{\operatorname{orb}}(S^{n},\mathfrak
S_{n}).$

Now let $L$ be a line bundle on $S$, then we have

$\displaystyle\operatorname{Ell}(S^{[n]},\mu(L))$
$\displaystyle=\intop\nolimits_{S^{[n]}}\operatorname{ELL}(S^{[n]})\operatorname{ch}(\mu(L))$
$\displaystyle=\intop\nolimits_{S^{(n)}}\pi_{*}\big{(}\operatorname{ELL}(S^{[n]})\big{)}\operatorname{ch}(L_{n})$
$\displaystyle=\intop\nolimits_{S^{n}}\operatorname{Ell}_{\operatorname{orb}}(S^{n},\mathfrak
S_{n})\operatorname{ch}\Big{(}\sumop\displaylimits_{i=1}^{n}pr_{i}^{*}L\Big{)}.$

In the second line we have used $\pi^{*}(L_{n})=\mu(L)$ and the projection
formula, and in the third line we have used (3.2),
$\eta^{*}(L_{n})=\otimes_{i=1}^{n}pr_{i}^{*}L$ and again the projection
formula.

Let $x_{1},x_{2}$ be the Chern roots of $T_{S}$. Then

(3.3)
$\begin{split}&\operatorname{Ell}(S,L)=\intop\nolimits_{S}(1+L+\frac{L^{2}}{2})\prodop\displaylimits_{j=1}^{2}x_{i}\frac{\theta(\frac{x_{j}}{2\pi
i}-z,\tau)}{\theta(\frac{x_{j}}{2\pi i},\tau)}\\\
&=\operatorname{Ell}(S)-\frac{K_{S}L}{2}\mathrm{Coeff}_{x^{1}}\left(x\frac{\theta(\frac{x}{2\pi
i}-z,\tau)}{\theta(\frac{x}{2\pi
i},\tau)}\right)^{2}+\frac{L^{2}}{2}\mathrm{Coeff}_{x^{2}}\left(x\frac{\theta(\frac{x}{2\pi
i}-z,\tau)}{\theta(\frac{x}{2\pi i},\tau)}\right)^{2}\\\
&=\operatorname{Ell}(S)+\frac{K_{S}L}{2}y\frac{\partial}{\partial
y}\phi_{-2,1}+\frac{L^{2}}{2}\phi_{-2,1}.\end{split}$

Now we prove Theorem 1.1 by adapting the proof of [BL05, Theorem 6.1]. Note
that in the notations of [BL05] we have $D=0$, which leads to many
simplifications.

Let $(g,h)$ be a commuting pair in $\mathfrak S_{n}$. We sum up the
description of the action of $g,h$ and their fixpoint sets in the proof of
[BL05, Theorem 6.1]. We have a decomposition
$\\{1,\ldots,n\\}=J_{1}\cup\ldots\cup J_{m}$ into the orbits of the subgroup
generated by $g,h$. Thus the action of $(g,h)$ on $S^{n}$ restricts to an
action on each of the corresponding products $S^{J_{l}}$. Furthermore we can
write $|J_{l}|=a_{l}b_{l}$ for positive integers $a_{l},b_{l}$, and up to
reordering of the elements of $J_{l}$ the action of $(h,g)$ on $S^{J_{l}}$ can
be described as follows. Write $(y_{i,j})_{i\in\mathbb Z/a_{l}Z,\ j\in\mathbb
Z/b_{l}\mathbb Z}$ the components of elements of $y\in S^{J_{l}}$. Then the
action of $g,h$ on $S^{J_{l}}$ is given by $h(y_{i,j})=y_{i,j+1}$,
$g(y_{i,j})=y_{i+1,j}$ for $0\leq i<a_{l}-1$, $g(y_{a_{l}-1,j})=y_{0,j+s}$,
for some $s\in\\{0,\ldots,b_{l}-1\\}$, and $s$ determines the action of
$(g,h)$ on $J_{l}$ uniquely. The fixpoint set $(S^{J_{l}})^{g,h}$ is $S$
embedded via the diagonal map $j_{l}:S\to S^{J_{l}}$.

Changing their notation slightly, we denote for $(a,b,s):=(a_{l},b_{l},s_{l})$
by

$F_{a,b,s}:=j^{*}_{l}\left(\Big{(}\prodop\displaylimits_{\lambda(g)=\lambda(h)=0}\prodop\displaylimits_{j}x_{\lambda,j}\Big{)}\prodop\displaylimits_{\lambda,j}\frac{\theta(\frac{x_{\lambda,j}}{2\pi
i}+\lambda(g)-\tau\lambda(h)-z)}{\theta(\frac{x_{\lambda,j}}{2\pi
i}+\lambda(g)-\tau\lambda(h))}e^{2\pi i\lambda(h)z}\right)$

the pullback of the contribution of the restriction of the pair $(f,g)$ to
$\operatorname{Ell}_{\operatorname{orb}}(S^{J_{l}},\mathfrak S_{J_{l}})$ in
(3.1) multiplied by $a!b!$. Then it is shown in [BL05, Lemma 6.4] that

$F_{a,b,s}=\prodop\displaylimits_{j=1,2}\frac{x_{j}\theta(\frac{ax_{j}}{2\pi
i}-az,\frac{a\tau-s}{b})}{\theta(\frac{ax_{j}}{2\pi i},\frac{a\tau-s}{b})}.$

Note that the left hand side is

$\frac{1}{a^{2}}\prodop\displaylimits_{j=1,2}\frac{ax_{j}\theta(\frac{ax_{j}}{2\pi
i}-az,\frac{a\tau-s}{b})}{\theta(\frac{ax_{j}}{2\pi
i},\frac{a\tau-s}{b})}=\sumop\displaylimits_{k=0}^{2}a^{k-2}\operatorname{ELL}(S,az,\frac{a\tau-s}{b})_{k},$

where $(\ )_{k}$ denotes the part in degree $k$. As
$j_{l}^{*}\eta^{*}(L_{n})=L^{ab}$, we obtain

$\displaystyle\intop\nolimits_{S}F_{a,b,s}\operatorname{ch}(j_{l}^{*}\eta^{*}(L_{n}))$
$\displaystyle=\intop\nolimits_{S}\Big{(}\sumop\displaylimits_{k=0}^{2}a^{k-2}\operatorname{ELL}(S,az,\frac{a\tau-s}{b})_{k}(ab)^{2-k}\operatorname{ch}_{2-k}(L)\Big{)}$
$\displaystyle=\operatorname{Ell}(S,L^{b},az,\frac{a\tau-s}{b}).$

By definition it is clear that the contribution of $(g,h)$ to
$\intop\nolimits_{S^{n}}\operatorname{Ell}_{\operatorname{orb}}(S^{n},\mathfrak
S_{n})\eta^{*}L_{n}$ is

$\frac{1}{n!}\prodop\displaylimits_{l=1}^{m}\intop\nolimits_{S}F_{a_{l},b_{l},s_{l}}j_{l}^{*}\eta^{*}(L_{n}).$

Thus arguing as after [BL05, Lemma 6.6], writing

$\operatorname{Ell}(S^{[n]})=\sumop\displaylimits_{l,n}c_{l,n}y^{l}q^{n},\quad\phi_{-2,1}=\sumop\displaylimits_{l,n}d_{l,n}y^{l}q^{n},$

and using (3.3) in the third line, we obtain

$\displaystyle\sumop\displaylimits_{n\geq 0}\operatorname{Ell}$
$\displaystyle(S^{[n]},\mu(L))p^{n}=\sumop\displaylimits_{n\geq
0}p^{n}\Big{(}\intop\nolimits_{S^{n}}\operatorname{Ell}_{\operatorname{orb}}(S^{n},\mathfrak
S_{n})\eta^{*}L_{n}\Big{)}$
$\displaystyle=\exp\left(\sumop\displaylimits_{a,b>0}\sumop\displaylimits_{s=0}^{b-1}\frac{p^{ab}}{ab}\operatorname{Ell}(X,L^{b},az,\frac{a\tau-s}{b})\right)$
$\displaystyle=\exp\left(\sumop\displaylimits_{a,b>0}\sumop\displaylimits_{m,l}\sumop\displaylimits_{s=0}^{b-1}\Big{(}c_{m,l}+\Big{(}bl\frac{LK_{S}}{2}+b^{2}\frac{L^{2}}{2}\Big{)}d_{m,l}\Big{)}\frac{p^{ab}}{ab}y^{al}q^{\frac{am}{b}}e^{2\pi
i\frac{ms}{b}}\right)$
$\displaystyle=\exp\left(\sumop\displaylimits_{a,b>0}\sumop\displaylimits_{m,l}\Big{(}c_{mb,l}+\Big{(}bl\frac{LK_{S}}{2}+b^{2}\frac{L^{2}}{2}\Big{)}d_{mb,l}\Big{)}\frac{p^{ab}}{a}y^{al}q^{am}\right)$
$\displaystyle=\prodop\displaylimits_{b=1}^{\infty}\prodop\displaylimits_{m,l}(1-p^{b}y^{l}q^{m})^{-c_{mb,l}-lbd_{mb,l}\frac{K_{S}L}{2}-b^{2}d_{mb,l}\frac{L^{2}}{2}}.$

This proves Theorem 1.1. ∎

To deduce Corollary 1.2 from Theorem 1.1, we note that
$\overline{\chi}_{-y}(S^{[n]},\mu(L))=\operatorname{Ell}(S^{[n]},\mu(L))\big{|}_{q=0}$,
and by definition $\mathbf{L}(f,p)\big{|}_{q=0}=\mathbf{L}(f|_{q=0},p)$. Thus
we have

$\displaystyle\mathbf{L}(-\phi_{-2,1},p)\big{|}_{q=0}$
$\displaystyle=\mathbf{L}(-y+2-y^{-1},p)=\prodop\displaylimits_{n>0}\frac{(1-p^{n})^{2}}{(1-yp^{n})(1-y^{-1}p^{n})},$
$\displaystyle\mathbf{L}^{(2,0)}(-\phi_{-2,1},p)\big{|}_{q=0}$
$\displaystyle=\prodop\displaylimits_{n>0}\left(\frac{(1-p^{n})^{2}}{(1-yp^{n})(1-y^{-1}p^{n})}\right)^{n^{2}},$
$\displaystyle\mathbf{L}^{(1,1)}(-\phi_{-2,1},p)\big{|}_{q=0}$
$\displaystyle=\mathbf{L}^{(1,0)}(y^{-1}-y,p)=\prodop\displaylimits_{n>0}\left(\frac{1-y^{-1}p^{n}}{1-yp^{n}}\right)^{n},$
$\displaystyle\mathbf{L}((\wp+h)\phi_{-2,1},p)\big{|}_{q=0}$
$\displaystyle=\mathbf{L}(1,p)=\overline{\eta}(p)\quad\mathrm{and}\quad\mathbf{L}(\phi_{0,1},p)\big{|}_{q=0}=\mathbf{L}(y+10+y^{-1})=\Delta(y,p)^{1/2}.$

∎

## 4\. The case of Hilbert schemes of points on K3 surfaces

Now we want to consider the case of Hilbert schemes of points on K3 surfaces.
We obtain a formula for all $r$.

######  Proposition 4.1.

Let $S$ be a K3 surface and $L\in\operatorname{Pic}(S)$. Then

$\operatorname{Ell}(S^{[n]},\mu(L)\otimes
E^{r})=\mathrm{Coeff}_{p^{n}}\left[{\big{(}\mathbf{L}^{(2,0)}(-\phi_{-2,1},p)\big{)}^{\frac{L^{2}}{2}-r^{2}(n-1)}\mathbf{L}(-2\phi_{0,1},p)}\right].$

###### Proof.

The Hilbert scheme $S^{[n]}$ is a holomorphic symplectic manifold; let
$q_{S^{[n]}}$ be its Beauville-Bogolomov quadratic form. By Corollary 2.2
there exists a polynomial $h_{S^{[n]}}(z)$ with coefficients in $\mathbb
Q[y^{\pm 1}][[q]]$ such that

(4.1)
$\operatorname{Ell}(S^{[n]},M)=h_{S^{[n]}}\big{(}q_{S^{[n]}}(c_{1}(M))\big{)},\quad\hbox{for
all }M\in\operatorname{Pic}(S^{[n]}).$

It is shown in [Bea83, lem. 9.1] that for $L\in\operatorname{Pic}(S)$ we have

(4.2) $q_{S^{[n]}}(L+rE)=L^{2}-2r^{2}(n-1).$

Therefore Proposition 4.1 follows from Theorem 1.1.∎

Now we want to deduce Theorem 1.3 from Proposition 4.1. We know

$\operatorname{Ell}(S^{[n]},\mu(L)\otimes
E^{r})=\intop\nolimits_{S^{[n]}}\operatorname{ELL}(S^{[n]})\operatorname{ch}(\mu(L)\otimes
E^{r}),$

where $\operatorname{ELL}$ is the genus associated to a power series, and
$\mu(L)\otimes E^{r}=\det((L+(r-1)\mathcal{O}_{S}))^{[n]}$. Therefore [EGL01,
Theorem 4.2] applies and gives the following.

######  Corollary 4.2.

For every $r\in\mathbb Z$, there are universal power series
$F_{r,1},F_{r,2},F_{r,3},F_{r,4}\in\mathbb Q[y^{\pm 1}][[q,p]]$ such that for
every smooth projective surface $S$ and every $L\in\operatorname{Pic}(S)$ we
have

$\sumop\displaylimits_{n\geq 0}\operatorname{Ell}(S^{[n]},\mu(L)\otimes
E^{r})p^{n}=F_{r,1}^{L^{2}/2}F_{r,2}^{LK_{S}/2}F_{r,3}^{K_{S}^{2}}F_{r,4}^{\chi(\mathcal{O}_{S})}.$

Using Corollary 4.2, in order to prove Theorem 1.3, we only need show the
formulas for $F_{r,1}$ and $F_{r,4}$, which are determined by their values for
$S$ a K3 surface. So let again $S$ be a K3 surface, and
$L\in\operatorname{Pic}(S)$. Then by Proposition 4.1 we get

$\sumop\displaylimits_{n\geq 0}\operatorname{Ell}(S^{[n]},\mu(L)\otimes
E^{r})p^{n}=\sumop\displaylimits_{n\geq
0}p^{n}\mathrm{Coeff}_{p^{n}}\left[\big{(}\mathbf{L}^{(2,0)}(-\phi_{-2,1})(p)^{\frac{L^{2}}{2}-(n-1)r^{2}}\mathbf{L}(-2\phi_{0,1})(p)\big{)}\right].$

Thus Theorem 1.3 follows from the following lemma.

######  Lemma 4.3.

Let $g(p)=1+\sumop\displaylimits_{i\geq 1}a_{i}p^{i}$ be a power series
starting with $1$, let $f(p)$ be a power series. Fix $w,k\in\mathbb Q$. Put

$h(p):=\sumop\displaylimits_{n\geq
0}p^{n}\mathrm{Coeff}_{p^{n}}\big{[}g(p)^{w-kn}f(p)\big{]}.$

Then

$h(p)=\frac{g(z)^{w}f(z)}{1+kz\frac{d}{dz}\log(g(z))}\quad\hbox{where
}p=zg(z)^{k}.$

###### Proof.

Without loss of generality we can assume that $w=0$ (otherwise replace $f(p)$
by $g(p)^{w}f(p)$), and $k=1$ (otherwise replace $g(p)$ by $g(p)^{k}$ and note
that $z\frac{d}{dz}\log(g(z)^{k})=kz\frac{d}{dz}\log(g(p))$). We can describe
$h(p)$ as follows: for a variable $u$ write
$g(p)^{-u}f(p)=\sumop\displaylimits_{n\geq 0}h_{n}(u)p^{n}$, with $h_{n}(u)$ a
polynomial in $u$, then $h(p)=\sumop\displaylimits_{n\geq
0}h_{n}(p\frac{d}{dp})p^{n}$, i.e. move all factors of $u$ to the left and
then replace $u$ by $p\frac{d}{dp}$. We make the variable transformation
$p=e^{x}$, so that $p\frac{p}{dp}=\frac{d}{dx}$, and we write $g(p)=e^{a(x)}$,
$f(p)=\phi(x)$. Then we obtain

$\displaystyle h(p)$ $\displaystyle=\sumop\displaylimits_{n\geq
0}\frac{(-1)^{n}}{n!}\frac{d^{n}}{dx^{n}}\big{(}a(x)^{n}\phi(x)\big{)}$
$\displaystyle=\frac{\phi(\eta)}{1+a^{\prime}(\eta)}\Big{|}_{\eta=x-a(\eta)}$
$\displaystyle=\frac{f(z)}{1+z\frac{d}{dz}\log g(z)}\Big{|}_{p=zg(z)}.$

In the second line we have used the Lagrange inversion formula

$\sumop\displaylimits_{n=0}^{\infty}\frac{1}{n!}\frac{d^{n}}{dx^{n}}\big{(}a(x)^{n}f(x)\big{)}=\frac{f(z)}{1-a^{\prime}(z)}\Big{|}_{z=x+a(z)}.$

In the third line we put $z:=e^{\eta}$, thus $p=e^{x}=e^{\eta+a(\eta)}=zg(z)$.
∎

## 5\. Moduli of sheaves on K3 surfaces

In this section we extend our results to moduli spaces of sheaves on K3
surfaces. First we briefly recall determinant line bundles on moduli spaces of
sheaves, for details see e.g. [GNY09, Section 1.1], [HL10, Chapter 8].

For a Noetherian scheme $Y$ denote by $K(Y)$ and $K^{0}(Y)$ the Grothendieck
groups of coherent sheaves and locally free sheaves. If $Y$ is nonsingular and
projective, then $K(Y)=K^{0}(Y)$. We denote by $[\mathcal{F}]$ the class of a
sheaf $\mathcal{F}$ in $K(Y)$. For a proper morphism $f:Z\to Y$ the
pushforward $f_{!}:K(Z)\to K(Y)$ is defined by
$f_{!}([\mathcal{F}]):=\sumop\displaylimits_{i}(-1)^{i}[R^{i}f_{*}\mathcal{F}]$.
For any morphism $f:Z\to Y$ the pullback $f^{*}:K^{0}(Y)\to K^{0}(Z)$ is
defined by $f^{*}[\mathcal{F}]=[f^{*}\mathcal{\mathcal{F}}]$ for $\mathcal{F}$
a locally free sheaf on $Y$.

Now let $S$ be a smooth projective surface. On $K(S)$ there is a quadratic
form $(u,v)\mapsto\chi(S,u\otimes v)$ to be denoted by $\chi(u\otimes v)$.
Classes $u,v\in K(S)$ are called numerically equivalent if $u-v$ is in the
radical of this quadratic form. Let $K(S)_{\operatorname{num}}$ be the set of
numerical equivalence classes. Let $c\in K(S)_{\operatorname{num}}$. For a
flat family $\mathcal{E}$ of coherent sheaves on $S$ of class $c\in
K(S)_{\operatorname{num}}$ parametrized by a scheme $T$, let $q:S\times T\to
S$, $p:S\times T\to T$ be the projections and define
$\lambda_{\mathcal{E}}:K(S)\to\operatorname{Pic}(S)$ by the composition

$K(S)\xrightarrow{q^{*}}K^{0}(S\times
T)\xrightarrow{\otimes[\mathcal{E}]}K^{0}(S\times
T)\xrightarrow{p_{!}}K^{0}(T)\xrightarrow{\det^{-1}}\operatorname{Pic}(T).$

For $c\in K_{\operatorname{num}}(S)$ let $K_{c}:=\big{\\{}v\in
K(S)\bigm{|}\chi(v\otimes c)=0\big{\\}}$. Let $H\in\operatorname{Pic}(S)$ be
ample, denote $M^{H}_{S}(c)$ the moduli space of $H$-semistable sheaves of
class $c$. Assume that $H$ is general, i.e. if $\mathcal{F}\in M^{H}_{S}(c)$
is strictly $H$-semistable, then it is strictly semistable for all
$H^{\prime}$ in a neighbourhood of $H$. Then there exists a homomorphism
$\lambda:K_{c}\to\operatorname{Pic}(M^{H}_{S}(c))$, such that if $\mathcal{E}$
is a flat family of coherent sheaves on $S$ of class $c$ parametrized by $T$,
then $\phi_{\mathcal{E}}^{*}(\lambda(v))=\lambda_{\mathcal{E}}(v)$, where
$\phi_{\mathcal{E}}:T\to M^{H}_{S}(c)$ is the classifying morphim associated
to $\mathcal{E}$. For a class $v\in K(S)$ denote

$\operatorname{vd}(v):=2\operatorname{rk}(v)c_{2}(v)-(\operatorname{rk}(v)-1)c_{1}(v)^{2}-(\operatorname{rk}(v)^{2}-1)$

the expected dimension of $M^{H}_{S}(v)$. We obtain the following result.

######  Proposition 5.1.

Let $S$ be a K3 surface and $c\in K_{\operatorname{num}}(S)$, with $r(c)>0$ or
with $r(c)=0$ and $c_{1}(c)$ nef and big. Let $H$ ample on $S$ such that
$M^{H}_{S}(c)$ only consists of stable sheaves. Assume furthermore
$\operatorname{vd}(c)>1$. Then

$\displaystyle\operatorname{Ell}(M^{H}_{S}(c),\lambda(v))$
$\displaystyle=\mathrm{Coeff}_{p^{\frac{\operatorname{vd}(c)}{2}}}\left[{\big{(}\mathbf{L}^{(2,0)}(-\phi_{-2,1},p)\big{)}^{\frac{\operatorname{vd}(v)}{2}-1}\mathbf{L}(-2\phi_{0,1},p)}\right],$
$\displaystyle\overline{\chi}_{-y}(M^{H}_{S}(c),\lambda(v))$
$\displaystyle=\mathrm{Coeff}_{p^{\frac{\operatorname{vd}(c)}{2}}}\left[\frac{1}{\widetilde{\Delta}(p,y)}\prodop\displaylimits_{n>0}\left(\frac{(1-p^{n})^{2}}{(1-p^{n}y)(1-p^{n}/y)}\right)^{n^{2}(\frac{\operatorname{vd}(v)}{2}-1)}\right].$

###### Proof.

We adapt the arguments in [GNY09, Section 1.5]. The Mukai lattice of $S$ is
$H^{*}(S,\mathbb Z)$ with the symmetric bilinear form

$\langle
w,w^{\prime}\rangle=\intop\nolimits_{S}(c_{1}c_{1}^{\prime}-ra^{\prime}-r^{\prime}a),\quad
w=(r,c_{1},a),\ w^{\prime}=(r^{\prime},c_{1}^{\prime},a^{\prime}).$

Let $\phi:K(S)\to H^{*}(S,\mathbb Z)$ be the homomorphism defined by
$\phi(E)=\operatorname{ch}(E)\sqrt{\mathrm{td}(S)}.$ Let $w:=\phi(c)$, then
$\phi$ induces a injective homomorphism $\phi:K_{c}\to w^{\perp}$. There is a
homomorphism $\theta_{w}:w^{\perp}\to H^{2}(M_{S}^{H}(c),\mathbb Z)$, such
that $\theta_{w}(\phi(v))=c_{1}(\lambda(v))$ for all $v\in K_{c}$. We have
assumed that $\operatorname{rk}(c)>0$ or that $\operatorname{rk}(c)=0$ and
$c_{1}(c)$ is nef and big, furthermore $\operatorname{vd}(c)>1$, and
$M^{H}_{S}(c)$ consists only of stable sheaves. Under these assumptions we
know ([GNY09, Theorem 1.14], [Yos01a], [Yos01b]), that $M^{H}_{S}(c)$ is an
irreducible symplectic manifold which is deformation equivalent to
$S^{[\operatorname{vd}(c)/2]}$, $\theta_{w}:w^{\perp}\to
H^{2}(M_{S}^{H}(c),\mathbb Z)$ is surjective, and for $x$ in $w^{\perp}$ we
have $\langle x,x\rangle=q_{M^{H}_{S}(c)}(\theta_{w}(x))$.

Let $v\in K_{c}$. By Corollary 2.2 we have

$\operatorname{Ell}(M^{H}_{S}(c),\lambda(v))=\intop\nolimits_{M^{H}_{S}(c)}\operatorname{ELL}(M^{H}_{S}(c))\exp(c_{1}(\lambda(v)))=h_{M^{H}_{S}(c)}\big{(}q_{M^{H}_{S}(c)}(\theta_{w}(c_{1}(\lambda(v))))\big{)}.$

As $M^{H}_{S}(c)$ is deformation equivalent to $S^{[\operatorname{vd}(c)/2]}$,
we have $h_{M^{H}_{S}(c)}=h_{S^{[\operatorname{vd}(c)/2]}}.$ We compute

$q_{M^{H}_{S}(c)}\big{(}\theta_{w}(c_{1}(\lambda(v)))\big{)}=\langle\phi(v),\phi(v)\rangle=\operatorname{vd}(v)-2.$

Thus we get by Proposition 4.1 and (4.1),(4.2) that

$\operatorname{Ell}(M^{H}_{S}(c),\lambda(v))=h_{S^{[\operatorname{vd}(c)/2]}}(\operatorname{vd}(v)-2)=\mathrm{Coeff}_{p^{\frac{\operatorname{vd}(c)}{2}}}\left[{\mathbf{L}^{(2)}(-\phi_{-2,1},p)^{\frac{\operatorname{vd}(v)}{2}-1}\mathbf{L}(-2\phi_{0,1},p)}\right].$

∎

In order to express this formula in terms of generating functions we denote
the line bundles $\lambda(v)$ on different moduli spaces $M^{H}_{S}(c)$ in a
unified way, generalizing our notation on Hilbert schemes of points.

######  Notation 5.2.

For fixed $c\in K(S)_{\operatorname{num}}$ we write $s=\operatorname{rk}(c)$,
$c_{1}=c_{1}(c)$, $c_{2}=c_{2}(c)$. Let us denote by $\det(c)$ and
$\det(v)\in\operatorname{Pic}(S)$ the determinant line bundles. We assume that
$s>0$, and denote $\operatorname{vd}:=\operatorname{vd}(c)$. Let $r\in\mathbb
Z$. Let $L\in\operatorname{Pic}(S)\otimes\det(c)^{r/s}$, and denote
$M:=L\otimes\det(c)^{-r/s}\in\operatorname{Pic}(S)$. If $s$ divides
$Mc_{1}+r(\frac{c_{1}^{2}}{2}-c_{2})$, we define $\mu(L)\otimes
E^{r}\in\operatorname{Pic}(M^{H}_{S}(c))\in\operatorname{Pic}(M^{H}_{S}(c))$
by $\mu(L)\otimes E^{r}:=\lambda(v)$ for $v\in K(S)$ with

(5.1) $\operatorname{rk}(v)=r,\quad c_{1}(v)=M,\quad
c_{2}(v)=\frac{M^{2}}{2}+2r+\frac{Mc_{1}}{s}+\frac{r}{s}\big{(}\frac{c_{1}^{2}}{2}-c_{2}\big{)}.$

Note that the condition on $c_{2}(v)$ is equivalent to $\chi(S,c\otimes v)=0$
i.e. to $v\in K_{c}$, so that $\mu(L)\otimes E^{r}$ is well-defined.

######  Remark 5.3.

  1. (1)

When $r=0$ this definition coincides with the definition of the Donaldson line
bundle $\mu(L)$ (see e.g. [GNY09], [GKW]).

  2. (2)

When $s=1$ the definition specializes to that of $\mu(L)\otimes E^{r}$ on
$S^{[n]}$ under the identification $S^{[n]}=M^{H}_{S}(1,c_{1},n)$, for any
first Chern class $c_{1}$.

  3. (3)

If all sheaves in $M^{H}_{S}(c)$ are slope stable, then twisting by a line
bundle $A$ gives an isomorphism $\phi_{A}:M^{H}_{S}(c)\to M^{H}_{S}(c\otimes
A)$, and it is easy to see that $\phi_{A}^{*}(\mu(L)\otimes
E^{r})=\mu(L)\otimes E^{r}.$

Now we prove Theorem 1.5.

###### Proof of Theorem 1.5.

Fix $c$ with $\operatorname{rk}(c)=s>0$, $c_{1}(c)=c_{1}$, $c_{2}(c)=c_{2}$,
fix $r\in\mathbb Z$, let $L\in\operatorname{Pic}(S)\otimes\det(c)^{r/s}$,
denote by $M:=L\otimes\det(c)^{-r/s}$ the corresponding element in
$\operatorname{Pic}(S)$, and assume that $s$ divides
$Mc_{1}+r(\frac{c_{1}^{2}}{2}-c_{2})$. Let $v\in K_{c}$ fullfill (5.1), so
that $\mu(L)\otimes E^{r}=\lambda(v)$. Plugging the relations

$c_{1}(v)=c_{1}(L)-\frac{r}{s}c_{1},\quad
c_{2}(v)=\frac{c_{1}(v)^{2}}{2}+2r+\frac{c_{1}(v)c_{1}}{s}+\frac{r}{s}\big{(}\frac{c_{1}^{2}}{2}-c_{2}\big{)}\quad\mathrm{and}\quad
c_{2}=\frac{1}{2s}\big{(}\operatorname{vd}(c)+(s-1)c_{1}^{2}+2(s^{2}-1)\big{)}$

into the formula

$\operatorname{vd}(v)=2rc_{2}(v)-(r-1)c_{1}(v)^{2}-2(r^{2}-1)$

gives by direct computation

$\operatorname{vd}(v)=L^{2}+2-\frac{r^{2}}{s^{2}}(\operatorname{vd}(c)-2).$

Thus Proposition 5.1 gives

$\displaystyle\operatorname{Ell}(M^{H}_{S}(c),\mu(L)\otimes E^{r})$
$\displaystyle=\mathrm{Coeff}_{p^{\operatorname{vd}(c)/2}}\left[\big{(}\mathbf{L}^{(2,0)}(-\phi_{-2,1},p)\big{)}^{\frac{L^{2}}{2}-\frac{r^{2}}{s^{2}}(\operatorname{vd}(c)/2-1)}\mathbf{L}(-2\phi_{0,1},p)\right]$

Applying Lemma 4.3 gives Theorem 1.5. ∎

## References

  * [Bea83] A. Beauville, Variétés kähleriennes dont la premiere classe de Chern est nulle, J. Diff.Geom. 18 (1983), 755–782.
  * [BL94] A. Beauville and Y. Laszlo, Conformal blocks and generalized theta functions, Comm. Math. Phys. 164 (1994), 385–419.
  * [BL00] L. A. Borisov and A. Libgober, Elliptic genera of toric varieties and applications to mirror symmetry, Invent. Math. 140 (2000), 453–485.
  * [BL03] by same author, Elliptic genera of singular varieties, Duke Math. Jour. 116 (2003), 319–351.
  * [BL05] by same author, McKay correspondence for elliptic genera, Annals of Math. 161 (2005), 1521–1569.
  * [Bor95] R. E. Borcherds, Automorphic forms on $O_{s+2,2}(\mathbb R)$ and infinite products, Invent. Math. 120 (1995), 161–213.
  * [DMVV97] R. Dijkgraaf, G. Moore, E. Verlinde and H. Verlinde, Elliptic genera of symmetric products and second quantized strings, Commun. Math. Phys. 185 (1997), 197–209.
  * [EGL01] G. Ellingsrud, L. Göttsche, and M. Lehn, On the cobordism class of the Hilbert scheme of a surface, Jour. Alg. Geom. 10 (2001), 81–100.
  * [Fal94] G. Faltings, A proof of the Verlinde formula, J. Alg. Geom. 3 (1994), 347–374.
  * [GK] L. Göttsche and M. Kool, A rank 2 Dijkgraaf-Moore-Verlinde-Verlinde formula, Commun. Number Theory Phys. 13 (2019), No. 1, 165–201.
  * [GKW] L. Göttsche, M. Kool and Runako Williams, Verlinde formulas on complex surfaces I: K-theoretic invariants, preprint arXiv:1903.03869 (2019).
  * [GNY09] L. Göttsche, H. Nakajima, and K. Yoshioka, K-theoretic Donaldson invariants via instanton counting, Pure Appl. Math. Q. 5 (2009), 1029–1111.
  * [GS93] L. Göttsche, W. Soergel, Perverse sheaves and the cohomology of Hilbert schemes of smooth algebraic surfaces, Math. Ann. 296 (1993), 235–245.
  * [Huy99] D. Huybrechts, Compact hyper-Kähler manifolds: basic results, Invent. Math. 135 (1999), 63–113.
  * [HL10] D. Huybrechts, M. Lehn, The geometry of moduli spaces of sheaves, Cambridge University Press (2010).
  * [NR93] M. S. Narasimhan and T. R. Ramadas, Factorisation of generalised theta functions. I, Invent. Math. 114 (1993), 565–623.
  * [Ver88] E. Verlinde, Fusion rules and modular transformations in $2d$ conformal field theory, Nucl. Phys., B 300 (1988), 360–376.
  * [Wit88] E. Witten, The index of the Dirac operator in loop space, in: Elliptic curves and modular forms in algebraic geometry, P. Landweber, ed., Springer-Verlag (1988), 161–181.
  * [Yos01a] K. Yoshioka, Moduli spaces of stable sheaves on abelian surfaces, Math. Ann. 321 (2001), 817–884.
  * [Yos01b] K. Yoshioka, A note on Fourier-Mukai transform, preprint arXiv:mathAG/0112267 (2001).


# Can LLM Substitute Human Labeling?
A Case Study of Fine-grained Chinese Address Entity Recognition Dataset for
UAV Delivery

Yuxuan Yao$\ast$, Sichun Luo$\ast$, Haohan Zhao$\ast$, Guanzhi Deng, Linqi
Song† Department of Computer Science, City University of Hong KongHong
KongChina City University of Hong Kong Shenzhen Research
InstituteShenzhenChina yuxuanyao3-c, sichun.luo, haohazhao2-c,
<EMAIL_ADDRESS><EMAIL_ADDRESS>
(2018; 2024)

###### Abstract.

We present CNER-UAV, a fine-grained Chinese Name Entity Recognition dataset
specifically designed for the task of address resolution in Unmanned Aerial
Vehicle delivery systems. The dataset encompasses a diverse range of five
categories, enabling comprehensive training and evaluation of NER models. To
construct this dataset, we sourced the data from a real-world UAV delivery
system and conducted a rigorous data cleaning and desensitization process to
ensure privacy and data integrity. The resulting dataset, consisting of around
12,000 annotated samples, underwent human experts and Large Language Model
annotation. We evaluated classical NER models on our dataset and provided in-
depth analysis. The dataset and models are publicly available at
https://github.com/zhhvvv/CNER-UAV.

$\ast$Equal Contribution
†Coresponding Author

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††journalyear: 2024††copyright: acmlicensed††conference:
Companion Proceedings of the ACM Web Conference 2024; May 13–17, 2024;
Singapore, Singapore††booktitle: Companion Proceedings of the ACM Web
Conference 2024 (WWW ’24 Companion), May 13–17, 2024, Singapore,
Singapore††doi: 10.1145/3589335.3651446††isbn: 979-8-4007-0172-6/24/05††ccs:
Computing methodologies Language resources††ccs: Computing methodologies
Information extraction

##  1\. Introduction

Named Entity Recognition (NER) categorizes entities in text based on
predefined categories (Li et al., 2020). Traditional grammar-based NER offers
precision but requires significant effort from linguists. Statistical NER
systems demand extensive manual annotation. Transformer models, like BERT
(Devlin et al., 2018), now dominate NER tasks (Vaswani et al., 2017; Liu et
al., 2022a), showcasing remarkable performance.

The unmanned aerial vehicle (UAV) delivery system in China employs drones for
goods delivery (Meituan, 2021). In this system, the address resolution module
utilizes NER to convert raw user addresses into precise locations using
language models (Liu et al., 2022a). However, these models require specific
fine-grained Chinese NER datasets for training, and there’s a shortage of such
datasets for Chinese, highly necessitating the creation of a new dataset.

In recent years, large language models (LLMs) have shown remarkable natural
language understanding and generation abilities (Brown et al., 2020; Luo et
al., 2024; Zhou et al., 2023; Luo et al., 2023; Wang et al., 2023). While some
studies have successfully used LLMs for labeling English datasets (Gilardi et
al., 2023; Wang et al., 2021), research on their application in labeling fine-
grained Chinese datasets is limited. This raises a natural question: _Can LLMs
surpass human performance in labeling fine-grained Chinese datasets with
superior quality?_

In this paper, we present CNER-UAV, a fine-grained Chinese NER dataset for the
UAV delivery task. Specifically, CNER-UAV consists of five categories, ranging
from buildings to rooms, allowing for fine-grained segmentation of Chinese
addresses. It contains around 12,000 labeled samples. The raw data were
collected from Meituan UAV delivery systems (Meituan, 2021), representing the
actual data distribution. After carefully processing the raw data, we
annotated the dataset using both Human annotators and LLMs, including GPT-3.5
(Brown et al., 2020) and ChatGLM (Zeng et al., 2022), resulting in 3 subsets:
CNER-UAV-H, CNER-UAV-G, and CNER-UAV-L correspondingly. We conducted
experiments on all subsets to evaluate their performance with various baseline
models. By comparing these datasets, we have determined that while LLMs can
serve as supplementary annotation tools, they cannot entirely substitute
manual annotation for fine-grained Chinese NER in the UAV delivery task.

In a nutshell, our contribution is threefold:

  * •

We introduce CNER-UAV, a novel and fine-grained Chinese NER dataset
specifically designed for the UAV delivery task. To the best of our knowledge,
this is the most up-to-date and comprehensive address dataset from China for
named entity recognition in UAV delivery services.

  * •

We compare the human annotation with the LLM annotation and conduct empirical
studies demonstrating that humans outperform GPT-3.5 and ChatGLM in labeling
the Chinese NER dataset, particularly for more fine-grained tags.

  * •

We evaluate several state-of-the-art baseline models and establish a benchmark
for further research on this task. The evaluation results validate the
usefulness of our proposed CNER-UAV dataset.

##  2\. Related Work

###  2.1. Chinese Named Entity Recognition Dataset

Named Entity Recognition (NER) has been extensively studied in Natural
Language Processing (NLP), and Chinese NER (CNER) datasets are pivotal for
Chinese-specific NER models. The People’s Daily (PD)
corpus111https://icl.pku.edu.cn/ and larger-scale datasets like OntoNotes
4.0222https://catalog.ldc.upenn.edu/LDC2011T13 offer annotations, but
challenges like domain coverage and noisy data remain. Datasets like WeiboNER
(Peng and Dredze, 2015) address microblog NER, but the need persists for
diverse, high-quality Chinese NER datasets catering to the language’s
uniqueness.

###  2.2. Labeling Dataset with LLMs

Labeling datasets with LLMs has gained recent attention for automating dataset
creation and annotation, leading to cost savings and reduced effort (Wang et
al., 2021). Collaborative approaches between humans and LLMs have also been
explored, as seen in the WANLI dataset (Liu et al., 2022b), and GPT-3.5’s
ability to outperform crowd-workers in text annotation tasks (Gilardi et al.,
2023). While LLMs show promise in improving annotation quality and cost-
efficiency, limited research explores their use in labeling fine-grained
Chinese NER datasets. This study compensates for this research gap by focusing
on LLMs’ application in fine-grained Chinese address recognition dataset
labeling.

Figure 1. The pipeline of dataset construction.

##  3\. Dataset Construction

###  3.1. Overview

Our dataset compilation process primarily adheres to the flowchart illustrated
in Figure 1. Initially, we parse the historical order logs to extract the
user’s order information. Subsequently, we identify and extract the addresses,
applying predefined rules to filter out the relevant ones. Next, we proceed to
sample and annotate the data, which involves labeling by human annotators,
GPT-3.5 and ChatGLM. Further elaboration on the dataset collection and
annotation procedures can be found in the subsequent sections.

###  3.2. Dataset Collection

The data originates from UAV delivery addresses provided by
Meituan333https://www.meituan.com/, which is one of the largest delivery
service providers in China. When utilizing Meituan’s online application, users
can obtain services based on their personal preferences or practical address
coverage through the following procedures: 1) Selecting a location point,
which corresponds to the Point of Interest (POI) database, containing
longitude and latitude coordinates. 2) Manually typing specific address
details as auxiliary support. Our dataset is mainly sourced from users’
textual descriptions in Shenzhen City and thus is a Chinese dataset,
consisting of 12,000 pieces of data in each subset.

###  3.3. Dataset Annotation

To facilitate a more effective comparison between the LLMs and manual
annotations, we established the following criteria for both manual and LLMs
annotations:

1\. Respect User Input and POI Information: Annotate all entities when
multiple references point to a single location.

2\. Priority to Highest Probability Entity: When multiple entities of the same
category appear in the address text, select the entity with the highest
probability as the user’s destination.

3\. Filter Ambiguous or Unlocatable Addresses: Exclude addresses with
ambiguous descriptions or those that cannot be geolocated based on longitude
and latitude.

Specifically, the annotation prompt template for GPT-3.5 and ChatGLM is
designed as:
Extract information from the address by referring to the examples, extract
POI, building, unit, level, and room information, output them in JSON format.
Example: {examples}
Instruction: {instructions}

Table 1. Count of sentences and tags in the dataset.

Dataset Sentence Count Tag Count(GPT) Tag Count (Manual) Tag Count(GLM) Train
10150 58778 53220 73566 Test 1269 7396 6731 9211 Validation 1269 7466 6690
9294 Overall 12688 73640 66641 73566

Table 2. CNER-UAV Dataset Description.

| Tags | Train | Test | Validation
---|---|---|---|---
CNER-UAV-L | Building | 11403 | 1441 | 1422
Unit | 6542 | 816 | 828
Level | 3489 | 447 | 441
Room | 3209 | 422 | 413
Others | 108324 | 13585 | 13962
CNER-UAV-G | Building | 7513 | 960 | 972
Unit | 1484 | 215 | 190
Level | 8155 | 1019 | 1019
Room | 8280 | 1018 | 1029
Others | 124153 | 15538 | 15869
CNER-UAV-H | Building | 8125 | 1037 | 1045
Unit | 1367 | 184 | 170
Level | 8642 | 1079 | 1070
Room | 7522 | 928 | 931
Others | 140635 | 17560 | 18003

Table 3. Sample Tagged Sentences, where ‘Others’ is represented as ‘O’,
‘Building’ as ’B’, ‘Level’ as ‘L’ and ‘Room’ as ‘R’

Sentence English Translation Tag Wanjia Lighting - Building 2 - Unit 1 (803) O
O O O O B-B I-B O B-U I-U I-U O B-L B-R I-R O Building 5, Nanqi Lane, Nankeng
(Building 1, Nanqi Lane, Nankeng) O O O O O O B-B I-B I-B O O O O O O B-B I-B
I-B O Lemon·Shuixie Spring Phase 3 (Building 10 18h) O O O O O O O O O O B-B
I-B B-L I-L B-R O

where examples are annotated addresses based on the BOI standard, and
instructions encompass Chinese language conventions and practical application
scenarios, serving as both restrictions and suggestions.

Furthermore, considering Chinese language patterns, we devised regularization
formulas to correlate the annotated addresses with Beginning of Inside (BOI)
(Lample et al., 2016) labels, forming the annotated dataset.

###  3.4. Dataset Statistics

Table 1 presents an overview of sentences and tags within our datasets. Table
2 shows the statistics of different labels in 3 subsets. Buildings are labeled
as ‘Building’, units as ‘Unit’, floors as ‘Level’, rooms as ‘Room’, and non-
entities are denoted as ‘Others’. We could observe from the table, in the
’Building’ classification, GPT annotations are approximately 7.47% less
compared to manual labeling. Conversely, in the ’Unit’ category, GPT’s
annotations exceeded human labeling by about 9.76%. These disparities suggest
a potential oversight in specific ’Unit’ labels by GPT, possibly arising from
a deficiency in relevant knowledge or an inherent limitation within the model.

For more clarity, some example sentences with tagged entities are mentioned in
Table 3.

Table 4. F1 score and accuracy of various transformer and normal models for BOI notation using the UAV-NER dataset, where Acc means accuracy, G indicates dataset annotated by GPT-3.5 and H represents manually labeled dataset. | BERT | ELECTRA | XLNET | RoBERTa | ERNIE | LERT | PERT
---|---|---|---|---|---|---|---
Acc $\uparrow$ | F1 $\uparrow$ | Acc $\uparrow$ | F1 $\uparrow$ | Acc $\uparrow$ | F1 $\uparrow$ | Acc $\uparrow$ | F1 $\uparrow$ | Acc $\uparrow$ | F1 $\uparrow$ | Acc $\uparrow$ | F1 $\uparrow$ | Acc $\uparrow$ | F1 $\uparrow$
Building | L | 70.69 | 82.73 | 66.98 | 80.15 | 69.82 | 82.13 | 70.76 | 82.77 | 70.01 | 82.77 | 70.92 | 82.89 | 72.42 | 83.86
G | 95.19 | 96.13 | 95.04 | 96.02 | 94.88 | 95.87 | 95.74 | 96.59 | 95.19 | 96.14 | 95.67 | 96.53 | 96.38 | 97.11
H | 95.74 | 96.89 | 94.88 | 96.23 | 94.48 | 95.95 | 95.67 | 96.83 | 94.96 | 96.29 | 96.30 | 97.30 | 96.22 | 97.25
Unit | L | 60.20 | 63.70 | 57.13 | 61.42 | 61.78 | 65.48 | 59.02 | 63.33 | 59.03 | 63.41 | 61.39 | 64.75 | 61.31 | 65.25
G | 94.09 | 76.64 | 94.01 | 76.97 | 94.01 | 76.54 | 93.38 | 74.85 | 93.70 | 74.85 | 94.09 | 77.48 | 94.41 | 78.02
H | 98.27 | 92.99 | 98.11 | 92.26 | 98.11 | 92.26 | 98.50 | 93.97 | 97.95 | 91.72 | 98.42 | 93.63 | 98.66 | 94.60
Level | L | 71.32 | 51.34 | 67.77 | 46.68 | 69.98 | 51.34 | 68.95 | 50.38 | 68.88 | 50.31 | 70.45 | 51.98 | 72.34 | 53.14
G | 91.10 | 94.45 | 90.86 | 94.35 | 90.54 | 94.13 | 90.94 | 94.35 | 90.94 | 94.35 | 89.83 | 93.65 | 90.31 | 93.97
H | 96.53 | 97.93 | 95.82 | 97.50 | 95.51 | 97.31 | 95.82 | 97.51 | 96.22 | 97.74 | 96.30 | 97.79 | 97.08 | 98.27
Room | L | 68.64 | 41.82 | 65.41 | 35.35 | 67.38 | 41.85 | 66.75 | 41.55 | 66.76 | 41.58 | 68.24 | 42.35 | 69.74 | 41.82
G | 79.91 | 86.56 | 78.88 | 86.14 | 79.67 | 86.78 | 79.59 | 86.49 | 79.04 | 86.49 | 79.28 | 86.21 | 79.20 | 86.22
H | 94.96 | 96.52 | 92.99 | 95.14 | 92.51 | 94.82 | 94.48 | 96.18 | 94.48 | 96.19 | 94.33 | 96.08 | 95.90 | 97.18
Others | L | 54.14 | 70.21 | 51.77 | 68.22 | 53.59 | 69.78 | 52.17 | 68.57 | 52.21 | 68.6 | 53.27 | 69.51 | 54.85 | 70.81
G | 78.17 | 87.75 | 78.88 | 88.19 | 77.86 | 87.05 | 79.35 | 88.49 | 79.12 | 88.49 | 78.17 | 87.75 | 79.35 | 88.49
H | 91.80 | 95.73 | 90.62 | 95.08 | 90.15 | 94.82 | 91.57 | 95.60 | 91.10 | 95.34 | 92.28 | 95.98 | 93.14 | 96.45
Overall @macro | L | 65.00 | 61.96 | 61.81 | 58.36 | 64.51 | 62.12 | 63.53 | 61.32 | 63.38 | 61.33 | 64.85 | 62.3 | 66.13 | 62.98
G | 87.69 | 88.31 | 87.53 | 88.33 | 87.39 | 88.07 | 87.80 | 88.15 | 87.60 | 88.06 | 87.41 | 88.32 | 87.93 | 88.76
H | 95.46 | 96.01 | 94.48 | 95.24 | 94.15 | 95.03 | 95.21 | 96.02 | 94.94 | 95.46 | 95.53 | 96.16 | 96.23 | 96.75

##  4\. Experiment

###  4.1. Experiment Setup

Baselines Transformer-based models have found success across multiple NLP
tasks, including named entity recognition, eliminating the need for manual
feature engineering. To evaluate the dataset, we implement the following
baselines: BERT (Devlin et al., 2018), ELECTRA (Clark et al., 2020), RoBERTa
(Liu et al., 2019), XLNET (Yang et al., 2019), ERNIE (Zhang et al., 2019),LERT
(Cui et al., 2022a), and PERT (Cui et al., 2022b).

Evaluation Metrics In line with (Litake et al., 2022), we evaluated various
NER models using standard performance metrics, including Accuracy and
F1-score. We assessed the performance at both the class level and overall
performance.

###  4.2. Results and Discussion

The experimental results are presented in Table 4. The table reveals a notable
disparity in annotation effectiveness, with ChatGLM-6B performing
significantly worse compared to other annotation methods. In essence, it falls
short of meeting the practical requirements of the UAV system. Our analysis
suggests that this performance discrepancy can be attributed to ChatGLM’s
primary training data, which mainly comprises Chinese conversations. Without
fine-tuning specifically for NER annotation tasks, its applicability to
production needs remains inadequate.

We’ve noticed that GPT annotation performs competitively with manual
annotation when labeling “Buildings.” However, it significantly lags behind in
accurately labeling “Rooms” and “Others” compared to human annotation. This
underscores the limitations of GPT annotation and highlights the challenges
GPT models encounter in achieving fine-grained understanding of Chinese text.
A comprehensive dataset review reveals several key insights. Building
information is relatively straightforward and often presented in formats like
“No. 2 Building.” However, variations abound in the descriptions of units or
levels due to diverse developer naming preferences and users’ desire for food
delivery convenience. Users may forego conventional terms like “3rd floor” in
favor of more familiar references such as “the floor where Star Bookstore is
located.” Consequently, GPT-3.5’s annotation effectiveness noticeably
diminishes when handling such diverse data.

Furthermore, it is worth noting that all language models achieve over 90%
accuracy in each category of the human-annotated dataset. This result
demonstrates the effectiveness and utility of our proposed dataset, affirming
its value for UAV delivery task.

Figure 2. Bad Cases of CNER-UAV-G and CNER-UAV-L datasets, where labels from
CNER-UAV-G dataset are shown as G-tag, CNER-UAV-L as L-tag,CNER-UAV-H as H-tag

###  4.3. Case Study

The experimental findings suggest that currently, large language models cannot
fully substitute manual annotation. Figure 2 lists four distinct types of bad
cases in language models’ address tagging:

1\. Label Ambiguity: In cases where users input a combination of building and
floor details, for instance, “Cheuk Neng Court(5B12D)”, GPT-3.5 cannot extract
the ‘5B’ as B-tag, ‘12’ as L-tag, and ‘D’ as R-tag, while ChatGLM cannot
separate the floor and room labels.

2\. Incomplete Entity Labeling: LLMs sometimes provide incomplete entity
labeling. For example, in the input ‘6th floor,’ GPT-3.5 and ChatGLM only
label ‘6th’ and overlook ‘floor’ in Chinese.

3\. Consecutive Number Confusion: When consecutive numbers are entered, LLMs
face challenges in correctly distinguishing between floor and room numbers.
For instance, they may fail to recognize the first “2” in “223” as a “Level”
tag. In contrast, human annotation effortlessly avoids such issues.

4\. Incapacity for Long Text: Compared to GPT-3.5, ChatGLM exhibits a
significant decline in its ability to parse Chinese long-text addresses,
exhibiting severe label confusion. This aligns with our experimental findings.

To summarize, LLMs excel with singular and standardized address information
but fall short in handling semantically rich and syntactically flexible
addresses. Enhancing LLM’s Chinese comprehension capabilities is a promising
research avenue.

##  5\. Ethics Information

The CNER-UAV dataset was collected in the first half of 2023. The task of data
annotation was outsourced to an external company. The data collection process
was approved by the Meituan Ethics Review Board, ensuring that the process
adhered to ethical guidelines. Meituan is responsible for data desensitization
and the deletion of user-sensitive information, ensuring that the privacy and
confidentiality of the users are well-protected.

##  6\. Conclusion

We present a fine-grained Chinese NER Dataset for the UAV delivery system,
referred to as CNER-UAV. The dataset consists of three subsets, but for
practical applications, we are releasing two of them, one labeled by human
annotators and the other labeled by GPT-3.5. These subsets are derived from
real-world system logs, reliably reflecting user behavior. We perform
experiments using popular baseline methods on both subsets. Empirical results
demonstrate that human annotators outperform GPT-3.5 in labeling the Fine-
Grained Chinese NER Dataset for the UAV delivery system.

###### Acknowledgements.

This work was supported in part by National Natural Science Foundation of
China under Grant 62371411, the Research Grants Council of the Hong Kong SAR
under Grant GRF 11217823, InnoHK initiative, the Government of the HKSAR,
Laboratory for AI-Powered Financial Technologies, the Meituan Robotics
Research Institute.

## References

  * (1)
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020\.  Language models are few-shot learners.  _Advances in neural information processing systems_ 33 (2020), 1877–1901.
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020.  Electra: Pre-training text encoders as discriminators rather than generators.  _arXiv preprint arXiv:2003.10555_ (2020).
  * Cui et al. (2022a) Yiming Cui, Wanxiang Che, Shijin Wang, and Ting Liu. 2022a.  LERT: A Linguistically-motivated Pre-trained Language Model.  arXiv:2211.05344 [cs.CL]
  * Cui et al. (2022b) Yiming Cui, Ziqing Yang, and Ting Liu. 2022b.  PERT: Pre-training BERT with Permuted Language Model.  arXiv:2203.06906 [cs.CL]
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).
  * Gilardi et al. (2023) Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. 2023.  Chatgpt outperforms crowd-workers for text-annotation tasks.  _arXiv preprint arXiv:2303.15056_ (2023).
  * Lample et al. (2016) Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016.  Neural architectures for named entity recognition.  _arXiv preprint arXiv:1603.01360_ (2016).
  * Li et al. (2020) Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. 2020.  A survey on deep learning for named entity recognition.  _IEEE Transactions on Knowledge and Data Engineering_ 34, 1 (2020), 50–70.
  * Litake et al. (2022) Onkar Litake, Maithili Ravindra Sabane, Parth Sachin Patil, Aparna Abhijeet Ranade, and Raviraj Joshi. 2022.  L3cube-mahaner: A marathi named entity recognition dataset and bert models. In _Proceedings of the WILDRE-6 Workshop within the 13th Language Resources and Evaluation Conference_. 29–34.
  * Liu et al. (2022b) Alisa Liu, Swabha Swayamdipta, Noah A Smith, and Yejin Choi. 2022b.  Wanli: Worker and ai collaboration for natural language inference dataset creation.  _arXiv preprint arXiv:2201.05955_ (2022).
  * Liu et al. (2022a) Pan Liu, Yanming Guo, Fenglei Wang, and Guohui Li. 2022a.  Chinese named entity recognition: The state of the art.  _Neurocomputing_ 473 (2022), 37–53.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ (2019).
  * Luo et al. (2023) Sichun Luo, Bowei He, Haohan Zhao, Yinya Huang, Aojun Zhou, Zongpeng Li, Yuanzhang Xiao, Mingjie Zhan, and Linqi Song. 2023.  RecRanker: Instruction Tuning Large Language Model as Ranker for Top-k Recommendation.  _arXiv preprint arXiv:2312.16018_ (2023).
  * Luo et al. (2024) Sichun Luo, Yuxuan Yao, Bowei He, Yinya Huang, Aojun Zhou, Xinyi Zhang, Yuanzhang Xiao, Mingjie Zhan, and Linqi Song. 2024.  Integrating Large Language Models into Recommendation via Mutual Augmentation and Adaptive Aggregation.  _arXiv preprint arXiv:2401.13870_ (2024).
  * Meituan (2021) Meituan. 2021.  _Food Delivery Giant Meituan Unveils Drones for Delivery Service, Offering New User Experience_.  https://pandaily.com/food-delivery-giant-meituan-unveils-drones-for-delivery-service/
  * Peng and Dredze (2015) Nanyun Peng and Mark Dredze. 2015.  Named entity recognition for chinese social media with jointly trained embeddings. In _Proceedings of the 2015 conference on empirical methods in natural language processing_. 548–554.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  _Advances in neural information processing systems_ 30 (2017).
  * Wang et al. (2023) Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. 2023.  Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning.  _arXiv preprint arXiv:2310.03731_ (2023).
  * Wang et al. (2021) Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, and Michael Zeng. 2021.  Want to reduce labeling cost? GPT-3 can help.  _arXiv preprint arXiv:2108.13487_ (2021).
  * Yang et al. (2019) Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019.  Xlnet: Generalized autoregressive pretraining for language understanding.  _Advances in neural information processing systems_ 32 (2019).
  * Zeng et al. (2022) Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022\.  Glm-130b: An open bilingual pre-trained model.  _arXiv preprint arXiv:2210.02414_ (2022).
  * Zhang et al. (2019) Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun, and Qun Liu. 2019.  ERNIE: Enhanced language representation with informative entities.  _arXiv preprint arXiv:1905.07129_ (2019).
  * Zhou et al. (2023) Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, et al. 2023\.  Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification.  _arXiv preprint arXiv:2308.07921_ (2023).