# Valuation rings of mixed characteristic as limits of complete intersection
rings

Dorin Popescu  Simion Stoilow Institute of Mathematics of the Romanian
Academy, Research unit 5, University of Bucharest, P.O. Box 1-764, Bucharest
014700, Romania, Email<EMAIL_ADDRESS>
###### Abstract.

We show that a mixed characteristic valuation ring with a value group $\Gamma$
and a residue field $k$ of characteristic $p>0$, is a filtered direct limit of
complete intersection $\bf Z$-algebras if $\Gamma/{\bf Z}\mathrm{val}(p)$ has
no torsion, $k$ is separably generated over $F_{p}$ and either $V$ is
Henselian, or $\dim V=1$ .

Key words  : Valuation Rings, Immediate Extensions, Complete Intersection
Algebras, Cross-sections, Ultrapowers, Henselian Rings

2020 Mathematics Subject Classification: Primary 13F30, Secondary 13A18,
13L05, 13B40.

## Introduction

In [7] we gave a different proof to a weak form of Zariski’s Uniformization
Theorem [11] using the following result (see [7, Theorem 21]).

######  Theorem 1.

Let $V\subset V^{\prime}$ be an immediate extension of valuation rings
containing $\bf Q$. Then $V^{\prime}$ is a filtered direct limit of smooth
$V$-algebras.

If the characteristic of the residue field of $V$ is $>0$ then this is not
true as shows for example [6, Example 3.13] (see also [6, Remark 6.10])
inspired by [5, Sect 9, No 57] and so in this case it is useful to have the
following result [8, Theorem 14].

######  Theorem 2.

Let $V^{\prime}$ be an immediate extension of a valuation ring $V$. Then
$V^{\prime}$ is a filtered union of its complete intersection $V$-subalgebras.

A complete intersection $V$-algebra is a local $V$-algebra of type $C/(P)$,
where $C$ is a localization of a polynomial $V$-algebra and $P$ is a regular
system of elements of $C$. Thus the above theorem says that $V^{\prime}$ is a
filtered union of its $V$-subalgebras of type $C/(P)$. Since $V^{\prime}$ is
local it is enough to say that $V^{\prime}$ is a filtered union of its
$V$-subalgebras of type $T_{h}/(P)$, $T$ being a polynomial $V$-algebra,
$0\not=h\in T$ and $P$ is a regular system of elements of $T$. Clearly,
$T_{h}$ is a smooth $V$-algebra and in fact it is enough to say that
$V^{\prime}$ is a filtered union of its $V$-subalgebras of type $G/(P)$, where
$G$ is a smooth $V$-algebra and $P$ is a regular system of elements of $G$.
Conversely, a $V$-algebra of such type $G/(P)$ has the form $T_{h}/(P)$ for
some $T,h,P$ using [9, Theorem 2.5]. By abuse we understand by a complete
intersection  $V$-algebra a $V$-algebra of such type $G/(P)$, or $T_{h}/(P)$.
More general, we understand by a complete intersection  algebra over a ring
$A$ an $A$-algebra of type $G/(P)$, or $T_{h}/(P)$, where $G$ is a smooth
$A$-algebra, $T$ is a polynomial $A$-algebra, $0\not=h\in T$ and $P$ is a
regular system of elements of $G$, respectively $T$.

Using the above theorem we obtained in [8, Theorem 1] the following result.

######  Theorem 3.

Let $V$ be a valuation ring containing a perfect field $F$ of characteristic
$p>0$, $k$ its residue field, $\Gamma$ its value group and $K$ its fraction
field. Then the following statements hold

  1. (1)

if $\Gamma$ is finitely generated, $k\subset V$ then $V$ is a filtered union
of its complete intersection $F$-subalgebras,

  2. (2)

if $\Gamma$ is finitely generated, $k/F$ is separably generated (that is there
exists a transcendental basis $x$ of $k$ over $F$ such that the field
extension $F(x)\subset k$ is algebraic separable) and $V$ is Henselian, then
$V$ is a filtered union of its complete intersection $F$-subalgebras,

  3. (3)

if $k\subset V$ then $V$ is a filtered direct limit of complete intersection
$F$-algebras,

  4. (4)

if $k/F$ is separably generated and either $\dim V=1$, or $V$ is Henselian if
$\dim V>1$, then $V$ is a filtered direct limit of complete intersection
$F$-algebras.

The goal of this paper is to extend the above result for the mixed valuation
rings (see Theorem 18, Corollary 19).

######  Theorem 4.

Let $V$ be a mixed characteristic valuation ring, $k$ its residue field,
$p=$char $k$, $\Gamma$ its value group. Assume $k$ is separably generated over
$F_{p}$. Then $V$ is a filtered direct limit of ${\bf Z}_{(p)}$-algebras if
one of the following statements hold:

  1. (1)

$V$ is Henselian and $\Gamma/{\bf Z}\mathrm{val}(p)$ has no torsion,

  2. (2)

$V$ is Henselian and $\Gamma/{\bf Z}\mathrm{val}(p)\cong{\bf Z}/t{\bf Z}\oplus
P$ for some $t\in{\bf N}\setminus p{\bf N}$ with $k=k^{t}$ and some $\bf
Z$-module $P$ without torsion,

  3. (3)

$\dim V=1$ and $\Gamma/{\bf Z}\mathrm{val}(p)$ has no torsion,

  4. (4)

$\dim V=1$ and $\Gamma/{\bf Z}\mathrm{val}(p)\cong{\bf Z}/t{\bf Z}\oplus P$
for some $t\in{\bf N}\setminus p{\bf N}$ with $k=k^{t}$ and some $\bf
Z$-module $P$ without torsion.

##  1\. Valuation rings of mixed characteristic and cross-sections

######  Lemma 5.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field and $p=$char $k$. Assume $k$ is separably generated over $F_{p}$ the
finite field with $p$ elements. Then there exists a DVR subring $R\subset V$
such that

  1. (1)

$pR$ is the maximal ideal of $R$,

  2. (2)

$R\subset V$ is an extension of valuation rings with the trivial residue field
extension.

###### Proof.

By hypothesis there exists a system of elements $x$ of $V$ inducing a
separable transcendental basis of $k$ over $F_{p}$. Then $V^{\prime}=({\bf
Z}_{(p)}[x])_{p{\bf Z}_{(p)}[x]}$ is a DVR, $pV^{\prime}$ is its maximal ideal
and the residue field extension of $V^{\prime}\subset V$ is algebraic
separable. Let ${\bar{y}}\in k$ which is not in the residue field $k^{\prime}$
of $V^{\prime}$ and ${\bar{f}}=$ Irr$({\bar{y}},k^{\prime})\in k^{\prime}[Y]$.
Let $f\in V^{\prime}[Y]$ be a monic polynomial lifting $\bar{f}$. As $V$ is
Henselian we may lift $\bar{y}$ to a solution of $f$ in $V$ and
$V_{1}=(V^{\prime}[Y]/(f))_{pV^{\prime}[Y]}$ is a DVR such that $\bar{y}$ is
contained in the residue field of $V_{1}$. Using this trick by transfinite
induction or by Zorn’s Lemma we find such $R$.

######  Proposition 6.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$ and $\Gamma$ its value group. Assume $\Gamma$ is finitely
generated, height$(pV)=1$, $\Gamma/{\bf Z}\mathrm{val}(p)$ has no torsion and
$k$ is separably generated over $F_{p}$. Then $V$ is a filtered union of its
complete intersection algebras over a DVR $R$ with its maximal ideal $pR$ and
its residue field $k$. In particular $V$ is a filtered union of its complete
intersection algebras over ${\bf Z}_{(p)}$.

###### Proof.

By Lemma 5 there exists a DVR subring $R\subset V$ such that $pR$ is the
maximal ideal of $R$ and the extension $R\subset V$ has the trivial residue
field extension. Then $\Gamma/{\bf Z}\mathrm{val}(p)$ is free and choose a
$\bf Z$ basis $\mathrm{val}(x)$, $x\in V^{e}$ in it. Thus $x$ is algebraically
independent over $R$ by [2, Theorem 1 in VI (10.3)]. Let $T$ be the fraction
field of $R$ and set $W=V\cap T(x)$. Note that $\Gamma$ is the value group of
$W$. By [7, Lemma 26 (2)], $W$ is a filtered union of its complete
intersection $R$-subalgebras, even regular local rings. Using Theorem 2 note
that $V$ is a filtered union of its complete intersection $W$-subalgebras
since $W\subset V$ is immediate. But $R$ is a filtered union of its smooth
${\bf Z}_{(p)}$-subalgebras by Néron desingularization [4] (see also [3]) and
we are done.

######  Proposition 7.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$ and $\Gamma$ its value group. Assume $k$ is separably
generated over $F_{p}$, height$(pV)=1$ and $\Gamma/{\bf
Z}\mathrm{val}(p)\cong({\bf Z}/t{\bf Z})\oplus P$ for some free finitely
generated $\bf Z$-module $P$ and a $t\in{\bf N}\setminus p{\bf N}$ with
$k=k^{t}$. Then $V$ is a filtered union of its complete intersection algebras
over a DVR $A\subset V$ with its residue field $k$ and its local parameter
$\pi$ such that $\Gamma/{\bf Z}\mathrm{val}(\pi)\cong P$ and $A\subset V$ is
an extension of valuation rings. Moreover, $V$ is a filtered union of its
complete intersection algebras over ${\bf Z}_{(p)}$.

###### Proof.

By Lemma 5 there exists a DVR subring $R\subset V$ such that $pR$ is the
maximal ideal of $R$ and $R\subset V$ is an extension with the trivial residue
field extension. Let $T$ be the fraction field of $R$. By hypothesis there
exists $\gamma\in\Gamma$ such that $t\gamma=\mathrm{val}(p)$ and $\Gamma/{\bf
Z}\gamma\cong P$ has no torsion. Let $\pi\in V$ be such that
$\mathrm{val}(x)=\gamma$. Thus $\pi^{t}=pu$ for some unit $u\in V$. The
equation $X^{t}-u=0$ has a solution modulo ${\mathfrak{m}}$ the maximal ideal
of $V$ because $k=k^{t}$, which can be lifted to a solution $u^{\prime}$ in
$V$ by the Implicit Function Theorem. Changing $\pi$ by $\pi/u^{\prime}$ we
may assume $\pi^{t}=p$. Note that the DVR $A=V\cap
T(\pi)\cong(R[X]/(X^{t}-p))_{(X)}$ is a complete intersection algebra over $R$
and $\Gamma/{\bf Z}\mathrm{val}(\pi)$ is without torsion.

As in the above proposition we see that $V$ is a filtered union of its
complete intersection $A$-subalgebras and $R$ is a filtered union of its
smooth ${\bf Z}_{(p)}$-subalgebras by Néron desingularization. Since $A$ is a
complete intersection $R$-algebra we see that $V$ is a filtered union of its
complete intersection algebras over ${\bf Z}_{(p)}$.

Let $q$ be the minimal prime ideal of $pV$ and $L$ the fraction field of
$L/q$. The proof of Proposition 6 goes also when $\Gamma$ is not finitely
generated but there exists a cross-section $s:\Gamma/{\bf
Z}\mathrm{val}(p)\rightarrow L^{*}=L\setminus\\{0\\}$, that is a section of
the map $\mathrm{val}:L^{*}\rightarrow\Gamma/{\bf Z}\mathrm{val}(p)$.

######  Proposition 8.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$, $\Gamma$ its value group and $q$ the minimal prime ideal
of $pV$. Assume height$(q)=1$, $\Gamma^{\prime}=\Gamma/{\bf Z}\mathrm{val}(p)$
has no torsion, $k$ is separably generated over $F_{p}$ and there exists a
cross-section $s:\Gamma^{\prime}\rightarrow L^{*}$, $L$ being the fraction
field of $V/q$. Then $V$ is a filtered union of its complete intersection
algebras over a DVR $R$ with its maximal ideal $pR$ and its residue field $k$.
Moreover, $V$ is a filtered union of its complete intersection algebras over
${\bf Z}_{(p)}$.

###### Proof.

Let $\Gamma_{1}$ be a finitely generated subgroup of $\Gamma$ containing
$\mathrm{val}(p)$. We may assume that $\Gamma_{1}/{\bf Z}\mathrm{val}(p)$ is
free of basis $\mathrm{val}(x)$ for some $x\in V^{e}$. Let $R$ be given by
Lemma 5, $T$ its fraction field and set $W_{1}=V\cap T(s(\Gamma_{1}/{\bf
Z}\mathrm{val}(p)))=V\cap T(x)$. The value group of $W_{1}$ is $\Gamma_{1}$.
As in Proposition 6 we see that $W_{1}$ is a filtered union of its complete
intersection $R$-subalgebras, in particular a filtered union of its complete
intersection ${\bf Z}_{(p)}$-subalgebras. Set $W=V\cap T(s(\Gamma/{\bf
Z}\mathrm{val}(p)))$. Then $W$ is a filtered union of such $W_{1}$ as above
and so of its complete intersection $R$-subalgebras and in particular a
filtered union of its complete intersection ${\bf Z}_{(p)}$-subalgebras. As
the extension $W\subset V$ is immediate we may apply Theorem 2 as in
Proposition 6.

Also the proof of Proposition 8 could be extended when $\Gamma/{\bf
Z}\mathrm{val}(p)$ has torsion but no $p$-torsion.

######  Proposition 9.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$, $\Gamma$ its value group and $q$ the minimal prime ideal
of $pV$. Assume height$(q)=1$, $\Gamma^{\prime}=\Gamma/{\bf Z}\mathrm{val}(p)$
has no $p$-torsion, $k$ is separably generated over $F_{p}$, $k=k^{t}$ for all
$t\in{\bf N}\setminus p{\bf N}$ and there exists a cross-section
$s:\Gamma^{\prime\prime}\rightarrow L^{*}$, $L$ being the fraction field of
$V/q$ and $\Gamma^{\prime\prime}$ is the factor of $\Gamma/{\bf
Z}\mathrm{val}(p)$ by its torsion. Then $V$ is a filtered union of its
complete intersection algebras over ${\bf Z}_{(p)}$.

###### Proof.

Let $R$ be given by Lemma 5 and $T$ its fraction field. Let
$\Gamma_{1}\subset\Gamma$ be a finitely generated subgroup containing
$\mathrm{val}(p)$. Note that $\Gamma_{1}$ is free and we may find a basis of
it such that $\mathrm{val}(p)$ is a multiple of one element from the basis.
Then $\Gamma_{1}/{\bf Z}\mathrm{val}(p)\cong({\bf Z}/t{\bf Z})\oplus P$ for
some free $\bf Z$-module $P$ and for some $t\in{\bf N}\setminus p{\bf N}$. Let
$A_{1}$ be given as in Proposition 7 with residue field $k$ and $\pi_{1}$ a
local parameter of $A_{1}$ such that $\mathrm{val}(\pi_{1})\in\Gamma_{1}$ and
$\Gamma_{1}/{\bf Z}\mathrm{val}(\pi_{1})$ has no torsion. Let
$s:\Gamma^{\prime\prime}\rightarrow L^{*}$ be the given cross-section. Let $G$
be the set of all $z\in V$ such that $z^{t}=p$ for some $t\in{\bf N}\setminus
p{\bf N}$ and set $W=V\cap T(G,s(\Gamma^{\prime\prime}))$. Let $T_{1}$ be the
fraction field of $A_{1}$. Set $W_{1}=V\cap T_{1}(s(\Gamma_{1}/{\bf
Z}\mathrm{val}(\pi_{1})))$. Note that $\Gamma_{1}$ is the value group of
$W_{1}$. The set of such valuation rings $W_{1}$ is filtered and their union
is $W$. The extension $W\subset V$ is immediate and $V$ is a filtered union of
its complete intersection algebras over $W$ by Theorem 2. But each $W_{1}$ is
a filtered union of its complete intersection algebras over ${\bf Z}_{(p)}$ by
Proposition 7.

##  2\. Some applications of the Zariski Uniformization Theorem

To extend the above propositions for the case height$(pV)>1$ we need to apply
the Zariski Uniformization Theorem and the following two lemmas, the first
being [7, Lemma 7] and the second one being very closed to [8, Lemma 19]. We
need to present firstly some notions.

For a finitely presented ring map $A\rightarrow B$, an element $b\in B$ is
_standard over $A$_ if there exists a presentation $B\cong
A[Y_{1},\dots,Y_{m}]/I$ and $f_{1},\dots,f_{r}\in I$ with $r\leq m$ such that
$b=b^{\prime}b^{\prime\prime}$ with $b^{\prime}=\det({(\partial f_{i}/\partial
Y_{j}))_{1\leq i,\,j\leq r}}\in A[Y_{1},\dots,Y_{m}]$ and a
$b^{\prime\prime}\in A[Y_{1},\dots,Y_{m}]$ that kills $I/(f_{1},\dotsc,f_{r})$
(our standard element is a special power of the standard element from [9,
Definition, page 9] given in the particular case of the valuation rings). The
loci of vanishing of standard elements of $B$ over $A$ cut out the locus of
non-smoothness of $\operatorname{Spec}(B)\rightarrow\operatorname{Spec}(A)$.
The radical of the ideal generated by the elements of $B$ standard over $A$ is
$H_{B/A}$.

######  Lemma 10.

For a commutative diagram of ring morphisms

$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\scriptstyle{b\,\mapsto\,a}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{V}$that
factors as
follows$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{V/a^{3}V}$$\textstyle{A^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A^{\prime}/a^{3}A^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$

with $B$ finitely presented over $A$, a $b\in B$ that is standard over $A$
(this means a special element from the ideal $H_{B/A}$ defining the non smooth
locus of $B$ over $A$, for details see for example [7, Lemma 4]), and a
nonzerodivisor $a\in A^{\prime}$ that maps to a nonzerodivisor in $V$ that
lies in every maximal ideal of $V$, there is a smooth $A^{\prime}$-algebra $S$
such that the original diagram factors as follows:

$\textstyle{B\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{A\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{V.}$$\textstyle{A^{\prime}\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$$\textstyle{S\ignorespaces\ignorespaces\ignorespaces\ignorespaces}$

######  Lemma 11.

Let $V\subset V^{\prime}$ be an extension of valuation rings, $E$ a finitely
presented $V$-algebra, $w:E\rightarrow V^{\prime}$ a $V$-morphism and $b\in V$
inducing a standard element for $E$ over $V$. Assume the map
$E/b^{3}E\rightarrow V^{\prime}/b^{3}V^{\prime}$ induced by $w$ factors
through a complete intersection $V/b^{3}V$-algebra. Then $w$ factors through a
complete intersection $V$-algebra.

###### Proof.

We follow the proof of [8, Lemma 19]. By assumption the composite map
$E\xrightarrow{w}V^{\prime}\subset V^{\prime}/b^{3}V^{\prime}$ factors through
a complete intersection $V/b^{3}V$-algebra $B$, let us say through a map
$B\xrightarrow{\rho^{\prime}}V^{\prime}/b^{3}V^{\prime}$. We may assume that
$B$ has the form $T_{h}/(P)$ for a polynomial $V/b^{3}V$-algebra $T$,
$U=(U_{1},\ldots,U_{l})$, $h\in T$ and a regular system of elements
$P=(P_{1},\ldots,P_{t})$ from $T$. Suppose the map $\rho^{\prime}$ is given by
$U\rightarrow u^{\prime}$ for some
$u^{\prime}\in(V^{\prime}/b^{3}V^{\prime})^{l}$. Let $u\in V^{l}$ be a lifting
of $u^{\prime}$. We have $P(u)=b^{3}z$ for some $z$ in $V^{\prime l}$ and
$h(u)$ invertible. Set $D=(V[U,Z]/(P-b^{3}Z))_{h}$, $Z=(Z_{1},\ldots,Z_{t})$
and let $\rho:D\rightarrow V$ be the map $(U,Z)\rightarrow(u,z)$. Note that
$D$ is a complete intersection $V$-algebra. As $w$ factors through $\rho$
modulo $b^{3}$ there exists a smooth $D$-algebra $D^{\prime}$ such that $w$
factors through $D^{\prime}$ using Lemma 10. It follows that $D^{\prime}$ is a
complete intersection $V$-algebra as well.

The next proposition is Proposition 8 when height$(pV)$ is not necessarily
$1$.

######  Proposition 12.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$, $\Gamma$ its value group and $q$ the minimal prime ideal
of $pV$. Assume $\Gamma^{\prime}=\Gamma/{\bf Z}\mathrm{val}(p)$ has no
torsion, $k$ is separably generated over $F_{p}$ and there exists a cross-
section $s:\Gamma^{\prime}\rightarrow L^{*}$, $L$ being the fraction field of
$V/q$. Then $V$ is a filtered direct limit of some complete intersection
algebras over a DVR $R$ with its maximal ideal $pR$ and its residue field $k$.
In particular $V$ is a filtered direct limit of some complete intersection
${\bf Z}_{(p)}$-algebras.

###### Proof.

Let $R$ be the DVR given by Lemma 5 and $T$ its fraction field. By Proposition
8 we may assume that height$(pV)>1$ and let $q^{\prime}\in Spec(V)$ the prime
ideal corresponding to the maximal ideal of $T\otimes_{R}V$. Then
height$(q/q^{\prime})=1$ and applying Proposition 8 for the extension
$R\subset V/q^{\prime}$ we see that $V/q^{\prime}$ is a filtered union of its
complete intersection $R$-subalgebras.

Let $E\subset V$ be a finitely generated $R$-algebra of $V$ and
$w:E\rightarrow V$ its inclusion. Using the Zariski Uniformization Theorem
[11] (see also [7, Theorem 35]) we see that the map $L\otimes_{R}E\rightarrow
V_{q^{\prime}}$ factors through a smooth $L$-algebra and so $w$ factors
through a finitely generated $R$-algebra $E^{\prime}$, let us say through a
map $w^{\prime}:E^{\prime}\rightarrow V$ such that
$w^{\prime}(H_{E/R})\not\subset q^{\prime}$ applying [7, Lemma 13]. Then a
power of $p$ belongs to $w^{\prime}(H_{E/R})$ and we may assume that
$p^{s^{\prime}}$ is standard for $E^{\prime}$ over $R$. Set $s=3s^{\prime}$.
As above we get in particular that the composite map
$E^{\prime}\xrightarrow{w^{\prime}}V\rightarrow V/p^{s}V$ factors through a
complete intersection $R/(p^{s})$-algebra and using Lemma 11 we see that
$w^{\prime}$ factors through a complete intersection $R$-algebra. Now it is
enough to apply [9, Lemma 1.5].

Similarly, we get the following proposition from Proposition 9 when
height$(pV)$ is not necessarily $1$.

######  Proposition 13.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$, $\Gamma$ its value group and $q$ the minimal prime ideal
of $pV$. Assume $\Gamma^{\prime}=\Gamma/{\bf Z}\mathrm{val}(p)$ has no
$p$-torsion, $k$ is separably generated over $F_{p}$, $k=k^{t}$ for all
$t\in{\bf N}\setminus p{\bf N}$ and there exists a cross-section
$s:\Gamma^{\prime\prime}\rightarrow L^{*}$, $L$ being the fraction field of
$V/q$ and $\Gamma^{\prime\prime}$ is the factor of $\Gamma/{\bf
Z}\mathrm{val}(p)$ by its torsion. Then $V$ is a filtered direct limit of some
complete intersection algebras over ${\bf Z}_{(p)}$.

######  Corollary 14.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$, $\Gamma$ its value group and $q$ the minimal prime ideal
of $pV$. Assume $\Gamma^{\prime}=\Gamma/{\bf Z}\mathrm{val}(p)\cong{\bf
Z}/t{\bf Z}\oplus P$ for some $t\in{\bf N}\setminus p{\bf N}$ and some $\bf
Z$-module without torsion, $k$ is separably generated over $F_{p}$, $k=k^{t}$
and there exists a cross-section $s:\Gamma^{\prime\prime}\rightarrow L^{*}$,
$L$ being the fraction field of $V/q$ and $\Gamma^{\prime\prime}$ is the
factor of $\Gamma/{\bf Z}\mathrm{val}(p)$ by its torsion. Then $V$ is a
filtered direct limit of some complete intersection algebras over ${\bf
Z}_{(p)}$.

We need the following proposition [7, Proposition 22].

######  Proposition 15.

Let $V\subset V^{\prime}$ be an extension of valuation rings. Suppose that

  1. (1)

$V$ is a discrete valuation ring with a local parameter $\pi$

  2. (2)

$\pi V^{\prime}$ is the maximal ideal of $V^{\prime}$,

  3. (3)

the residue field extension of $V\subset V^{\prime}$ is separable.

Then $V^{\prime}$ is a filtered direct limit of some smooth $V$-algebras.

The proof of the Proposition 12 could be modified to show the following lemma.

######  Lemma 16.

Let $V$ be a mixed characteristic valuation ring, $k$ its residue field,
$p=$char $k$ and $\Gamma$ its value group. Assume there exist

  1. (1)

a DVR $A\subset V$ with $\pi$ as a local parameter such that $A\subset V$ is
an extension and its residue field extension is trivial,

  2. (2)

a valuation subring $B\subset V$ containing $A$ with $\pi B$ the maximal ideal
of $B$ and with value group $\Gamma^{\prime}\subset\Gamma$ such that $B\subset
V$ is an extension and $\Gamma^{\prime\prime}=\Gamma/\Gamma^{\prime}$ has no
torsion,

  3. (3)

there exists a cross-section $s:\Gamma^{\prime\prime}\rightarrow L^{*}$, $L$
being the fraction field of $V/q$, $q$ being the minimal prime ideal of $\pi
V$.

Then $V$ is a filtered direct limit of some complete intersection algebras
over $A$.

###### Proof.

Let $q$ be the minimal prime ideal of $\pi V$ and
$q^{\prime}\in\operatorname{Spec}V$ the prime ideal corresponding to the
maximal ideal of the fraction ring of $V$ by the multiplicative system
generated by $\pi$. Then height$(q/q^{\prime})=1$, $q\cap B=\pi B$ and
height$(\pi B/(q^{\prime}\cap B)=1$. Using (3) and [7, Lemma 26 (2)] as in
Proposition 8 we see that $V/q^{\prime}$ is a filtered union of its complete
intersection $B/(q^{\prime}\cap B)$-subalgebras. This shows that $V$ is a
filtered union of its complete intersection $B$-algebras if $q^{\prime}=0$.
Otherwise we apply the Zariski Uniformization Theorem as in Proposition 12 to
get $V$ as a filtered direct limit of complete intersection algebras over $B$.
Using Proposition 15 we see that $B$ is a filtered direct limit of smooth
$A$-algebras, which is enough.

##  3\. Main results

To extend the Propositions 6, 7 for the case when $\Gamma$ is not finitely
generated we need to apply some facts from Model Theory. We will need [7,
Theorem A10] which we present below (we could use also [10, Proposition 5.4],
or [1, 3..3.39, 3.3.40]).

######  Theorem 17.

For a valuation ring $V$ with value group $\Gamma$, there is a countable
sequence of ultrafilters ${\mathscr{U}}_{1},{\mathscr{U}}_{2},\dots$ on some
respective sets $U_{1},U_{2},\dots$ for which the valuation rings
$\\{V_{n}\\}_{n\geq 0}$ defined inductively by $V_{0}:=V$ and
$V_{n+1}:=\prod_{{\mathscr{U}}_{n+1}}V_{n}$ are such that the valuation ring

$\textstyle\widetilde{V}:=\varinjlim_{n\geq 0}V_{n}\quad\text{has a cross-
section}\quad\widetilde{s}:\widetilde{\Gamma}\rightarrow\widetilde{K}^{*},$

where $\widetilde{K}$ and $\widetilde{\Gamma}$ are the fraction field and the
value group of $\widetilde{V}$.

######  Theorem 18.

Let $V$ be a Henselian mixed characteristic valuation ring, $k$ its residue
field, $p=$char $k$ and $\Gamma$ its value group. Assume $k$ is separably
generated over $F_{p}$ and either

  1. (1)

$\Gamma/{\bf Z}\mathrm{val}(p)$ has no torsion, or

  2. (2)

$\Gamma/{\bf Z}\mathrm{val}(p)\cong{\bf Z}/t{\bf Z}\oplus P$ for some
$t\in{\bf N}\setminus p{\bf N}$ with $k=k^{t}$ and some $\bf Z$-module $P$
without torsion.

Then $V$ is a filtered direct limit of some complete intersection algebras
over ${\bf Z}_{(p)}$.

###### Proof.

Let $R$ be given by Lemma 5 and $L$ its fraction field. Let $q$ be the minimal
prime ideal of $pV$. In case (2) as in Proposition 9 we find an element
$\pi\in V$ such that $\pi^{t}=p$ and set $A=V\cap L(\pi)$. In both cases we
have a DVR subring $A$ of $V$ with $\Gamma/{\bf Z}\pi$ without torsion and the
residue field extension of $A\subset V$ trivial. Apply the above theorem for
$V/q$. Then there is a countable sequence of ultrafilters
${\mathscr{U}}_{1},{\mathscr{U}}_{2},\dots$ on some respective sets
$U_{1},U_{2},\dots$ for which the valuation rings $\\{W_{n}\\}_{n\geq 0}$
defined inductively by $W_{0}:=V/q$ and
$W_{n+1}:=\prod_{{\mathscr{U}}_{n+1}}W_{n}$ are such that the valuation ring

$\textstyle\widetilde{W}:=\varinjlim_{n\geq 0}W_{n}\quad\text{has a cross-
section}\quad\widetilde{s}:\widetilde{\Gamma}\rightarrow\widetilde{K}^{*},$

where $\widetilde{K}$ and $\widetilde{\Gamma}$ are the fraction field and the
value group of $\widetilde{W}$. Set $V_{0}=V$ and
$V_{n+1}:=\prod_{{\mathscr{U}}_{n+1}}V_{n}$. Then the valuation ring
$\widetilde{V}:=\varinjlim_{n\geq 0}V_{n}$ has the property $q\widetilde{V}$
is a prime ideal and $\widetilde{V}/q\widetilde{V}\cong\widetilde{W}$.
Similarly we define $\widetilde{A}$. Note that the residue field extension of
$\widetilde{A}\subset\widetilde{V}$ is trivial and $\pi\widetilde{A}$ is the
maximal ideal of $\widetilde{A}$. As $\Gamma/{\bf Z}\mathrm{val}(\pi)$ has no
torsion we see that $\widetilde{\Gamma}/{\bf Z}\mathrm{val}(\pi)$ has also no
torsion. Thus $\widetilde{V}$ is a filtered direct limit of complete
intersection algebras of $\widetilde{A}$-algebras as in Proposition 13 and
Corollary 14. Since $\widetilde{(}A)$ is a filtered direct limit of some
complete intersection algebras over $A$ by Proposition 15 we see that
$\widetilde{V}$ is a filtered direct limit of some complete intersection
algebras over $A$ and even over ${\bf Z}_{(p)}$ (see Lemma 16).

Let $E\subset V$ be a finitely generated ${\bf Z}_{(p)}$-algebra and $w$ its
inclusion. Then the composite map $E\rightarrow V\rightarrow\widetilde{V}$
induced by $w$ factors through a complete intersection ${\bf Z}_{(p)}$-algebra
$G$, let us say through a map $\rho:G\rightarrow\widetilde{V}$. As $G$ is
finitely generated over ${\bf Z}_{(p)}$ we may consider $\rho:G\rightarrow
V_{n}$ for some $n$. Set $\rho_{n}=\rho$.

As in the proof of [7, Theorem 35] we may find also such map
$\rho_{n-1}:G\rightarrow V_{n-1}$ if $n>0$. Indeed, let $E={\bf
Z}_{(p)}[Z]/(h)$, $Z=(Z_{1},\ldots,Z_{s})$, where $h$ is a system of
polynomials from ${\bf Z}[Z]$. Let $G={\bf Z}_{(p)}[Y]/(P)$,
$Y=(Y_{1},\ldots,Y_{n})$ and $P=(P_{1},\ldots,P_{l})$ a regular system of
elements of ${\bf Z}_{(p)}[Y]$. Assume the map $E\rightarrow G$ be given by
$Z\rightarrow g$ for some polynomials $g$ in ${\bf Z}_{(p)}[Y]$. Suppose that
the map $\rho_{n}$ is given by $Y\rightarrow y$ for some $y$ in $V_{n}$ and
$w$ is given by $Z\rightarrow z$ for some $z$ in $V$. Then the system of
polynomial equations $P(Y)=0$ , $g(Y)=z$ over $V$ has a solution $y$ in
$V_{n}$. As $V_{n}$ is the ultrapower of $V_{n-1}$ we see that this system has
also a solution in $V_{n-1}$, that is the composite map $E\rightarrow
V\rightarrow V_{n-1}$ factors also through $G$.

Step by step we see that $w$ factors through $G$ and it is enough to apply [9,
Lemma 1.5].

######  Corollary 19.

Let $V$ be a mixed characteristic valuation ring of one dimension, $k$ its
residue field, $p=$char $k$ and $\Gamma$ its value group. Assume $k$ is
separably generated over $F_{p}$ and either

  1. (1)

$\Gamma/{\bf Z}\mathrm{val}(p)$ has no torsion, or

  2. (2)

$\Gamma/{\bf Z}\mathrm{val}(p)\cong{\bf Z}/t{\bf Z}\oplus P$ for some
$t\in{\bf N}\setminus p{\bf N}$ with $k=k^{t}$ and some $\bf Z$-module $P$
without torsion.

Then $V$ is a filtered direct limit of some complete intersection algebras
over ${\bf Z}_{(p)}$.

For the proof we reduce the problem to the completion of $V$ using [7,
Proposition 9], and we may apply the above theorem.

## References

  * [1] M. Aschenbrenner, L. van den Dries, J. van der Hoeven, Asymptotic differential algebra and model theory of transseries, Annals of Mathematics Studies, 195, Princeton University Press, Princeton, NJ, 2017.
  * [2] N. Bourbaki, Éléments de mathématique. Algèbre commutative, chap. I-VII, Hermann (1961, 1964, 1965); chap. VIII-X, Springer (2006, 2007) (French).
  * [3] A. Khalid, A. Popescu, D. Popescu, Algorithms in the classical Néron Desingularization, Bull. Math. Soc. Sci. Math. Roumanie, 61(109), (2018), 73-83, arXiv:AC/1702.01445.
  * [4] A. Néron, Modèles minimaux des varietés abéliennes sur les corps locaux et globaux, Publ. Math. IHES, 21, 1964, 5-128.
  * [5] A. Ostrowski, Untersuchungen zur arithmetischen Theorie der Körper, Math. Z. 39 (1935), no. 1, 321–404.
  * [6] D. Popescu, Algebraic extensions of valued fields, J. Algebra 108 (1987), no. 2, 513–533.
  * [7] D. Popescu, Néron desingularization of extensions of valuation rings with an Appendix by Kęstutis Česnavičius, to appear in Proceedings of Transient Transcendence in Transylvania 2019, Eds. Alin Bostan, Kilian Raschel (possible in Springer Collection PROMS), arxiv/AC:1910.09123v4.
  * [8] D. Popescu, Valuation rings as limits of complete intersection rings,
arxiv/AC:2004.11004v5.

  * [9] R. Swan, Neron-Popescu desingularization, in "Algebra and Geometry", Ed. M. Kang, International Press, Cambridge, (1998), 135-192.
  * [10] L. van den Dries, Lectures on the model theory of valued fields., in Model theory in algebra, analysis and arithmetic, 55–157, Lect. Notes in Math., 2111, Springer, Heidelberg, 2014,
  * [11] O. Zariski, Local uniformization on algebraic varieties, Ann. of Math. 41 (1940), 852-896,


# IITP in COLIEE@ICAIL 2019: Legal Information Retrieval using BM25 and BERT

Baban Gain<EMAIL_ADDRESS>Government College of Engineering and Textile
Technology, BerhamporeBerhamporeWest Bengal742101 ,  Dibyanayan Bandyopadhyay
<EMAIL_ADDRESS>Government College of Engineering and Textile Technology,
BerhamporeBerhamporeWest Bengal742101 ,  Tanik Saikh<EMAIL_ADDRESS>Indian Institute of Technology, PatnaBihta, PatnaBihar801103 and  Asif Ekbal
<EMAIL_ADDRESS>Indian Institute of Technology, PatnaBihta, PatnaBihar801103

###### Abstract.

Natural Language Processing (NLP) and Information Retrieval (IR) in the
judicial domain is an essential task. With the advent of availability domain-
specific data in electronic form and aid of different Artificial intelligence
(AI) technologies, automated language processing becomes more comfortable, and
hence it becomes feasible for researchers and developers to provide various
automated tools to the legal community to reduce human burden. The Competition
on Legal Information Extraction/Entailment (COLIEE-2019) run in association
with the International Conference on Artificial Intelligence and Law
(ICAIL)-2019 has come up with few challenging tasks. The shared defined four
sub-tasks (i.e. Task1, Task2, Task3 and Task4), which will be able to provide
few automated systems to the judicial system. The paper presents our working
note on the experiments carried out as a part of our participation in all the
sub-tasks defined in this shared task. We make use of different Information
Retrieval(IR) and deep learning based approaches to tackle these problems. We
obtain encouraging results in all these four sub-tasks.

BM25, tf-idf, text similarity, Entailment

††copyright: rightsretained††conference: COLIEE 2019 workshop: Competition on
Legal Information Extraction/Entailment ; June 21, 2019; Montreal, Quebec

##  1\. Introduction

Language technology based automated system has a great impact and demand in
judicial domain. As for example, it will mitigate the problem of manual
checking of large amount of previous files to find relevancy to a current
case. This kind of system would be very effective to lawyers which will
expedite the process of providing verdict on a particular case. Previously, it
took couple of months and/or even a year to get verdict of a particular case.
It is observed that there are many cases where practitioners rely on previous
court case files to extract information and try to find relevancy with him/her
current solving case. In normal practise they do it process manually. However,
computerised processing of such judicial domain texts is taxing as the files
are very large in size and remain unstructured. This stipulates lot of
research in this domain. The Competition on Legal Information Extraction and
Entailment (COLIEE-2019) an associated event of (ICAIL)-2019 has defined four
problems related to IR in this domain. Task 1 and Task 2 are involved the case
law competition, Task 1 and Task 2 are associated with statute law
competition. The data for these two types of tasks are extracted accordingly.
We participate in all the four tasks (i.e. Tasks 1, Tasks 2, Tasks 3 and Tasks
4) defined in this competition. All of these four tasks are vary emerging and
are of utmost priority.

Task 1: This task is basically legal case retrieval task. Overall the task is:
after reading an new case Q, the system has to extract supporting cases S1,
S2, S3, ….Sn from the provided case law dataset. This will support the
decision for Q.
Task 2: This legal case entailment task, which involves the detection of a
paragraph from the existing cases that entails the decision of a new case.
Task 3: The task 3 is essentially a legal Question-Answering task. The task
could be defined as: reading a legal bar examination question Q and extracting
a subset of Japanese Civil Code Articles S1, S2,…, Sn from the entire Civil
Code which are those appropriate for answering the question such that i.e.
Entails(S1, S2, …, Sn , Q) or Entails(S1, S2, …, Sn , not Q).
Overall, the task is given a question Q and the entire Civil Code Articles,
the system has to retrieve the set of articles ”S1, S2, …, Sn” as the answer
of this track.
Task 4: This task involves legal question answering task which associates the
identification of an entailment relationship such that
Entails(S1, S2, …, Sn , Q) or Entails(S1, S2, …, Sn , not Q).
Given a question Q, the participating systems have to retrieve relevant
articles S1, S2, …, Sn in the first phase, and in the second phase, the
systems have to determine if the relevant articles entail ”Q” or ”not Q”. The
answer of this track would be binary: i.e. ”YES”(”Q”) or ”NO”(”not Q”).
We propose various approaches to combat these problems. We discuss all our
approaches in the section 4.

##  2\. Our Approaches

The propose approaches for the all four tasks are based on Doc2Vec, BM25, tf-
idf, Bidirectional Encoder Representations from Transformers (BERT) (Devlin et
al., 2018). The following subsections will discuss all the approaches one by
one.

###  2.1. Doc2Vec

Doc2Vec (Le and Mikolov, 2014) is an unsupervised algorithm that learns fixed-
length feature representations from variable-length pieces of texts, such as
sentences, paragraphs, and even a whole documents. This algorithm represents
each document by a dense vector which is trained to predict words in the
document. More precisely, we concatenate on the paragraph vector with several
word vectors from a paragraph and predict the following word in the given
context. Both word vectors and paragraph vectors are trained by the stochastic
gradient descent and back-propagation (Rumelhart et al., 1988). While
paragraph vectors are unique among paragraphs, the word vectors are shared. At
prediction time, the paragraph vectors are inferred by fixing the word vectors
and training the new paragraph vector until convergence.

###  2.2. BM25

BM25 (Robertson et al., 2009) is one of the most successful text-retrieval
algorithm. BM25 is a bag-of-words retrieval function that ranks a set of
documents based on the query terms appearing in each document, regardless of
the inter-relationship between the query terms within a document. Given a
query Q, containing keywords q1,q2,….qn the BM25 score of a document D is:

(1)
$Score(D,Q)=\sum_{i=1}^{n}IDF(q_{i})\frac{f(q_{i},D).(k_{1}+1)}{f(q_{i},D)+k_{i}.(1-b+b.\frac{|D|}{avgdl})}$

where f($q_{i}$, D) is $q_{i}$’s term frequency in the document D, —D— is the
length of the document D in words, and avgdl is the average document length in
the text collection from which documents are drawn.

###  2.3. Term Frequency * Inverse Document Frequency

Term Frequency and Inverse Document Frequency (tf*idf) is a numerical
statistic that is intended to reflect how important a word is to a document in
a collection of corpus. It is often used as a weighting factor in searches of
information retrieval, text mining, and user modeling. The tf*idf value
increases proportionally to the number of times a word appears in the document
and is offset by the number of documents in the corpus that contain the word,
which helps to adjust for the fact that some words appear more frequently in
general. The formula that is used to compute the tf*idf for a term t of a
document d in a document set is:

(2) $tf-idf(t,d)=tf(t,d)*idf(t)$

We implement this using python library (Pedregosa et al., 2011).

###  2.4. Bidirectional Encoder Representations from Transformers (BERT)

We make use of a novel language representation model called Bidirectional
Encoder Representations from Transformers (BERT). The model is the latest
language representation model used in various tasks of NLP. The task 4 defined
in this shared task is a classification task. We treat the task as a sentence
classification task by considering ’t1’ and ’t2’ as two sentences
respectively. Using BERT we obtain a vector corresponding to the ’[CLS]’ token
at the last layer. This vector is subsequently used for classification. So per
’t1’ and ’t2’ we get one vector to be subsequently used in a downstream model
(e.g a feed-forward neural net layer) for further classification. We tried
with adding 1 or 2 Dense layers but actually the accuracy obtained was not
good to report. The vector obtained was of 768 dimensions and the training set
contains 724 examples. Each example contains more dimensions of a vector than
the number of examples combined. So the data is prone to be over-fitted. So we
resort to use a popular machine learning model known as eXtreme Gradient
Boosting (XGBoost) (Chen and Guestrin, 2016) for classification. XGBoost
gained much popularity recently as a winning solution in several machine
learning competitions. It is actually a library which provides gradient
boosting framework in several languages. We treat each of the dimension of the
vector corresponding to ’[CLS]’ as a separate feature. So we got 724 training
examples each with 768 features. We use a matrix representing this data with
dimension [724,768]. This is given to the XGBoost model for classification.

##  3\. Data

The whole competition task is divided into two categories viz 1. Case Law
Competition (Task 1 and task 2) 2. Statute Law Competition (Task 3 and Task
4). The task organizer’s released the data for all these four tasks defined.
The data is obtained from an existing collection of predominantly Federal
Court of Canada case law. In task-1 the training data consists of 285 case
queries and each query has 200 candidate notice cases. So each query has 9.87
notice cases on average. The training data of task-2 comprises of 181 queries
and each query has an average of 48.59 candidate paragraphs for recognizing
entailment relation. In the training data of this task on an average 1.32
paragraphs have an entailment relation with a query. For the task 3 and 4 data
are borrowed from Japanese civil law articles (translated to English) and the
size of the data for these two tasks are same having 1044 articles.

##  4\. System Description

There were four sub-tasks in this competition. We attempt all of those. In the
following subsections we discussed about the experimental procedures, models
etc. Basically, the pre-processing module for task 1 and task 2 is the same,
which is described in the following paragraph.

Pre-processing: The pre-processing module comprises of two stages. In the
first stage, i.e. for the first model (named as IITPd2v), we perform minimal
amount of pre-processing. All the candidate cases and the base cases are
extracted and the paragraph numbers are removed. In the second stage i.e. for
second model (named as iitpBM25), we remove all the words having length lower
than 3 character and all the numerical values from every documents. We also
remove all the stop-words using NLTK tool. Then we lemmatize the words using
NLTK WordNetLemmatizer 111https://pythonprogramming.net/lemmatizing-nltk-
tutorial/.

###  4.1. Task1

The goal of this task is to explore and evaluate case law retrieval
technologies that are both effective and reliable. The task investigates the
performance of systems that search a set of legal cases that support a
previously unseen case description. The input of this system is a query and
return noticed cases in the given collection as output. We say a case is
’noticed’ with respect to a query iff the case supports the decision of the
query case. In this task, the query case does not include a decision, because
our goal is to determine how accurately a machine can capture decision-
supporting cases for a new case (with no decision).
The training data consists of 285 numbers case queries for this task, and each
query has 200 numbers candidate noticed cases. In the training data of this
task, each case query has an average of 9.87 noticed cases. We propose three
models for this task namely IITP Doc2Vec (IITPd2v), iitpBM25 and  iitpDocBM

####  4.1.1. Models

We have three models to tackle this problem, we will discuss all these in
detail in the following paragraphs.
iitpd2v: In this model, we train the data obtained from stage-1 of pre-
processing module. For training purpose we make use of gensim Doc2Vec with
vector size = 150, window = 10, and with 50 epochs. For every query, we
compute the Doc2Vec similarity between the base case and all of its
candidates. As the task contains variable number of noticed cases, top 10
candidates whose similarity was greater than 90% of average score of top two
are returned. Total 245 documents are returned for 61 base cases. This model
yields the precision, recall, and F-measure as 0.4653, 0.3455, and 0.3965
respectively.
iitpBM25: From this model, we collect the data yielded by the stage-2 of pre-
processing module. We compute BM25 score of all the candidate cases for every
base case using gensim library. Then the top 10 candidates whose similarity is
greater than 90% of average score of top two are returned. Thus, the total of
203 documents are returned. This model produces the precision, recall, and
F-measure as 0.6256, 0.3848, and 0.4765 respectively.
iitpDocBM: In this model we combine the scores obtained from the previous two
models i.e. from the iitpd2v and from the iitpBM25. The make multiplication of
those two scores, so that only documents with high scores on both Doc2Vec and
BM25 are ranked. Top 10 candidates whose similarity was greater than 80% of
average score of top two are returned. In this way total of 201 documents are
returned. We compute the precision, recall, and F-measure and get the scores
as 0.6368, 0.3879, and 0.4821 respectively for this model.

###  4.2. Task2

In this task, the goal is to predict the decision of a new case by entailment
from previous relevant cases. As a simpler version of predicting a decision, a
decision of a new case and a noticed case will be given as a query. Then a
case law textual entailment system must identify which paragraph in the
noticed case entails the decision, by comparing the extracting and the
meanings of the query and paragraph.

####  4.2.1. Model

We propose three models for this task. The name of the models are viz. i.
iitp2D2v ii. iitpBM25 and iii. iitp2DocBM iitp2D2v: For iitp2D2v module, we
take the data from stage-1 of pre-processing module which we train using
gensim Doc2Vec. The specifications for the training is as follows: vector size
= 100, window = 5, and epochs = 50. For every query, Doc2Vec similarity
between the entailed fragment and all of its candidates are computed and the
candidate with highest score are returned. This yields the Precision, Recall,
and F-measure of 0.0455, 0.0444, and 0.0449 respectively.
iitpBM25:For iitpBM25, we take the data from stage-2 of the pre-processing
module. For every entailed fragment BM25 score of all the candidates are
computed by employing gensim library. Then candidate with highest score are
returned. This system yields the Precision, Recall, F-measure scores of
0.7045, 0.6889, and 0.6966 respectively. Please note that we got the highest
Precision score among all the participants in this task. We obtain the F-score
which is the second highest among unique teams.
iitp2DocBM: In iitp2DocBM module, the scores obtained from iitp2D2v and
iitpBM25 are multiplied, so that only candidates with high scores on both the
methods are ranked. Then the candidate with the highest score is returned. We
obtain the Precision, Recall, and F-measure score as 0.6591, 0.6444, and
0.6517 respectively in this task.

###  4.3. Task3

This task investigates the performance of the systems that search a static set
of civil law articles using previously unseen queries. The goal of this task
is to return relevant articles in a collection to a query. We call an article
as ”Relevant” to a query iff the query sentence can be answered as Yes/No,
i.e. entailed from the meaning of the article. If combining the meanings of
more than one article (e.g., ”A”, ”B”, and ”C”) can be answered a query
sentence, then all the articles (”A”, ”B”, and ”C”) are considered as
”Relevant”. If a query can be answered by an article ”D”, and it can also be
answered by another article ”E” independently, we also consider both of these
articles ”D” and ”E” as ”Relevant”. This task requires the retrieval of all
the articles that are relevant to answering a query.

####  4.3.1. Model

We propose four models in this task namely iitpBM25, iitpBM25-L, iitptfidf and
iitptfidf-L. The following points will discuss all the models in detail.
iitpBM25: In this module, for every query BM25 score of all the articles are
computed using gensim library. Then candidates having the highest score are
returned. The total 98 results are retrieved among which 54 are correct. Total
number of correct entries in the corpus was 121. We obtain the Precision,
Recall, and F2-measure of 0.5510, 0.4462, and 0.4639 respectively.
iitpBM25-L: This model is the modified version of the previous one. We extract
the top 100 documents in Descending Order of their score for each query. 109
out of 121 articles in the corpus are retrieved correctly. iitptfidf: In this
model, for every query tf-idf score of all the articles are computed using
sklearn TfIdfVectorizer library222https://scikit-
learn.org/stable/modules/generated/sklearn.feature_extraction.text.TfidfVectorizer.html.
The candidates having the highest scores are returned. Total 98 articles are
retrieved among which 49 are correct. Total number of correct entries in the
corpus was 121. So the precision, recall, and F2-measure provided by this
system are 0.50, 0.4049, and 0.4209 respectively.
iitptfidf-L: This model also the modified version of the previous one. We
retrieve top 100 documents for each query. The model retrieves 108 articles
out of which 121 articles correct.

###  4.4. Task4

Pre-processing: In the Task 4 the pre-processing procedure is as follows. We
extract the data from the given .xml files containing tags named as ’pairs’,
’t1’, and ’t2’. Under a ’pair’ of ’t1’ and ’t2’, we have to find their
entailment relationship. We use the ’id’ attribute under the ’pair’ tag as a
key to store the corresponding article(t1) and the document(t2). For the
training of the model we make use of the training data provided for Task 3 and
tested the system using the test data provided in the Task 4. We extract the
task-4 data for training in similar ways as of task-3 discussed above. Here
along with ’t1’ and ’t2’, we also got the ’label’ attribute under pair tag
which are used as training labels. We gather all ’t1’ś and ’t2’ś along with
their labels to form the training dataset. In Task 4 for training purpose we
got 12 files containing 724 training examples with 353 instances as ’Yes’ and
371 instances ’No’.

####  4.4.1. Model

Here only one model is offered. The description of the model is as follows.
iitpbert: In this task we propose a model named as ’iitpbert’. We make use of
BERT-Base-Uncased (Bidirectional Encoder Representation from
Transformer)(Devlin et al., 2018) model to combat this problem. We provide all
the ’t1’s and ’t2’s as inputs to the BERT model. This task is basically a
sentence pair classification task. So we obtain the final hidden state (i.e.
output of Transformer) corresponding to the first ’[CLS]’ token as the vector
representation for the classification tasks. We check that the combined length
of t1 and t2 should be less than 512 or not. BERT model can handle the maximum
length of 512. We strip the tokens after index 512, if it exceeds 512. We
obtain a matrix of shape N*M where N is number of training examples and M is
the dimension of the output vector of BERT (here M = 768). We try to add dense
layers and other machine learning models like SVM and XgBoost on top of that
output vectors to train them end to end. This yields an accuracy of around 55%
when validated on a held out train set. We choose to stay with the trained
XgBoost classifier and use it to predict labels for the task-4. We submit that
file with predicted file for evaluation. This approach produces an accuracy of
59.18% on the test set for this task.

##  5\. Results and Discussions

The task organizer’s defined various measures to evaluate the participating
systems. The task 1 and task 2 rely on precision, recall and F-measure. In
task 3, in addition to precision, recall and F2-measure they also encourage
the participant to take Mean Average precision (MAP) into account. Whereas, in
task 4 only accuracy is provided as the evaluation metric. The results for the
task1, and task-2 in three methods are shown in the Table 1 and Table 2
respectively.

Table 1. Results obtained by proposed models in Task-1 Models | Precision | Recall | F-measure
---|---|---|---
iitpd2v | 0.4653 | 0.3455 | 0.3965
iitpBM25 | 0.6256 | 0.3848 | 0.4765
iitpDocBM | 0.6368 | 0.3879 | 0.4821
Table 2. Results obtained by proposed models in Task-2 Models | Precision | Recall | F-measure
---|---|---|---
iitpd2v | 0.0455 | 0.0444 | 0.0449
iitpBM25 | 0.7045 | 0.6889 | 0.6966
iitpDocBM | 0.6591 | 0.6444 | 0.6517

The results for the task-3 using four models are shown in the Table 3

Table 3. Results obtained by four proposed models in Task-3 Models | Precision | Recall | F2-measure | MAP
---|---|---|---|---
iitpBM25 | 0.4898 | 0.3967 | 0.4142 | X
iitpBM25-L | X | X | X | 0.5409
iitptfidf | 0.4388 | 0.3554 | 0.3694 | X
iitptfidf-L | X | X | X | 0.5056

Please note that here we have calculated Mean Average Precision (MAP) by
taking top 100 ranked documents into account. The MAP scores are shown in
iitpBM25-L and iitptfidf-L methods in the 5th column of the Table 3. In task 4
participants are encouraged to compute the accuracy to evaluate their
submitted systems. So in this task we compute the accuracy on from our Yes/No
prediction. We obtain the accuracy of 59.18% for the task with our proposed
model.

##  6\. Conclusion and Future work

The paper presents our experiment’s report performed as a part of our
participation in all the four sub-tasks defined in the shared task entitled
COLIEE-2019 organized in ICAIL 2019. We propose various text retrieval methods
(Doc2Vec, BM25, and tf-idf ) for sub-task 1, sub-task 2, and sub-task 3. BERT
model is employed for tackling the sub-task 4. We obtain encouraging results
in all the four sub-tasks and our ranks in all the four task are in a quite
modest position among all the participants in the leaderboard. Our future line
of research could be something as follows:

  * •

Using some good pre-trained vector embedding with sent2vec could produce
better result, so we would link to foster this in future.

  * •

We would like to use BERT-large model (another version of BERT) to explore how
it is performing.

  * •

We would like to do retrospective analysis of the best three models in this
competition to get some insights and for better improvement further.

##  7\. Acknowledgements

Mr Tanik Saikh acknowledges all the co-authors for their contributing efforts.
He is also acknowledging the funding agency to provide the fund for the
registration (if anything needs to be done).

## References

  * (1)
  * Chen and Guestrin (2016) Tianqi Chen and Carlos Guestrin. 2016.  Xgboost: A scalable tree boosting system. In _Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining_. ACM, 785–794.
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding.  _CoRR_ abs/1810.04805 (2018).  arXiv:1810.04805 http://arxiv.org/abs/1810.04805
  * Le and Mikolov (2014) Quoc V. Le and Tomas Mikolov. 2014.  Distributed Representations of Sentences and Documents. In _Proceedings of the 31th International Conference on Machine Learning, ICML 2014, Beijing, China, 21-26 June 2014_. 1188–1196.  http://jmlr.org/proceedings/papers/v32/le14.html
  * Pedregosa et al. (2011) Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011\.  Scikit-learn: Machine learning in Python.  _Journal of machine learning research_ 12, Oct (2011), 2825–2830.
  * Robertson et al. (2009) Stephen Robertson, Hugo Zaragoza, et al. 2009\.  The probabilistic relevance framework: BM25 and beyond.  _Foundations and Trends® in Information Retrieval_ 3, 4 (2009), 333–389.
  * Rumelhart et al. (1988) David E Rumelhart, Geoffrey E Hinton, Ronald J Williams, et al. 1988\.  Learning representations by back-propagating errors.  _Cognitive modeling_ 5, 3 (1988), 1.