# The homogeneous spectrum of a $\mathbb{Z}_{2}$-graded commutative ring

Mohamed Aqalmoun  Department of Mathematics, Higher Normal School, Sidi
Mohamed Ben Abdellah University, Fez, Morocco<EMAIL_ADDRESS>
###### Abstract.

Let $\mathbb{Z}_{2}:=\mathbb{Z}/2\mathbb{Z}$ be the additive group with two
elements. In this article, we focus only on $\mathbb{Z}_{2}$-graded
commutative ring i.e commutative ring $R$ such that $R=R_{0}\oplus R_{1}$ as
Abelian group and $R_{i}R_{j}\subseteq R_{i+j}$ for all
$i,j\in\mathbb{Z}_{2}$. Our main goals is to establish a strong relation
between $\mathbb{Z}_{2}$-graded prime ( maximal ) ideals of $R$ and prime (
maximal) ideals of $R_{0}$, for instance, it is showed that, the
$\mathbb{Z}_{2}$-graded spectrum of $R$ is homeomorphic to the spectrum of
$R_{0}$ with respect to the Zariski topologies.
Keywords: $\mathbb{Z}_{2}$-graded ring, $\mathbb{Z}_{2}$-graded prime ideal,
$\mathbb{Z}_{2}$-graded maximal ideal, $\mathbb{Z}_{2}$-graded field.
MSC(2010): Primary: 13A02 ; Secondary: 13A99.

##  1\. Introduction

Let $G$ be an Abelian group with identity $e$ and $R$ be a commutative ring
with unit. Then $R$ is called a $G$-graded ring if there exist additive
subgroups $R_{g}$ of $R$ indexed by elements $g\in G$ such that
$R=\bigoplus_{g\in G}R_{g}$ and $R_{g}R_{g^{\prime}}\subseteq R_{gg^{\prime}}$
for all $g,g^{\prime}\in G$, where $R_{g}R_{g^{\prime}}$ consists of the
finite sums of ring products $ab$ with $a\in R_{g}$ and $b\in R_{g^{\prime}}$.
The elements of $R_{g}$ are called homogeneous elements of $R$ of degree $g$.
The homogeneous elements of the ring $R$ are denoted by $h(R)$, i.e.
$h(R)=\cup_{g\in G}R_{g}$. If $a\in R$, then the element $a$ can be written
uniquely as $\sum_{g\in G}a_{g}$, where $a_{g}\in R_{g}$ is called the
$g$-component of $a$ in $R_{g}$.

Let $R$ be a $G$-graded ring and $I$ be an ideal of $R$. Then $I$ is called a
graded ideal of $R$ if $I=\bigoplus_{g\in G}(I\cap R_{g})$. If $I$ is a
$G$-graded ideal of $R$, then the quotient ring $R/I$ is a $G$-graded ring.
Indeed $R/I=\bigoplus_{g\in G}(R/I)_{g}$ where $(R/I)_{g}=(R_{g}+I)/I=\\{x+I\
/x\in R_{g}\\}$. Let $S\subseteq h(R)$ be a multiplicatively closed subset of
$R$. Then the ring of fractions $S^{-1}R$ is a $G$-graded ring. Indeed
$S^{-1}R=\bigoplus_{g\in G}(S^{-1}R)_{g}$ where $(S^{-1}R)_{g}=\\{r/s\ \ /\
r\in h(R),s\in S\text{ and }\deg r=g\deg s\\}$.

A $G$-graded ideal $P$ of a $G$-graded ring $R$ is called $G$-graded prime
ideal of $R$ if $P\neq R$ and if whenever $r$ and $s$ are homogeneous elements
of $R$ such that $rs\in P$, then either $r\in P$ or $s\in P$. The $G$-graded
spectrum or homogeneous spectrum of $R$ is the set of all $G$-graded prime
ideals of $R$, it is denoted by $G\mathrm{Spec}R$ . A graded ideal $M$ of $R$
is said to be graded maximal ideal of $R$ if $M\neq R$ and if $J$ is a graded
ideal of $R$ such that $M\subseteq J\subseteq R$, then $J=M$ or $J=R$, the set
of all $G$-graded maximal ideals of $R$ is denoted by $G\mathrm{Max}R$.

For a commutative ring $R$ and ideal $I$ of $R$, the variety of $I$ is the
subset $V(I):=\\{P\in\mathrm{Spec}R\ /I\subseteq P\\}$. Then the collection
$\\{V(I)\ /I\text{ ideal of }R\\}$ satisfies the axioms for the closed sets of
a topology on $\mathrm{Spec}R$, called the Zariski topology. Note that for
each ideal $I$, the set $\mathrm{Spec}R-V(I)$ is an open subset, it is denoted
$D(I)$. Recall that, the collection $D(f)$ where $f$ runs through $R$ forms a
basis of opens. Similarly, let $R$ be a $G$-graded commutative ring and $I$ a
$G$-graded ideal of $R$, the $G$-variety of $I$ is the subset
$V_{G}(I):=\\{P\in G\mathrm{Spec}R\ /I\subseteq P\\}$. The collection
$\\{V_{G}(I)\ /I\text{ is a }G\text{-graded ideal of }R\\}$ satisfies the
axioms for the closed sets of a topology on $G\mathrm{Spec}R$, called the
Zariski topology, see [7], [10], [11]. Note that, the collection $D(r)$ where
$r$ runs through $h(R)$ forms a basis of opens for the Zariski topology.

Let $G=\mathbb{Z}_{2}=\mathbb{Z}/2\mathbb{Z}$ and $R$ be
$\mathbb{Z}_{2}$-graded commutative ring. Then $R=R_{0}\oplus R_{1}$, where
$R_{0}$ is a subring of $R$ and $R_{1}$ is an $R_{0}$-module such that
$R_{1}R_{1}\subseteq R_{0}$. Note that $R_{1}R_{1}$ is an ideal of $R_{0}$, we
denote it by $R_{1}^{2}$. Similarly $R_{1}^{3}$ is an $R_{0}$-submodule of
$R_{1}$. If $P$ is a $\mathbb{Z}_{2}$-graded prime ideal of $R$, then $P\cap
R_{0}$ is a prime ideal of $R_{0}$. This yields a natural map
$\mathbb{Z}_{2}\mathrm{Spec}R\to\mathrm{Spec}R_{0}$.

######  Example 1.1.

Let $A$ be a commutative ring, then the polynomial ring $R=A[X]$ is
$\mathbb{Z}_{2}$-graded ring with the grading $R_{0}=\oplus_{n}A(X^{2n})$ and
$R_{1}=\oplus_{n}A(X^{2n+1})$. In fact every $\mathbb{N}$-graded (respectively
$\mathbb{Z}$-graded ) commutative ring $R$ has a natural
$\mathbb{Z}_{2}$-grading with $R_{0}^{\prime}=\oplus_{n}R_{2n}$ and
$R^{\prime}_{1}=\oplus_{n}R_{2n+1}$.

######  Example 1.2.

Let $A$ be a commutative ring and $X=(X_{\alpha})_{\alpha}$ be a set of
indeterminate over $A$ and $(a_{\alpha\beta})_{\alpha,\beta}$ be a family of
elements of $A$. Consider the ideal $I$ generated by all elements
$X_{\alpha}X_{\beta}-a_{\alpha\beta}$. Then $R:=\dfrac{A[X]}{I}$ is a
$\mathbb{Z}_{2}$-graded ring with the grading $R_{0}=A$ and $R_{1}$ is the
$A$-submodule of $R$ generated by all $X_{\alpha}+I$.

######  Remark 1.3.

Matrix ring representation of a $\mathbb{Z}_{2}$-graded commutative ring. Let
$R$ be a $\mathbb{Z}_{2}$-graded commutative ring. If
$r=r_{0}+r_{1},s=s_{0}+s_{1}\in R=R_{0}\oplus R_{1}$, then
$rs=(r_{0}s_{0}+r_{1}s_{1})+(r_{0}s_{1}+s_{0}r_{1})$, that is
$(rs)_{0}=r_{0}s_{0}+r_{1}s_{1}$ and $(rs)_{1}=r_{0}s_{1}+s_{0}r_{1}$. So, one
can identifies $R$ with the matrix ring

$R^{\prime}=\left\\{\begin{pmatrix}r_{0}&r_{1}\\\ r_{1}&r_{0}\end{pmatrix}\ /\
r_{0}\in R_{0},\ r_{1}\in R_{1}\right\\}$

The purpose of this paper is to establish a strong relation between
$\mathbb{Z}_{2}$-graded prime ideals of a $\mathbb{Z}_{2}$-graded commutative
ring $R$ and prime ideals of $R_{0}$ via the natural map
$\mathbb{Z}_{2}\mathrm{Spec}R\to\mathrm{Spec}R_{0}$. It is proved that, this
map is one to one, and in fact it is a homeomorphism with respect to the
Zariski topologies. In fact, to fulfill this goal, we give the form of a
$\mathbb{Z}_{2}$-graded prime ideal $P$ from its degree zero homogeneous part
$P\cap R_{0}$, their expression lies on the fact that $P\cap R_{0}\in
V(R_{1}^{2})$ or $P\cap R_{0}\in D(R_{1}^{2})$. Precisely, every
$\mathbb{Z}_{2}$-graded ideal can be discovered from its degree zero
homogeneous part.

##  2\. $\mathbb{Z}_{2}$-Graded ideals

We start this section with some useful properties for $\mathbb{Z}_{2}$-graded
ideals of a $\mathbb{Z}_{2}$-graded commutative ring. We will see that, when
$R$ is a strongly $\mathbb{Z}_{2}$-graded commutative ring, those ideals are
no thing but as ideals of $R_{0}$.

######  Proposition 2.1.

Let $R$ be a $\mathbb{Z}_{2}$-graded ring and $J$ be an ideal of $R$. The
following statements are equivalent.

  1. (1)

$J$ is a $\mathbb{Z}_{2}$-graded ideal of $R$.

  2. (2)

$J=I+R^{\prime}$ where $I$ is an ideal of $R_{0}$ and $R^{\prime}$ is an
$R_{0}$-submodule of $R_{1}$ with $IR_{1}\subseteq R^{\prime}$ and
$R_{1}R^{\prime}\subseteq I$.

###### Proof.

Let $J$ be a $\mathbb{Z}_{2}$-graded ideal of $R$. Then $J=I+R^{\prime}$ where
$I:=J\cap R_{0}$ is an ideal of $R_{0}$ and $R^{\prime}:=J\cap R_{1}$ is an
$R_{0}$-submodule of $R_{1}$. If $x\in R_{1}$ and $y\in R^{\prime}$, then
$xy\in J$ is an element of degree $0$, so $xy\in J\cap R_{0}=I$. It follows
that $R_{1}R^{\prime}\subseteq I$. If $i\in I$ and $y\in R_{1}$, then $iy\in
J$ is an element of degree $1$, so $iy\in J\cap R_{1}=R^{\prime}$. Therefore
$IR_{1}\subseteq R^{\prime}$.

Let $J=I+R^{\prime}$ where $I$ is an ideal of $R_{0}$ and $R^{\prime}$ is an
$R_{0}$-submodule of $R_{1}$ with $IR_{1}\subseteq R^{\prime}$ and
$R_{1}R^{\prime}\subseteq R^{\prime}$. Clearly $J$ is a subgroup of $R$. Let
$x=i+r^{\prime}\in I+R^{\prime}$ and $y=a+r\in R=R_{0}+R_{1}$. Then
$xy=(ia+r^{\prime}r)+(ir+ar^{\prime})\in I+R^{\prime}=J$. ∎

If $R^{\prime}$ is an $R_{0}$-submodule of $R_{1}$. Then the residual
$R^{\prime}$ by $R_{1}$ is defined as follow

$(R^{\prime}:_{R_{0}}R_{1})=\\{a\in R_{0}\ /aR_{1}\subseteq R^{\prime}\\}$

It is an ideal of $R_{0}$. So, the condition $IR_{1}\subseteq R^{\prime}$ is
equivalent to $I\subseteq(R^{\prime}:_{R_{0}}R_{1})$.

######  Remark 2.2.

As a consequence, we have the two classes of $\mathbb{Z}_{2}$-graded ideals,
which will appear for our discussion.

  1. (1)

If $I$ is an ideal of $R_{0}$, then $I+IR_{1}$ is a $\mathbb{Z}_{2}$-graded
ideal of $R$. Indeed, $IR_{1}\subseteq IR_{1}$ and
$R_{1}(IR_{1})=IR_{1}R_{1}\subseteq IR_{0}=I$.

  2. (2)

If $R^{\prime}$ is a submodule of $R_{1}$, then
$(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ is a $\mathbb{Z}_{2}$-graded ideal of
$R$. Indeed; $(R^{\prime}:_{R_{0}}R_{1})R_{1}\subseteq R^{\prime}$ and
$(R^{\prime}R_{1})R_{1}\subseteq R^{\prime}R_{0}\subseteq R^{\prime}$ that is
$R^{\prime}R_{1}\subseteq(R^{\prime}:_{R_{0}}R_{1})$.

######  Definition 2.3.

A $G$-graded commutative ring is called strongly graded if
$R_{g}R_{g^{\prime}}=R_{gg^{\prime}}$ for all $g,g^{\prime}\in G$.

Note that, if $R$ is a $\mathbb{Z}_{2}$-graded commutative ring, then $R$ is
strongly graded if and only if $R_{1}^{2}=R_{0}$.

The following result establish a relation between graded ideals of a strongly
$\mathbb{Z}_{2}$-graded commutative ring $R$ and ideals of $R_{0}$.

######  Proposition 2.4.

Let $R$ be a strongly $\mathbb{Z}_{2}$-graded commutative ring. The following
statements hold.

  1. (1)

$R_{1}$ is a finitely generated $R_{0}$-module.

  2. (2)

$R_{1}$ is a multiplication module.

  3. (3)

If $I$ and $I^{\prime}$ are ideals of $R_{0}$ such that $IR_{1}\subseteq
I^{\prime}R_{1}$, then $I\subseteq I^{\prime}$.

  4. (4)

The $\mathbb{Z}_{2}$-graded ideals of $R$ are $I+IR_{1}$ where $I$ is an ideal
of $R_{0}$.

###### Proof.

  1. (1)

Since $R_{1}^{2}=R_{0}$, there exist $a_{i},b_{i}\in R_{1}$ such that
$1=\displaystyle\sum_{i=1}^{n}a_{i}b_{i}$. Then $R_{1}=(a_{1},\ldots,a_{n})$.
Indeed, if $x\in R_{1}$, then $x=\displaystyle\sum_{i=1}^{n}(xb_{i})a_{i}$.
The elements $xb_{i}$ are of degree zero, so they are in $R_{0}$. Therefore
$R_{1}=(a_{1},\ldots,a_{n})$.

  2. (2)

Let $R^{\prime}$ be an $R_{0}$-submodule of $R_{1}$. Clearly,
$(R^{\prime}:_{R_{0}}R_{1})R_{1}\subseteq R^{\prime}$. Let $x\in R^{\prime}$,
then $x=\displaystyle\sum_{i=1}^{n}(xb_{i})a_{i}$. Each $xb_{i}$ is an element
of $(R^{\prime}:_{R_{0}}R_{1})$, in fact if $c\in R_{1}$, then
$(xb_{i})c=(b_{i}c)x\in R^{\prime}$ since $b_{i}c\in R_{0}$ and $x\in
R^{\prime}$. Thus $R^{\prime}=(R^{\prime}:_{R_{0}}R_{1})R_{1}$. Therefore
$R_{1}$ is a multiplication module.

  3. (3)

If $IR_{1}\subseteq I^{\prime}R_{1}$, then $IR_{1}^{2}\subseteq
I^{\prime}R_{1}^{2}$, so $I\subseteq I^{\prime}$.

  4. (4)

Let $J$ be a $\mathbb{Z}_{2}$-graded ideal of $R$, then $J=I+R^{\prime}$ where
$I$ is an ideal of $R_{0}$ and $R^{\prime}$ is an $R_{0}$-submodule of $R_{1}$
such that $IR_{1}\subseteq R^{\prime}$ and $R^{\prime}R_{1}\subseteq I$. Since
$R_{1}$ is a multiplication module, there is an ideal $I^{\prime}$ of $R_{0}$
such that $R^{\prime}=I^{\prime}R_{1}$. But $IR_{1}\subseteq
R^{\prime}=I^{\prime}R_{1}$, so $I\subseteq I^{\prime}$. On other hand,
$I^{\prime}=I^{\prime}R_{1}^{2}=R^{\prime}R_{1}\subseteq I$. Thus
$I=I^{\prime}$ and $J=I+IR_{1}$. Conversely, if $J=I+IR_{1}$ where $I$ is an
ideal of $R_{0}$, then $J$ is a $\mathbb{Z}_{2}$-graded ideal (see the Remark
2.2).

∎

######  Remark 2.5.

Let $R$ be a strongly $\mathbb{Z}_{2}$-graded ring. The map $J\mapsto J\cap
R_{0}$ realize a bijection between the set of $\mathbb{Z}_{2}$-graded ideals
of $R$ and the set of ideals of $R_{0}$.

One can show that, for a strongly $\mathbb{Z}_{2}$-graded commutative ring,
the $\mathbb{Z}_{2}$-graded ideals of $R$ are
$(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ where $R^{\prime}$ is an
$R_{0}$-submodule of $R_{1}$. It follows that the map $J\mapsto J\cap R_{1}$
realize a bijection between the set of $\mathbb{Z}_{2}$-graded ideals of $R$
and the set of $R_{0}$-submodule $R_{1}$.

##  3\. $\mathbb{Z}_{2}$-Graded prime ideals

######  Definition 3.1.

Let $R$ be a $G$-graded commutative ring and $P$ be a $G$-graded ideal of $R$.
Then $P$ is called $G$-graded prime ideal (or homogeneous prime ideal) of $R$
if $P\neq R$ and if whenever $r$ and $s$ are homogeneous elements of $R$ such
that $rs\in P$, then either $r\in P$ or $s\in P$

######  Remark 3.2.

A $G$-graded prime ideal of a $G$-graded ring $R$ is not necessarily a prime
ideal. Let $R=\mathbb{Z}+i\mathbb{Z}$ (where $i^{2}=-1$) as a
$\mathbb{Z}_{2}$-graded ring with $R_{0}=\mathbb{Z}$ and $R_{1}=i\mathbb{Z}$.
It is easy to see that $P:=2\mathbb{Z}+i2\mathbb{Z}$ is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$. But $P$ is not a prime ideal of
$R$, since $(3+i)(3-i)=10\in P$ and $3+i,3-i\not\in P$.

Recall from [3] that if $A$ is a commutative ring and $M$ is an $A$-module,
then a prime submodule of $M$ is a proper submodule $N$ of $M$ having the
property that $am\in N$ implies that $a\in(N:_{S}M)$ or $m\in N$ for each
$a\in S$ and $m\in M$.

The following Theorem describes the $\mathbb{Z}_{2}$-graded prime ideals of
$R$ from prime ideals of $R_{0}$ and prime submodules of $R_{1}$.

######  Theorem 3.3.

Let $Q$ be a $\mathbb{Z}_{2}$-graded ideal of $R$. Then $Q$ is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$ if and only if

  1. (1)

either $Q=p+R_{1}$ where $p\in V(R_{1}^{2})$, or

  2. (2)

$Q=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ where $R^{\prime}$ is a prime
submodule of $R_{1}$ such that $R_{1}^{3}\not\subseteq R^{\prime}$.

###### Proof.

($\Leftarrow$): Let $p$ be a prime ideal of $R_{0}$ containing $R_{1}^{2}$ .
Then $p+R_{1}$ is $\mathbb{Z}_{2}$-graded ideal of $R$ and is a prime ideal of
$R$ since $R/(p+R_{1})=R_{0}/p$ is an integral domain, so it is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$.

Let $R^{\prime}$ be prime submodule of $R_{1}$ such that
$R_{1}^{3}\not\subseteq R^{\prime}$. Then
$(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ is a graded ideal of $R$. Let $a,b\in
R_{0}$ such that $ab\in(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$, then
$ab\in(R^{\prime}:_{R_{0}}R_{1})$. Since $R^{\prime}$ is a prime submodule of
$R_{1}$, $(R^{\prime}:_{R_{0}}R_{1})$ is a prime ideal of $R_{0}$. Thus
$a\in(R^{\prime}:_{R_{0}}R_{1})$ or $b\in(R^{\prime}:_{R_{0}}R_{1})$. If $a\in
R_{0}$ and $r\in R_{1}$ such that
$ar\in(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$, then $ar\in R^{\prime}$. It
follows that $a\in(R^{\prime}:_{R_{0}}R_{1})$ or $r\in R^{\prime}$ since
$R^{\prime}$ is a prime submodule of $R_{1}$. Now, let $r,r^{\prime}\in R_{1}$
such that $rr^{\prime}\in(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$. Assume that
$r,r^{\prime}\not\in R^{\prime}$. We have
$rr^{\prime}\in(R^{\prime}:_{R_{0}}R_{1})$, so
$rr^{\prime}R_{1}^{2}\subseteq(R^{\prime}:_{R_{0}}R_{1})$, that is
$(rR_{1})(r^{\prime}R_{1})\subseteq(R^{\prime}:_{R_{0}}R_{1})$. Since $rR_{1}$
and $r^{\prime}R_{1}$ are ideals of $R_{0}$ and $(R^{\prime}:_{R_{0}}R_{1})$
is a prime ideal of $R_{0}$, it follows that
$rR_{1}\subseteq(R^{\prime}:_{R_{0}}R_{1})$ or
$r^{\prime}R_{1}\subseteq(R^{\prime}:_{R_{0}}R_{1})$, that is
$rR_{1}^{2}\subseteq R^{\prime}$ or $r^{\prime}R_{1}^{2}\subseteq R^{\prime}$.
Since $r,r^{\prime}\not\in R^{\prime}$ and $R^{\prime}$ is a prime submodule,
we get $R_{1}^{2}\subseteq(R^{\prime}:_{R_{0}}R_{1})$, so that
$R_{1}^{3}\subseteq R^{\prime}$, a contradiction. Thus $r\in R^{\prime}$ or
$r^{\prime}\in R^{\prime}$.

($\Rightarrow$): Let $Q$ be a graded prime ideal of $R$. Then, by Proposition
2.1, $Q=p+R^{\prime}$ where $p$ is an ideal of $R_{0}$ and $R^{\prime}$ is a
submodule of $R_{1}$ such that $pR_{1}\subseteq R^{\prime}$ and
$R^{\prime}R_{1}\subseteq p$. It is easy to see that $p$ is a prime ideal of
$R_{0}$. Indeed, if $a,b\in R_{0}$ such that $ab\in p$, then $ab\in Q$, so
$a\in Q$ or $b\in Q$. That is $a\in p$ or $b\in p$.

First case, $R_{1}\subseteq Q$: In this case $R_{1}\subseteq R^{\prime}$ that
is $R^{\prime}=R_{1}$. Therefore $Q=p+R_{1}$. Moreover $R_{1}^{2}\subseteq
Q\cap R_{0}=p$. That is $p$ is a prime ideal of $R_{0}$ containing
$R_{1}^{2}$.

Second case $R_{1}\not\subseteq Q$: In this case $R^{\prime}\neq R_{1}$. Let
$a\in R_{0}$ and $r\in R_{1}$ such that $ar\in R^{\prime}$. We see that $ar\in
Q$, so $a\in Q$ or $r\in Q$ that is $a\in
p\subseteq(R^{\prime}:_{R_{0}}R_{1})$ or $r\in R^{\prime}$. It follows that
$R^{\prime}$ is a prime submodule of $R_{1}$. Clearly,
$p\subseteq(R^{\prime}:_{R_{0}}R_{1})$. Let $a\in(R^{\prime}:_{R_{0}}R_{1})$.
Since $R^{\prime}\neq R_{1}$, there exist $b\in R_{1}$ such that $b\not\in
R^{\prime}$. But $ab\in R^{\prime}$, so that $ab\in Q$. Since $Q$ is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$ and $a,b$ are homogeneous elements,
$a\in Q$ or $b\in Q$. Thus $a\in p=Q\cap R_{0}$ since $b\not\in R^{\prime}$.
It follows that $p=(R^{\prime}:_{R_{0}}R_{1})$. Now, if $R_{1}^{3}\subseteq
R^{\prime}$, then $R_{1}^{2}\subseteq(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}=Q$.
Pick $b\in R_{1}-R^{\prime}$, then $b^{2}\in Q$, so $b\in Q\cap
R_{1}=R^{\prime}$, a contradiction. Thus $R_{1}^{3}\not\subseteq R^{\prime}$.
∎

In oder to determine the homogeneous spectrum of a strongly
$\mathbb{Z}_{2}$-graded commutative ring, we need the following result.

######  Proposition 3.4.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring.

  1. (1)

If $p$ is a prime ideal of $R_{0}$ such that $R_{1}^{2}\not\subseteq p$, then
$(pR_{1}:_{R_{0}}R_{1})=p$.

  2. (2)

Let $Q$ be a $\mathbb{Z}_{2}$-graded ideal of $R$ such that $R_{1}^{2}+Q\cap
R_{0}=R_{0}$. Then $Q$ is a graded prime ideal if and only if $Q=p+pR_{1}$
where $p$ is a prime ideal of $R_{0}$.

###### Proof.

  1. (1)

Clearly, $p\subseteq(pR_{1}:_{R_{0}}R_{1})$. Let $a\in(pR_{1}:_{R_{0}}R_{1})$,
then $aR_{1}\subseteq pR_{1}$, so $aR_{1}^{2}\subseteq pR_{1}^{2}\subseteq p$,
so that $a\in p$ since $R_{1}^{2}\not\subseteq p$.

  2. (2)

Since $R_{1}^{2}+Q\cap R_{0}=R_{0}$,
$1=\displaystyle\sum_{i=1}^{n}a_{i}b_{i}+\alpha$ where $a_{i},b_{i}\in R_{1}$
and $\alpha\in Q\cap R_{0}$. If $Q$ is a graded prime ideal of $R$, then
$R_{1}\not\subseteq Q$ since $R_{1}^{2}\not\subseteq Q$. By Theorem 3.3,
$Q=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ where $R^{\prime}$ is a prime
submodule of $R_{1}$ with $R_{1}^{3}\not\subseteq R_{1}$. We see that
$p:=(R^{\prime}:_{R_{0}}R_{1})=Q\cap R_{0}$ is a prime ideal and
$pR_{1}\subseteq R^{\prime}$. Let $r\in R^{\prime}$, then
$r=\displaystyle\sum_{i=1}^{n}(b_{i}r)a_{i}+\alpha r$. If $r^{\prime}\in
R_{1}$, then $(b_{i}r)r^{\prime}=(b_{i}r^{\prime})r\in R^{\prime}$. It follows
that $b_{i}r\in(R^{\prime}:_{R_{0}}R_{1})=p$. So
$r=\displaystyle\sum_{i=1}^{n}(b_{i}r)a_{i}+\alpha r\in pR_{1}$. Thus
$R^{\prime}=pR_{1}$, that is $Q=p+pR_{1}$. Conversely, assume that
$Q=p+pR_{1}$ where $p$ is a prime ideal of $R_{0}$, in fact $p=Q\cap R_{0}$.
Since $R_{1}^{2}+p=R_{0}$, $R_{1}^{2}\not\subseteq p$. By the previous result
$p=(pR_{1}:_{R_{0}}R_{1})$. Now, it is enough to show that $pR_{1}$ is a prime
submodule of $R_{1}$ and $R_{1}^{3}\not\subseteq pR_{1}$. Let $a\in R_{0}$ and
$r\in R_{1}$ such that $ar\in pR_{1}$. We have
$r=\displaystyle\sum_{i=1}^{n}(b_{i}r)a_{i}+\alpha r$, so that
$ar=\displaystyle\sum_{i=1}^{n}(ab_{i}r)a_{i}+\alpha ar$. We see that
$a(b_{i}r)=(ar)b_{i}\in(pR_{1})R_{1}\subseteq p$, so $a\in p$ or $b_{i}r\in p$
for all $i$. If $a\in p$, then $a\in(pR_{1},R_{1})$. If $a\not\in p$, then
$r=\displaystyle\sum_{i=1}^{n}(b_{i}r)a_{i}+\alpha r\in pR_{1}$. On other hand
$pR_{1}\neq R_{1}$ since $(pR_{1}:R_{1})=p\neq R_{0}$. Thus $pR_{1}$ is a
prime submodule of $R_{1}$. If $R_{1}^{3}\subseteq pR_{1}$, then
$(R_{1}^{2})^{2}\subseteq pR_{1}^{2}\subseteq p$, and in this case
$R_{1}^{2}\subseteq p$, which is not compatible with the hypothesis. It
follows that $p+pR_{1}$ is a $\mathbb{Z}_{2}$-graded prime ideal of $R$.

∎

Now, we have the following Corollary, which describes the homogeneous spectrum
of a strongly $\mathbb{Z}_{2}$-graded commutative ring.

######  Corollary 3.5.

Let $R$ be a strongly $\mathbb{Z}_{2}$-graded commutative ring. Then

$\mathbb{Z}_{2}\mathrm{Spec}R=\\{p+pR_{1}\ /\ p\in\mathrm{Spec}R\\}.$

###### Proof.

This follows from the previous Proposition and the fact that, for every prime
ideal $p$ of $R_{0}$, $R_{1}^{2}+p=R_{0}$. ∎

The following Corollary point out a class of $\mathbb{Z}_{2}$-graded
commutative ring for which prime ideals and $\mathbb{Z}_{2}$-graded prime
ideals coincide.

######  Corollary 3.6.

Let $R$ be a $\mathbb{Z}_{2}$-graded ring such that
$R_{1}^{2}\subseteq\sqrt{0}$. Then

  1. (1)

The $\mathbb{Z}_{2}$-graded prime of $R$ are $p+R_{1}$ where $p$ is a prime
ideal of $R_{0}$.

  2. (2)

An ideal of $R$ is $\mathbb{Z}_{2}$-graded prime if and only if it is prime.

###### Proof.

  1. (1)

Immediate from 3.3.

  2. (2)

If $Q$ is a $\mathbb{Z}_{2}$-graded prime ideal of $R$, then
$R_{1}R_{1}\subseteq Q$, so $R_{1}\subseteq Q$. By Theorem 3.3, $Q=p+R_{1}$
where $p$ is a prime ideal of $R_{0}$. It is easy to see $R/Q=R_{0}/p$ is an
integral domain, so $Q$ is a prime ideal of $R$. Let $P$ be any prime ideal of
$R$ and set $p=P\cap R_{0}$. Since $R_{1}^{2}\subseteq\sqrt{0}\subseteq
p\subseteq Q$, $R_{1}\subseteq Q$. Therefore $p+R_{1}\subseteq Q$. If
$z=a+r\in Q$ where $a\in R_{0}$ and $r\in R_{1}$, then $a=z-r\in Q\cap
R_{0}=p$, so $z\in p+R_{1}$. It follows that $Q=p+R_{1}$ is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$.

∎

######  Example 3.7.

Let $R$ be a commutative ring and $M$ be an $R$-module. Consider the trivial
ring extension $S=R\propto M$ as $\mathbb{Z}_{2}$-graded ring with $S_{0}=R$
and $S_{1}=M$. Clearly, $S_{1}^{2}=0$. Therefore the $\mathbb{Z}_{2}$-graded
prime ideals and prime ideals coincides.

An immediate consequence of the Corollary 3.5 is as follow; if $R$ is a
strongly $\mathbb{Z}_{2}$-graded commutative ring, then the homogeneous
spectrum of $R$ and the prime spectrum of $R_{0}$ are homeomorphic with
respect to the Zariski topologies. The next Theorem extend this result by
dropping the strongly graded hypothesis.

######  Theorem 3.8.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. Then the map
$\varphi:\mathbb{Z}_{2}\mathrm{Spec}R\to\mathrm{Spec}R_{0}$, $\varphi(P)=P\cap
R_{0}$ is an homeomorphism of topological spaces with respect to the Zariski
topologies.

###### Proof.

Let $P,Q\in\mathbb{Z}_{2}\mathrm{Spec}R$ with $P\cap R_{0}=Q\cap R_{0}$. If
$x\in P\cap R_{1}$, then $x^{2}\in P\cap R_{0}=Q\cap R_{0}$, so $x^{2}\in Q$,
hence $x\in Q\cap R_{1}$. It follows that $P=P\cap R_{0}+P\cap R_{1}\subseteq
Q\cap R_{0}+Q\cap R_{1}=Q$. By the same argument we have $Q\subseteq P$, thus
$P=Q$.

Let $p\in\mathrm{Spec}R_{0}$. Consider $R^{\prime}=\\{x\in R_{1}\
/xR_{1}\subseteq p\\}$. Then $R^{\prime}$ is clearly an $R_{0}$-submodule of
$R_{1}$.
First case, $R_{1}^{2}\subseteq p$. By Theorem 3.3, $p+R_{1}$ is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$ and $\varphi(p+R_{1})=p$.
Second case, $R_{1}^{2}\not\subseteq p$. Note that using the Theorem 3.3, it
enough to show that $R^{\prime}$ is a prime submodule of $R_{1}$ with
$R_{1}^{3}\not\subseteq R^{\prime}$ and $p=(R^{\prime}:_{R_{0}}R_{1})$. If
$R_{1}^{3}\subseteq R^{\prime}$, then $R_{1}^{2}R_{1}^{2}=R^{4}\subseteq p$,
since $p$ is prime, we get $R_{1}^{2}\subseteq p$, a contradiction, so
$R_{1}^{3}\not\subseteq R^{\prime}$. Let $a\in R_{0}$ and $x\in R_{1}$ such
that $ax\in R^{\prime}$. Then $axR_{1}\subseteq p$. Since $p$ is a prime
ideal, $a\in p$ or $xR_{1}\subseteq p$. If $xR_{1}\subseteq p$, then $x\in
R^{\prime}$. If $a\in p$, then $(aR_{1})R_{1}\subseteq pR_{1}^{2}\subseteq p$,
so $aR_{1}\subseteq R^{\prime}$ that is $a\in(R^{\prime}:_{R_{0}}R_{1})$.
Therefore $R^{\prime}$ is prime submodule of $R_{1}$. Next, we show that
$(R^{\prime}:_{R_{0}}R_{1})=p$. We have $(pR_{1})R_{1}=pR_{1}^{2}\subseteq p$,
so $pR_{1}\subseteq R^{\prime}$, that is
$p\subseteq(R^{\prime}:_{R_{0}}R_{1})$. Now, let
$a\in(R^{\prime}:_{R_{0}}R_{1})$. Then $aR_{1}\subseteq R^{\prime}$, that is
$aR_{1}^{2}=(aR_{1})R_{1}\subseteq p$. Since $p$ is a prime ideal of $R_{0}$
and $R_{1}^{2}\not\subseteq p$, we get $a\in p$. Thus
$(R^{\prime}:_{R_{0}}R_{1})=p$. Finally, $p+R^{\prime}$ is a
$\mathbb{Z}_{2}$-graded prime ideal of $R$ and $\varphi(p+R^{\prime})=p$.

If $a\in R_{0}$, then $\varphi^{-1}(V(a))=V(a)$. Thus $\varphi$ is a
continuous map.

If $r\in R$ is a homogeneous elements, then $\varphi(V(r))=V(r^{2})$. Thus
$\varphi^{-1}$ is also a continuous map. It follows that $\varphi$ is an
homeomorphism with respect to the Zariski topologies. ∎

Next, we list a consequences of the previous Theorem. Recall that, a
$G$-graded commutative ring has Noetherian homogeneous spectrum if
$G\mathrm{Spec}R$ is a Noetherian topological space with respect to the
Zariski topology. The homogeneous dimension of $R$, denoted by $h\dim R$, is
the maximum length $n$ of a chain $P_{0}\subset P_{1}\subset\ldots\subset
P_{n}$ of homogeneous prime ideals of $R$.

######  Corollary 3.9.

Let $R$ be $\mathbb{Z}_{2}$-graded commutative ring. The following statements
hold.

  1. (1)

$R$ has Noetherian homogeneous spectrum if and only if $R_{0}$ has Noetherian
spectrum.

  2. (2)

$h\dim R=\dim R_{0}$.

###### Proof.

  1. (1)

Immediate.

  2. (2)

In $P_{0}\subset P_{1}\subset\ldots\subset P_{n}$ is a chain of homogeneous
ideals of $R$, then
$\varphi(P_{0})\subset\varphi(P_{1})\subset\ldots\subset\varphi(P_{n})$ is a
chain of prime ideals of $R_{0}$. Conversely, if $p_{0}\subset
p_{1}\subset\ldots\subset p_{n}$ is a chain of prime ideals of $R_{0}$. For
each $i$, $p_{i}=\varphi(P_{i})$ for some
$P_{i}\in\mathbb{Z}_{2}\mathrm{Spec}R$, we see that $P_{0}\subset
P_{1}\subset\ldots\subset P_{n}$. it follows that $h\dim R=\dim R_{0}$.

∎

We close this section by a discuss of the homogeneous radical of a graded
ideal. Recall that, if $I$ is a $G$-graded ideal of a $G$-graded commutative
ring $R$, then the graded radical of $I$, denoted by $\mathrm{Grad}(I)$, is

$\mathrm{Grad}(I)=\\{x=\sum_{g\in G}x_{g}\ /x_{g}\in\sqrt{I}\text{ for
all}g\in G\\}$

Note that $\mathrm{Grad}(I)$ is a $G$-graded ideal of $R$.

######  Lemma 3.10.

Let $R$ be a $G$-graded commutative ring and $I$ be a $G$-graded ideal of $R$.
Then

$\mathrm{Grad}(I)=\cap_{P}P$

Where $P$ runs through all homogeneous prime ideals of $R$ containing $I$.

###### Proof.

It is easy to see that $\mathrm{Grad}(I)\subseteq\cap_{P}P$. Let
$x\in\cap_{P}P$ and assume that $x\not\in\mathrm{Grad}(I)$. Then there exists
$x_{g}$ ($g$-component of $x$) for some $g\in G$ such that
$x_{g}\not\in\sqrt{I}$, that is $x_{g}^{k}\not\in I$ for all $k\in\mathbb{N}$.
Consider the homogeneous multiplicative subset $S:=\\{x_{g}^{k}\
/k\in\mathbb{N}\\}$. We see that $I\cap S=\emptyset$. By [Lemma $4.7$,[9]],
there exists a graded prime ideal $P$ such that $I\subseteq P$ and $P\cap
S=\emptyset$, that is $I\subseteq P$ and $x_{g}^{k}\not\in P$ for all
$k\in\mathbb{N}$, a contradiction. ∎

For an ideal $I_{0}$ of $R_{0}$, denote $R_{1}[I_{0}]$ the subset
$R_{1}[I_{0}]=\\{x\in R_{1}\ /x^{2}\in I_{0}\\}$.

######  Lemma 3.11.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring and $P$ be a
$\mathbb{Z}_{2}$-graded prime ideal of $R$. Then

$P=p+R_{1}[P]$

Where $p=P\cap R_{0}$.

###### Proof.

As in the proof of Theorem 3.8, we have $P=p+R^{\prime}$ where $p=P\cap R_{0}$
and $R^{\prime}=\\{x\in R_{1}\ /xR_{1}\subseteq p\\}$. So, it is enough to
show that $R^{\prime}=R_{1}[p]$. Clearly, $R^{\prime}\subseteq R_{1}[p]$. Let
$x\in R_{1}[p]$. Then $(xR_{1})^{2}=x^{2}R_{1}^{2}\subseteq p$, hence
$xR_{1}\subseteq p$ since $p$ is a prime ideal of $R_{0}$ and $xR_{1}$ is an
ideal of $R_{0}$. Thus $x\in R^{\prime}$. ∎

######  Theorem 3.12.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring and $J$ be a
$\mathbb{Z}_{2}$-graded ideal of $R$. Then

$\mathrm{Grad}(J)=\sqrt{J_{0}}+R_{1}[J_{0}]$

Were $J_{0}=J\cap R_{0}$.

###### Proof.

Note that if $P$ is a $\mathbb{Z}_{2}$-graded ideal of $R$, then $J\subseteq
P$ if and only if $J_{0}\subseteq p$ where $p=P\cap R_{0}$. Thus

$\displaystyle\mathrm{Grad}(J)$ $\displaystyle=$
$\displaystyle\cap_{J_{0}\subseteq p}(p+R_{1}[p])=\cap_{J_{0}\subseteq
p}p+\cap_{J_{0}\subseteq p}R_{1}[p]$ $\displaystyle=$
$\displaystyle\sqrt{J_{0}}+\cap_{J_{0}\subseteq p}R_{1}[p]$

On other hand,

$\displaystyle\cap_{J_{0}\subseteq p}R_{1}[p]$ $\displaystyle=$
$\displaystyle\\{x\in R_{1}\ /x^{2}\in p,\text{ for all }p\supseteq J_{0}\\}$
$\displaystyle=$ $\displaystyle\\{x\in R_{1}\
/x^{2}\in\sqrt{J_{0}}\\}=R_{1}[J_{0}]$

It follows that $R^{\prime}=R_{1}[p]$ as desired. ∎

##  4\. $\mathbb{Z}_{2}$-Maximal graded ideals

######  Definition 4.1.

Let $R$ be a $G$-graded ring. A graded ideal $M$ of $R$ is said to be graded
maximal ideal of $R$ if $M\neq R$ and if $J$ is a graded ideal of $R$ such
that $M\subseteq J\subseteq R$, then $J=M$ or $J=R$.

######  Example 4.2.

Let $R=\mathbb{Z}+i\mathbb{Z}$ be a $\mathbb{Z}_{2}$-graded ring with
$R_{0}=\mathbb{Z}$ and $R_{1}=i\mathbb{Z}$. Let $M=2\mathbb{Z}+i2\mathbb{Z}$.
Then $M$ is a $\mathbb{Z}_{2}$-graded maximal ideal of $R$ and it is not a
maximal (prime ) ideal of $R$.

###### Proof.

Let $J$ be a $\mathbb{Z}_{2}$-graded ideal of $R$ with $M\subsetneq J\subseteq
R$. Let $x=a+ib\in J-M$ with $(a,b)\in\mathbb{Z}^{2}$. Then $a\not\in
2\mathbb{Z}$ or $b\not\in 2\mathbb{Z}$. If $a\not\in\mathbb{Z}$, then $a=2k+1$
where $k\in\mathbb{Z}$. So $1=a-(2k)\in J$, hence $J=R$. If $b\not\in
2\mathbb{Z}$, then $b=(2l)+1$ where $l\in\mathbb{Z}$, so $i=ib-2il\in J$.
Hence $-1=i^{2}\in J$. Thus $J=R$. Moreover $M$ is not a prime ideal see the
Remark 3.2, hence it is not a maximal ideal. ∎

In the following Theorem we give a characterization of $\mathbb{Z}_{2}$-graded
maximal ideals of $R$.

######  Theorem 4.3.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring and $Q$ be a
$\mathbb{Z}_{2}$-graded ideal of $R$. Then $Q$ is a $\mathbb{Z}_{2}$-graded
maximal ideal of $R$ if and only if

  1. (1)

either $Q=p+R_{1}$ where $p\in V(R_{1}^{2})$ is a maximal ideal of $R_{0}$, or

  2. (2)

$Q=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ where $R^{\prime}$ is a maximal
submodule of $R_{1}$ such that $R_{1}^{3}\not\subseteq R^{\prime}$.

###### Proof.

($\Rightarrow$): Let $Q$ be a $\mathbb{Z}_{2}$-graded maximal ideal of $R$, in
particular $Q$ is a $\mathbb{Z}_{2}$-graded prime ideal of $R$.

First case; $R_{1}\subseteq Q$, from Theorem 3.3, $Q=p+R_{1}$ where $p\in
V(R_{1}^{2})$. Let $p^{\prime}$ be a maximal ideal containing $p$, then
$Q\subseteq p^{\prime}+R_{1}$ and $p^{\prime}+R_{1}$ is a proper
$\mathbb{Z}_{2}$-graded ideal of $R$, by maximality, $Q=p^{\prime}+R_{1}$, so
$p=p^{\prime}$ is a maximal ideal of $R_{0}$ containing $R_{1}^{2}$.

Second case; $R_{1}\not\subseteq Q$, by Theorem 3.3,
$Q=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ where $R^{\prime}$ is a prime
$R_{0}$-submodule of $R_{1}$ such that $R_{1}^{3}\not\subseteq R^{\prime}$.
Let $R^{\prime\prime}$ be an $R_{0}$-submodule of $R_{1}$ such that
$R^{\prime}\subseteq R^{\prime\prime}$ and consider the
$\mathbb{Z}_{2}$-graded ideal
$J:=(R^{\prime\prime}:_{R_{0}}R_{1})+R^{\prime\prime}$. Clearly $Q\subseteq
J$, by maximality $J=Q$ or $J=R$, that is $R^{\prime\prime}=R^{\prime}$ or
$R^{\prime\prime}=R_{1}$. Thus $R^{\prime}$ is a maximal submodule of $R_{1}$.
($\Leftarrow$): Let $p$ be a maximal ideal of $R_{0}$ containing $R_{1}^{2}$
and $Q:=p+R_{1}$. Let $J$ be a $\mathbb{Z}_{2}$-graded ideal of $R$ such that
$Q\subseteq J$, then $R_{1}=Q\cap R_{1}\subseteq J\cap R_{1}$, so $J\cap
R_{1}=R_{1}$. Also $p=Q\cap R_{0}\subseteq J\cap R_{0}$. Since $p$ is a
maximal ideal of $R_{0}$, $J\cap R_{0}=p$ or $J\cap R_{0}=R_{0}$ that is $J=Q$
or $J=R$. Thus $Q$ is a $\mathbb{Z}_{2}$-graded maximal ideal of $R$.

Let $R^{\prime}$ be a maximal submodule of $R_{1}$ such that
$R_{1}^{3}\not\subseteq R^{\prime}$ and
$Q:=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$. Clearly $Q$ is a proper
$\mathbb{Z}_{2}$-graded ideal of $R$. Now, let $P$ be any
$\mathbb{Z}_{2}$-graded maximal ideal of $R$ containing $Q$. Denote
$p^{\prime}:=P\cap R_{0}$. We have $(R^{\prime}:_{R_{0}}R_{1})=Q\cap
R_{0}\subseteq P\cap R_{0}$. Since $R^{\prime}$ is a maximal submodule of
$R_{1}$, $(R^{\prime}:_{R_{0}}R_{1})$ is a maximal ideal of $R_{0}$, it
follows that $p^{\prime}=(R^{\prime}:_{R_{0}}R_{1})$ or $p^{\prime}=R_{0}$. If
$p^{\prime}=R_{0}$, then $P=R$ since $1\in P$, with is not compatible with the
fact that $P$ is a $\mathbb{Z}_{2}$-graded maximal ideal. So
$p^{\prime}=(R^{\prime}:_{R_{0}}R_{1})$, hence $\varphi(Q)=\varphi(P)$ where
$\varphi$ is the map defined in Theorem 3.8, by the said Theorem, we have
$P=Q$. Thus $Q$ is a $\mathbb{Z}_{2}$-graded maximal ideal of $R$. ∎

Next, we establish a relation between $\mathbb{Z}_{2}$-graded maximal ideal of
$R$ and maximal ideal of $R_{0}$. Before, we establish the following Lemma.

######  Lemma 4.4.

Let $R$ be a $\mathbb{Z}_{2}$-graded ring and $R^{\prime}$ be an
$R_{0}$-submodule of $R_{1}$ such that $R_{1}^{3}\not\subseteq R^{\prime}$.
The followings statements are equivalent.

  1. (1)

$R^{\prime}$ is a maximal submodule of $R_{1}$.

  2. (2)

$(R^{\prime}:_{R_{0}}R_{1})$ is maximal ideal of $R_{0}$.

In this case $R^{\prime}=(R^{\prime}:_{R_{0}}R_{1})R_{1}$.

###### Proof.

If $R^{\prime}$ is a maximal submodule of $R_{1}$, then clearly
$(R^{\prime}:_{R_{0}}R_{1})\neq R_{0}$. Pike $\alpha\in
R_{0}-(R^{\prime}:_{R_{0}}R_{1})$, then $\alpha r\not\in R^{\prime}$ for some
$r\in R_{1}$. It follows that $R^{\prime}\subsetneq R^{\prime}+R_{0}(\alpha
r)$, so $R^{\prime}+R_{0}(\alpha r)=R_{1}$. In particular, $r=\beta\alpha
r+r^{\prime}$ where $\beta\in R_{0}$ and $r(\in R^{\prime}$. So that
$(1-\beta\alpha)r=r^{\prime}\in R^{\prime}$. Thus
$1-\beta\alpha\in(R^{\prime}:_{R_{0}}R_{1})$, that is
$(R^{\prime}:_{R_{0}}R_{1})+(\alpha)=R_{0}$. Hence
$(R^{\prime}:_{R_{0}}R_{1})$ is a maximal ideal of $R_{0}$. For the converse,
assume that $(R^{\prime}:_{R_{0}}R_{1})$ is a maximal ideal of $R_{0}$. Let
$R^{\prime\prime}$ be a submodule of $R_{1}$ with $R^{\prime}\subseteq
R^{\prime\prime}$. Then
$(R^{\prime}:_{R_{0}}R_{1})\subseteq(R^{\prime\prime}:_{R_{0}}R_{1})$, hence
$(R^{\prime\prime}:_{R_{0}}R_{1})=R_{0}$ or
$(R^{\prime\prime}:_{R_{0}}R_{1})=(R^{\prime}:_{R_{0}}R_{1})$. If
$(R^{\prime\prime}:_{R_{0}}R_{1})=R_{0}$, then clearly
$R^{\prime\prime}=R_{1}$. Assume that
$(R^{\prime\prime}:_{R_{0}}R_{1})=(R^{\prime}:_{R_{0}}R_{1})$. Let $x\in
R^{\prime\prime}$, then
$xR_{1}\subseteq(R^{\prime\prime}:_{R_{0}}R_{1})=(R^{\prime}:_{R_{0}}R_{1})$,
so $xR_{1}^{2}\subseteq R^{\prime}$. Since $R^{\prime}$ is a prime submodule
of $R_{1}$ and $R_{1}^{2}R_{1}=R_{1}^{3}\not\subseteq R^{\prime}$, it follows
that $x\in R^{\prime}$. Thus $R^{\prime\prime}=R^{\prime}$. Therefore
$R^{\prime}$ is maximal submodule of $R_{1}$.

We see that $R_{1}^{2}\not\subseteq(R^{\prime}:_{R_{0}}R_{1})$, so that
$(R^{\prime}:_{R_{0}}R_{1})+R_{1}^{2}=R_{0}$ since
$(R^{\prime}:_{R_{0}}R_{1})$ is a maximal ideal of $R_{0}$. Thus
$1=\displaystyle\sum_{k=1}^{n}a_{i}b_{i}+\alpha$ where $a_{k},b_{k}\in R_{1}$
and $\alpha\in(R^{\prime}:_{R_{0}}R_{1})$. Now, if $r\in R^{\prime}$, then
$r=\displaystyle\sum_{k=1}^{n}ra_{i}b_{i}+\alpha
r\in(R^{\prime}:_{R_{0}}R_{1})R_{1}$ since
$a_{i}r\in(R^{\prime}:_{R_{0}}R_{1})$ and
$\alpha\in(R^{\prime}:_{R_{0}}R_{1})$. Therefore
$R^{\prime}=(R^{\prime}:_{R_{0}}R_{1})R_{1}$. ∎

Now, we are able to establish a relation between $\mathbb{Z}_{2}$-graded
maximal ideals of $R$ and maximal ideals of $R_{0}$, as follow.

######  Theorem 4.5.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. Then

$\mathbb{Z}_{2}\mathrm{Max}R=\\{p+R_{1}/\ p\in\mathrm{Max}R_{0}\cap
V(R_{1}^{2})\\}\cup\\{p+pR_{1}\ /\ p\in\mathrm{Max}R_{0}\cap D(R_{1}^{2})\\}$

###### Proof.

Let $R^{\prime}$ be a maximal submodule of $R_{1}$ such that
$R_{1}^{3}\not\subseteq R^{\prime}$ and denote
$p:=(R^{\prime}:_{R_{0}}R_{1})$. By the previous Lemma, $p$ is a maximal ideal
of $R_{0}$ and $R^{\prime}=pR_{1}$. Moreover $R_{1}^{2}\not\subseteq p$ since
$R_{1}^{3}\not\subseteq R^{\prime}$. Thus
$(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}=p+pR_{1}$. Conversely, let $p$ be a
maximal ideal of $R_{0}$ such that $R_{1}^{2}\not\subseteq p$ and denote
$R^{\prime}=pR_{1}$. By the Proposition 3.4, $(R^{\prime}:_{R_{0}}R_{1})=p$ is
a maximal ideal of $R_{0}$. Since $R_{1}^{2}\not\subseteq p$,
$R_{1}^{3}\not\subseteq R^{\prime}$. It follows from the previous Lemma that
$R^{\prime}$ is a maximal submodule of $R_{1}$. ∎

An immediate consequence, we have the following Corollary.

######  Corollary 4.6.

Let $R$ be a strongly $\mathbb{Z}_{2}$-graded commutative ring. Then

$\mathbb{Z}_{2}\mathrm{Max}R=\\{p+pR_{1}\ /\ p\in\mathrm{Max}R_{0}\\}$

###### Proof.

Follows immediately from the previous Proposition. ∎

######  Corollary 4.7.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. Then

  1. (1)

$\varphi$ induce an homeomorphism between $\mathbb{Z}_{2}\mathrm{Max}R$ and
$\mathrm{Max}R_{0}$.

  2. (2)

$R$ is a $\mathbb{Z}_{2}$-graded local ring if and only if $R_{0}$ is a local
ring.

###### Proof.

  1. (1)

Due to the fact that, $\mathbb{Z}_{2}$-graded maximal ideals of $R$
(respectively, maximal ideals of $R_{0}$) are exactly the closed points of
$\mathbb{Z}_{2}\mathrm{Spec}R$ (respectively $\mathrm{Spec}R_{0}$) with
respect to the Zariski topology.

  2. (2)

This is the case of one closed point.

∎

##  5\. $\mathbb{Z}_{2}$-Graded field

######  Definition 5.1.

Let $R$ be a $G$-graded ring.

  1. (1)

$R$ is called a $G$-graded integral domain if whenever $r$ and $s$ are
homogeneous elements of $R$ such that $rs=0$, then either $r=0$ or $s=0$.

  2. (2)

$R$ is called a $G$-graded field if every nonzero homogeneous element is a
unit.

######  Remark 5.2.

A $G$-graded commutative ring $R$ is a $G$-graded domain if and only if the
zero ideal is a $G$-graded prime ideal of $R$. Whereas, a $G$-graded
commutative ring is a $G$-graded field if and only if the zero ideal is a
$G$-graded maximal ideal of $R$.

######  Theorem 5.3.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. Then $R$ is a
$\mathbb{Z}_{2}$-graded integral domain if and only if $R_{1}=0$ and $R_{0}$
is an integral domain or $0$ is a prime submodule of $R_{1}$ such that
$(0:_{R_{0}}R_{1})=0$ and $R_{1}^{3}\neq 0$.

###### Proof.

$(\Rightarrow)$ If $R$ is a $\mathbb{Z}_{2}$-graded integral domain, then $0$
is a $\mathbb{Z}_{2}$-graded prime ideal of $R$. By Theorem 3.3, $0=p+R_{1}$
where $p\in V(R_{1}^{2})$ or $0=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$ where
$R^{\prime}$ is a prime submodule of $R_{1}$ with $R_{1}^{3}\not\subseteq
R^{\prime}$. If $0=p+R_{1}$, then $R_{0}$ and $p=0$ is a prime ideal of
$R_{0}$, that is $R_{1}=0$ and $R_{0}$ is an integral domain. If
$0=(R^{\prime}:_{R_{0}}R_{1})+R^{\prime}$, then $R^{\prime}=0$ and
$(R^{\prime}:_{R_{0}}R_{1})=0$, as desired.
$(\Leftarrow)$ Immediate. ∎

######  Corollary 5.4.

Let $R$ be a strongly $\mathbb{Z}_{2}$-graded commutative ring. Then $R$ is a
$\mathbb{Z}_{2}$-graded integral domain if and only if $R_{0}$ is an integral
domain.

###### Proof.

By the Corollary 3.5. ∎

######  Theorem 5.5.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. Then $R$ is a
$\mathbb{Z}_{2}$-graded field if and only if $R_{0}$ is a field and either
$R_{1}=0$ or $R_{1}^{2}\neq 0$.

###### Proof.

$R$ is a $\mathbb{Z}_{2}$-graded field if and only if $0$ is a
$\mathbb{Z}_{2}$-graded maximal ideal of $R$. According to Theorem 4.5,
$0=p+R_{1}$ with $p$ maximal ideal of $R_{0}$ containing $R_{1}^{2}$, which is
equivalent to $R_{1}=0$ and $0=p$ is a maximal ideal of $R_{0}$. Or
$0=p+pR_{1}$ with $p$ is a maximal ideal of $R_{0}$ such that
$R_{1}^{2}\not\subseteq p$, which is equivalent to $0=p$ is a maximal ideal of
$R_{0}$ and $R_{1}^{2}\neq 0$. ∎

######  Corollary 5.6.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. Then $R$ is a
$\mathbb{Z}_{2}$-graded field if and only if $R_{0}$ is a field and either
$R_{1}=0$ or $R_{1}=R_{0}b$ where $b\in R_{1}$ such that $b^{2}\neq 0$.

###### Proof.

We may assume that $R_{1}\neq 0$. If $R$ is a $\mathbb{Z}_{2}$-graded field,
then by the previous Theorem $R_{0}$ is a field and $R_{1}^{2}\neq 0$. So
$bb^{\prime}\neq 0$ for some $b,b^{\prime}\in R_{1}$ since $R_{1}^{2}$ is
generated by all elements $bb^{\prime}$ where $b,b^{\prime}\in R_{1}$. As
$bb^{\prime}\neq 0$ and $R_{0}$ is a filed, $bb^{\prime}$ is a unit in
$R_{0}$. There exists $\alpha\in R_{0}$ such that $bb^{\prime}\alpha=1$. It
follows that $r=\alpha(rb^{\prime})b$ for all $r\in R_{1}$. Therefore
$R_{1}=R_{0}b$. Clearly $b^{2}\neq 0$ since $R_{1}^{2}\neq 0$. The converse is
immediate, it is easy to see that every homogeneous elements in a unit. ∎

For a commutative ring $S$ we look at $S[X]$ as a $\mathbb{Z}_{2}$-graded
commutative ring see Example 1.1. We see that $(S[X])_{0}=\\{P(X^{2})\ /\ P\in
S[X]\\}$ and $(S[X])_{1}=\\{XP(X^{2})\ /\ P\in S[X]\\}$. For $\alpha\in S$,
$(X^{2}-\alpha)$ is a $\mathbb{Z}_{2}$-graded ideal of $S[X]$. Therefor
$\frac{S[X]}{(X^{2}-\alpha)}$ is a $\mathbb{Z}_{2}$-graded commutative ring
and clearly $\left(\frac{S[X]}{(X^{2}-\alpha)}\right)_{0}=S$ and
$\left(\frac{S[X]}{(X^{2}-\alpha)}\right)_{1}=S\overline{X}$. Note that if $S$
is a field and $\alpha\neq 0$, then $\frac{S[X]}{(X^{2}-\alpha)}$ is a graded
field.

######  Corollary 5.7.

Let $R$ be a $\mathbb{Z}_{2}$-graded commutative ring. If $R$ is a
$\mathbb{Z}_{2}$-graded field, then $R_{0}$ is a field and either $R_{1}=0$ or
$R$ is isomorphic to $\frac{R_{0}[X]}{(X^{2}-\alpha)}$ as
$\mathbb{Z}_{2}$-graded rings, where $\alpha$ is a nonzero element of $R_{0}$.

###### Proof.

We may assume that $R_{1}\neq 0$. By the previous Corollary $R_{0}$ is a field
and $R_{1}=R_{0}b$ where $b\in R_{1}$ such that $b^{2}=0$. Let $f:R_{9}[X]\to
R$ the map defined by $f(P)=P(b)$. Clearly $f$ is a rings morphism. We have
$f(P(X^{2}))=P(b^{2})\in R_{0}$ since $b^{2}\in R_{0}$, also
$f(XP(X^{2}))=bP(b^{2})\in R_{1}$. It follows that $f$ is a
$\mathbb{Z}_{2}$-graded morphism. For $r=r_{0}+r_{1}\in R$. We see that
$r_{1}=ab$ for some $a\in R_{0}$, so that $f(r_{0}+aX)=r_{0}+ab=r$. Now, let
$\alpha=b^{2}\in R_{0}$, then $(X^{2}-\alpha)\subseteq\ker f$. Let $P\in\ker
f$, by the euclidean division, $P=(X^{2}-\alpha)Q+dX+c$ where $Q\in R_{0}[X]$
and $d,c\in R_{0}$. But $P(b)=0$ implies that $db+c=0$, so that $d=0$ and
$c=0$ ( if $d\neq 0$, then $d$ is a unit, so $b=d^{-1}c\in R_{0}\cap R_{1}$
which is not possible). It follows that $\ker f=(X^{2}-\alpha)$. ∎

##  6\. $\mathbb{Z}_{2}$-Graded integral domain and integral domain

For $x=a+r\in R$ where $(a,r)\in R_{0}\times R_{1}$, define
$N(x):=a^{2}-r^{2}$. Then, $N(xy)=N(x)N(y)$ and $N(x)\in R_{0}$ for all
$x,y\in R$. Also $N(0)=0$ and $N(1)=1$. Next we establish the relation between
some properties of $N$ and the integrity of $R$. To do, let
$\mathcal{N}:=\\{x\in R\ /N(x)=0\\}$.

######  Theorem 6.1.

Let $R$ be a $\mathbb{Z}_{2}$-graded ring. The following statements are
equivalent.

  1. (1)

$R$ is an integral domain.

  2. (2)

$R$ is a $\mathbb{Z}_{2}$-graded integral domain and $\mathcal{N}=0$.

###### Proof.

$(1)\Rightarrow(2)$: Clearly, if $R$ is an integral domain then $R$ is a
$\mathbb{Z}_{2}$-graded integral domain. Let $x=a+r\in\mathcal{N}$, then
$a^{2}-r^{2}=0$, so that $(a+r)(a-r)=0$, thus $a+r=0$ or $a-r=0$. Hence $x=0$.
It follows that $\mathcal{N}=0$.
$(2)\Rightarrow(1)$: Let $x,y\in R$ with $xy=0$. Then $N(x)N(y)=0$. Note that
$N(x)$ and $N(y)$ are homogeneous, so that $N(x)=0$ or $N(y)=0$ that is
$x\in\mathcal{N}$ or $y\in\mathcal{N}$. Thus $x=0$ or $y=0$. ∎

## References

  * [1] W. Heinzer and M. Roitman. The homogeneous spectrum of a graded commutative ring . Proc. Amer. Math. Soc. 130 (2002), 1573-1580
  * [2] H. A. Khashan, Graded rings in which every graded ideal is a product of Gr-primary ideals, Int. J. Algebra, 2(13-16) (2008), 779-788.
  * [3] CP. Lu Prime submodules of modules. Commentarii Mathematici Universitatis Sancti Pauli $1984$; $33$ $(1)$: $61-69$.
  * [4] C. Nastasescu. Strongly Graded Rings of Finite Groups, Communications in Algebra, 11(10) (1983), 1033-1071.
  * [5] C. Nastasescu and F. Van Oystaeyen, Graded ring theory, Mathematical Library 28, North Holland, Amesterdam, 1982.
  * [6] J. Ohm and R. Pendleton. Rings with Noetherian spectrum. Duke Math. J.35:631-639, (1968).
  * [7] M. Refai, M. Hailat and S. Obiedat, Graded radicals on graded prime spectra, FarEast J. of Math. Sci., part I (2000), 59-73.
  * [8] A. Seidenberg.A note on the dimension theory of rings, Pac. J. Math., 3 (1953), 505-512.
  * [9] X. Wu Graded $w$-Noetherian modules over graded rings. Bull. Korean Math. Soc. 2020;57:1319-1334. https://doi.org/10.4134/BKMS.b191007
  * [10] A. Yousefian Darani. Topologies on $\mathrm{Spec}_{g}(M)$, Buletinul Academiei de Ştiinţe a Moldovei. Matematica. 3(67), 2011, 45-53.
  * [11] A. Yousefian Darania and S. Motmaen. Zariski topology on the spectrum of graded classical prime submodules. Appl. Gen. Topol. 14, no. 2 (2013), 159-169.


# Combining Static Word Embeddings and Contextual Representations for
Bilingual Lexicon Induction

Jinpeng Zhang1, Baijun Ji1, Nini Xiao1, Xiangyu Duan1, Min Zhang1
Yangbin Shi2, Weihua Luo2
1 Institute of Aritificial Intelligence, School of Computer Science and
Technology,
Soochow university
2 Alibaba DAMO Academy
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
Corresponding Author.

###### Abstract

Bilingual Lexicon Induction (BLI) aims to map words in one language to their
translations in another, and is typically through learning linear projections
to align monolingual word representation spaces. Two classes of word
representations have been explored for BLI: static word embeddings and
contextual representations, but there is no studies to combine both. In this
paper, we propose a simple yet effective mechanism to combine the static word
embeddings and the contextual representations to utilize the advantages of
both paradigms. We test the combination mechanism on various language pairs
under the supervised and unsupervised BLI benchmark settings. Experiments show
that our mechanism consistently improves performances over robust BLI
baselines on all language pairs by averagely improving 3.2 points in the
supervised setting, and 3.1 points in the unsupervised setting111Code is
released at https://github.com/zjpbinary/CSCBLI.

##  1 Introduction

Bilingual Lexicon Induction (BLI) aims to find bilingual translation lexicons
from monolingual corpora in two languages (Haghighi et al., 2008; Xing et al.,
2015; Zhang et al., 2017a; Artetxe et al., 2017; Conneau et al., 2017), and is
applied on numerous NLP tasks such as POS tagging (Zhang et al., 2016),
parsing Xiao and Guo (2014), and machine translation Irvine and Callison-Burch
(2013); Qi et al. (2018).

Most work on BLI learns a mapping between two static word embedding spaces,
which are pre-trained on large monolingual corpora Ruder et al. (2019). Both
linear mapping Mikolov et al. (2013); Xing et al. (2015); Artetxe et al.
(2016); Smith et al. (2017) and non-linear mapping Mohiuddin et al. (2020)
methods have been studied to align the two spaces. Recently, other than the
static word embeddings, contextual representations are used for BLI due to the
significant progresses on cross-lingual applications Aldarmaki and Diab
(2019); Schuster et al. (2019). Although the static word embeddings and the
contextual representations exhibit properties suited for alignment, there is
no works to combine the two paradigms.

On one hand, the static word embeddings have been widely used for BLI, but one
specific embedding mapping function does not ensure that in all conditions,
words in a translation pair are nearest neighbors in the mapped common space.
On the other hand, the contextual representations contain rich semantic
information beneficial for alignment, but the dynamic contexts of word tokens
pose a challenge for aligning word types.

In this paper, we propose a combination mechanism to utilize the static word
embeddings and the contextual representations simultaneously. The combination
mechanism consists of two parts. The first part is the unified word
representations, in which a spring network is proposed to use the contextual
representations to pull the static word embeddings to better positions in the
unified space for easy alignment. The spring network and the unified word
representations are trained via a contrastive loss that encourages words of a
translation pair to become closer in the unified space, and penalizes words of
a non-translation pair to be farther. The second part is the weighted
interpolation between the words similarity in the unified word representation
space and the words similarity in the contextual representation space.

We test the proposed combination mechanism in both the supervised BLI setting
which can utilize a bilingual dictionary as the training set, and the
unsupervised BLI setting which does not allow using any parallel resources as
supervision signal. On BLI benchmark sets of multiple language pairs, our
combination mechanism performs significantly better than systems using only
the static word embeddings and systems using only the contextual
representations. Our mechanism improves over robust BLI baselines on all
language pairs, achieving average 3.2 points improvement in the supervised
setting, and average 3.1 points improvement in the unsupervised setting.

##  2 Background

The early works on Bilingual Lexicon Induction (BLI) date back to several
decades ago, including feature-based retrieval (Fung and Yee, 1998),
distributional hypothesis (Rapp, 1999; Vulić and Moens, 2013), and
decipherment (Ravi and Knight, 2011). Following Mikolov et al. (2013), which
pioneered the embedding based BLI method, word representation based method
becomes the dominant approach, and can be categorized into two classes: static
word embedding based method, and contextual representation based method.

  * •

Static Word Embedding Based Method

Word embeddings of different languages are pre-trained in large monolingual
corpora independently. Then a mapping function is applied to align the
embedding spaces of the two languages (Mikolov et al., 2013; Xing et al.,
2015; Artetxe et al., 2016; Smith et al., 2017).

We follow one robust BLI system VecMap (Artetxe et al., 2018a, b), which maps
both source space and target space into a third common space. Let $E_{x}$ and
$E_{y}$ be the word embedding matrices in two languages for a given bilingual
dictionary such that their $i$th rows are the embeddings of words of the $i$th
translation pair in the dictionary. The training objective is to find mapping
functions $W_{x}$ and $W_{y}$ such that

$W_{x}^{\star},W_{y}^{\star}=\underset{W_{x},W_{y}\in
M_{d}(\mathbb{R})}{\mathop{\arg\max}}cos(E_{x}W_{x},E_{y}W_{y})$ (1)

where $d$ is the dimension of the embeddings, $M_{d}(\mathbb{R})$ is the space
of $d\times d$ matrices of real numbers. The optimal $W_{x}$ and $W_{y}$
maximizes the cosine similarity between words of each translation pair in the
mapped common space. In the unsupervised version where no bilingual dictionary
is given, an artificial dictionary is initialized and iteratively updated
through training $W_{x}$ and $W_{y}$ according to equation (1) (Artetxe et
al., 2018b).

Both mapping functions are constrained to be orthogonal during training by
setting $W_{x}=U$ and $W_{y}=V$, where $U\Sigma V^{T}=X^{T}Y$ is the singular
value decomposition of $X^{T}Y$. Such orthogonal constraint is based on the
assumption that the source embedding space and the target embedding space are
isometric, which is a particularly strong assumption that does not hold in all
conditions (Zhang et al., 2017b; Søgaard et al., 2018). To depart from the
isometry assumption, Patra et al. (2019) uses a semi-supervised technique that
leverages both seed dictionary and a larger set of unaligned word embeddings,
Mohiuddin et al. (2020) uses a non-linear mapping function that is not
constrained to be orthogonal.

We propose another method to relax the isometry assumption by combining the
contextual representations with the word embeddings to compensate the shortage
of the overly strong assumption.

  * •

Contextual Representation Based Method

Contextual representations can be obtained through multilingual pre-training,
which encodes whole sentence and outputs contextual representation for each
word (Devlin et al., 2019; Lample and Conneau, 2019). Due to the rich context
information contained in the contextual representations, there are endeavors
to align them in different languages Schuster et al. (2019); Aldarmaki and
Diab (2019); Wang et al. (2020); Kulshreshtha et al. (2020); Cao et al.
(2020).

Since a word may appear in different sentences with different contexts,
Schuster et al. (2019) use an average anchor to summarize multiple contexts
for a word type and align the anchors of different languages, while other
works aim to align each individual context representation based on parallel
corpora, including learning alignment on sentence level representations and
applying the learned mapping on word level contextual representations
Aldarmaki and Diab (2019), using word alignments in a parallel corpora to
learn the mapping for word contextual representations Wang et al. (2020), and
directly minimizing the distance between two contextual representations of an
aligned word pair in parallel corpora without mapping Cao et al. (2020).

We adopt the average anchor method for the contextual representations Schuster
et al. (2019), which does not depend on parallel corpora. Let the contextual
representation of a source word $x$ in context $c_{i}$ be denoted as
$r_{x,c_{i}}$. If $x$ appears a total of $p$ times in the source corpus, the
average anchor for $x$ across all contexts is:

$a_{x}=\frac{\sum_{i=1}^{p}r_{x,c_{i}}}{p}$ (2)

Similar to the mapping for the static word embeddings, we conduct mapping for
the average anchors. Let $A_{x}$ and $A_{y}$ be the matrices of average
anchors in two languages with correspondence to word pairs from a given
bilingual dictionary. The mapping functions $V_{x}$ and $V_{y}$ are optimized
by maximizing $cos(A_{x}V_{x},A_{y}V_{y})$, where $A_{x}$ and $A_{y}$ are
fixed, $V_{x}$ and $V_{y}\in M_{d^{\prime}}(\mathbb{R})$, and $d^{\prime}$ is
the dimension of the contextual representations.

Besides the above methods, there is another direction that extracts word
alignments in pseudo parallel corpora for BLI. The pseudo parallel corpora are
built by either the unsupervised machine translation (Artetxe et al., 2019) or
the unsupervised bitext mining Shi et al. (2021). Both methods need
significant computation overload or use monolingual corpora that are
magnitudes larger than ours, and are beyond the scope of this paper that
focuses on representation based methods.

##  3 Proposed Combination Mechanism for BLI

Since there is no work to combine both the static word embeddings and the
contextual representations, we propose a combination mechanism illustrated in
Figure LABEL:fig:framework. The mechanism first builds a unified word
representation space that unifies the static word embeddings and the
contextual representations, then performs similarity interpolation between the
unified space and the contextual space.

###  3.1 The Unified Word Representations

As shown in Figure LABEL:fig:framework, the original word embedding space (A)
is mapped to (B) through the mapping functions. Since the mapping functions
are orthogonal, (A) is just rotated to (B). Notice that the spaces of the two
languages are not necessarily isometric everywhere, some words in certain
translation pairs are still far away from each other after rotation. To pull
words in a translation pair getting closer, we propose a spring network that
can pull the mapped embedding points to better positions such that words in a
translation pair are nearest neighbors of each other. Since the contextual
representations contain rich context information that can be used as the
flexible adjustment, the spring network takes the contextual representations
as input, and outputs offsets for the word embeddings.

Specifically, in the unified word representations, the mapped word embeddings
are pulled to new positions by offsets, which are produced by the spring
network with the the contextual representations as input:

$\displaystyle U_{x}=E_{x}^{{}^{\prime}}+\gamma_{1}\odot F_{x}(A_{x})$
$\displaystyle U_{y}=E_{y}^{{}^{\prime}}+\gamma_{2}\odot F_{y}(A_{y})$ (3)

where $U_{x}$ and $U_{y}$ are the unified word representations,
$E_{x}^{{}^{\prime}}$ and $E_{y}^{{}^{\prime}}$ are the mapped word
embeddings, $F_{x}$ and $F_{y}$ are the spring networks, and
$\gamma_{1\rm{or}2}$ is the weight vector, which is used to element-wisely
multiply each row of the output of the spring network. Take the source side
for example, the mapped word embedding matrix $E_{x}^{{}^{\prime}}$ is added
with a weighted offset produced by the spring network $F_{x}$ on the
contextual representation (i.e., the average anchor) matrix $A_{x}$.

The Spring Network stacks two feedforward layers with Tanh activations on top
of the contextual representation matrices. The first layer transforms the
dimension of the contextual representation $d^{\prime}$ to the dimension of
the word embedding $d$. Equations (4-5) list the network structure of both
sides.

$\displaystyle A_{x}^{1}$ $\displaystyle=\phi(\theta_{x}^{0}(A_{x})),$
$\displaystyle A_{y}^{1}$ $\displaystyle=\phi(\theta_{y}^{0}(A_{y}))$ (4)
$\displaystyle A_{x}^{2}$ $\displaystyle=\phi(\theta_{x}^{1}(A_{x}^{1})),$
$\displaystyle A_{y}^{2}$ $\displaystyle=\phi(\theta_{y}^{1}(A_{y}^{1}))$ (5)

where $\phi$ denotes the Tanh activation, and $\theta$ denotes the feedforward
layer. $A_{x/y}^{2}$ is the output of the spring network, and fulfills as the
offset distance to compensate the deviation of words in each translation pair
in the mapped word embedding space.

Since we use cross-lingual pre-training (Lample and Conneau, 2019) to generate
the contextual representations, which are actually BPE’s (Sennrich et al.,
2016) contextual representations, we have to form the contextual
representations in the word level. Suppose a word $x$ has $q$ BPEs, and $x$
appears $p$ times in the monolingual corpus, then the word level contextual
representation
$a_{x}=\sum_{i=1}^{p}{(\sum_{j=1}^{q}r_{b_{j},c_{i,j}}/{q})}/{p}$, where
$r_{b_{j},c_{i,j}}$ denotes the representation of the $j$th BPE with the $i$th
context $c_{i,j}$. $a_{x}$ actually averages $q$ BPEs’ representation at
first, then averages $p$ contexts. After this cascaded averaging, it
constitutes one row of $A_{x}$.

Contrastive Training is used to train the spring networks $F_{x}$ and $F_{y}$
with the pre-trained mapped word embeddings and the contextual representations
fixed in the unified space. Basically, through the spring adjustment, the
training encourages parallel words to get closer, and drives non-parallel
words to be farther. It is divided into two scenarios: supervised contrastive
training and unsupervised contrastive training.

  * •

In the supervised contrastive training, given a bilingual dictionary with $I$
translation pairs, the contrastive loss is:

$\displaystyle\mathcal{L}_{sup}=$ $\displaystyle-\sum_{i=1}^{I}(J\times
cos(u_{x}^{i},u_{y}^{i})$
$\displaystyle-\sum_{j=1}^{J}cos(u_{x}^{i},u_{\bar{y}}^{j}))$ (6)

where $u_{x}^{i}$ and $u_{y}^{i}$ are the unified representations
corresponding to the $i$th entry of the given bilingual dictionary.

In equation (6), $(u_{x}^{i},u_{y}^{i})$ is the positive translation pair
according to the given dictionary, and the cosine similarity of this pair is
maximized during training, while $(u_{x}^{i},u_{\bar{y}}^{j})$ is the negative
pair where $\bar{y}$ is not aligned to $x$. The cosine similarity of
$(u_{x}^{i},u_{\bar{y}}^{j})$ is minimized during training.

We select $J$ negative pairs for a source word $x$. In the implementation, we
use $J$best outputs of the current model excluding the correct translation as
the negative pairs. To keep balance between positive and negative pairs, the
positive pair is copied $J$ times to pair with negative pairs.

During inference, we select $y=\mathop{\arg\max}_{y}cos(u_{x},u_{y})$ as the
translation of $x$.

  * •

In the unsupervised contrastive training, no bilingual dictionary is given.
The contrastive loss is the same to that of the supervised contrastive
training, except that the bilingual dictionary is not given. We initialize the
bilingual dictionary using the output of the static word embedding based
unsupervised method, and iteratively update it by using the trained model of
last iteration to find new translations for given source words and compose a
new dictionary, which is used to train the new model. Such process iterates
until the dictionary does not change any more.

###  3.2 Similarity Interpolation

The similarity interpolation is for inference. As shown in Figure
LABEL:fig:framework, both the unified word representation space and the mapped
contextual representation space can output the cosine similarities between
words. Given a source word $x$, we interpolate both similarities as below:

$\displaystyle\mathcal{S}=cos(u_{x},u_{y})+\lambda
cos(a_{x}^{{}^{\prime}},a_{y}^{{}^{\prime}})$ (7)

where $\lambda$ is the weight, $a_{x/y}^{{}^{\prime}}$ is the mapped
contextual representation, which is pre-trained as introduced in the section
of the background of the contextual representation based method. We aim to
find $y$ that has the maximal $\mathcal{S}$ as the translation of $x$.

In the supervised setting, $\lambda$ is tuned on the validation set consisting
of translation pairs. In the unsupervised setting, $\lambda$ is tuned by an
unsupervised procedure: when source-to-target model and target-to-source model
have been trained, the word $x$ in the validation set is aligned to
$y^{\prime}$ based on equation (7), then $y^{\prime}$ is back aligned to $x$
based on the inverse version of equation (7). We select $\lambda$ that has the
highest accuracy of this back alignment to $x$.

##  4 Experiments

We test our combination mechanism in supervised and unsupervised BLI tasks on
English-Espanish (EN-ES), English-Arabic (EN-AR), English-Chinese (EN-ZH),
English-German (EN-DE), and English-French (EN-FR).

| EN-ES | EN-AR | EN-ZH | EN-DE | EN-FR | avg
---|---|---|---|---|---|---
| $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$
Supervised BLI |
MuseConneau et al. (2017) | 77.80 | 81.40 | 49.47 | 55.07 | 40.87 | 43.07 | 69.80 | 71.07 | 78.67 | 79.33 | 64.65
VecMapArtetxe et al. (2018a) | 77.20 | 82.13 | 53.53 | 57.73 | 52.13 | 46.67 | 70.33 | 72.80 | 78.40 | 80.47 | 67.14
RCSLSJoulin et al. (2018) | 79.27 | 84.30 | 55.53 | 61.00 | 53.07 | 48.87 | 73.07 | 75.40 | 79.60 | 82.07 | 69.22
BLISSPatra et al. (2019) | 79.67 | 84.87 | 54.47 | 59.60 | 50.80 | 48.80 | 73.33 | 76.13 | 79.20 | 82.93 | 68.98
UnifiedVecMap | 79.60 | 84.73 | 56.20 | 60.73 | 53.80 | 48.93 | 73.27 | 74.47 | 79.33 | 81.33 | 69.24
ContextualVecMap | 44.07 | 50.33 | 5.07 | 7.73 | 21.93 | 9.67 | 44.60 | 47.87 | 57.47 | 65.20 | 35.39
InterpolationVecMap | 80.47 | 85.70 | 57.13 | 61.47 | 56.27 | 50.60 | 74.13 | 77.13 | 80.80 | 83.40 | 70.71
UnifiedRCSLS | 80.13 | 86.60 | 55.87 | 62.47 | 56.67 | 51.13 | 74.26 | 77.93 | 81.20 | 83.87 | 71.01
ContextualRCSLS | 46.27 | 51.40 | 3.67 | 7.13 | 19.47 | 7.93 | 45.67 | 47.80 | 58.00 | 65.20 | 35.25
InterpolationRCSLS | 80.67 | 87.67 | 59.40 | 62.73 | 59.40 | 52.27 | 74.87 | 79.67 | 81.80 | 85.40 | 72.39
Unsupervised BLI |
MuseConneau et al. (2017) | 77.06 | 81.53 | 48.00 | 55.47 | 24.26 | 43.00 | 70.13 | 71.20 | 78.73 | 78.40 | 62.77
VecMapArtetxe et al. (2018b) | 77.60 | 81.67 | 50.87 | 56.73 | 34.33 | 44.00 | 70.00 | 71.80 | 78.73 | 80.27 | 64.60
Ad.Mohiuddin and Joty (2019) | 77.93 | 82.20 | 50.07 | 57.33 | 34.67 | 43.67 | 69.13 | 72.47 | 78.46 | 80.13 | 64.61
Unified | 79.47 | 82.60 | 52.47 | 57.87 | 35.93 | 46.07 | 71.07 | 74.00 | 80.27 | 80.87 | 66.06
Contextual | 46.33 | 55.93 | 3.87 | 7.53 | 17.40 | 4.87 | 43.53 | 44.33 | 56.20 | 63.93 | 34.39
Interpolation | 79.93 | 85.33 | 52.73 | 58.47 | 37.07 | 46.27 | 72.53 | 78.73 | 81.80 | 84.13 | 67.70
Table 1:  P@1 on all language pairs. “Unified” denotes our unified word
representation based method, which computes $cos(u_{x},u_{y})$, “Contextual”
denotes the contextual representation based method, which computes
$cos(a_{x}^{{}^{\prime}},a_{y}^{{}^{\prime}})$, “Interpolation” denotes our
similarity interpolation, which computes $cos(u_{x},u_{y})+\lambda
cos(a_{x}^{{}^{\prime}},a_{y}^{{}^{\prime}})$. The subscript “${\rm{VecMap}}$”
denotes that our method is based on the work of Artetxe et al. (2018a), the
subscript “$\rm RCSLS$” denotes that our method is based on the RCSLS
criterion in training Joulin et al. (2018). In unsupervised BLI, there is no
subscript in our method, which means using the default “$\rm VecMap$” (Artetxe
et al., 2018b).

###  4.1 Data

We need monolingual corpora to compute the contextual representations.
Unfortunately, most existing BLI datasets distribute pre-trained word
embeddings alone, but not the monolingual corpora used to train them. For that
reason, we use WikiExtractor222https://github.com/attardi/wikiextractor to
extract plain text from Wikipedia dumps, and preprocess the resulting corpora
using standard Moses (Koehn et al., 2007) tools by applying sentence
splitting, punctuation normalization, tokenization, and lowercasing. On these
corpora, we use the cross-lingual pre-training system XLM Lample and Conneau
(2019)333https://github.com/facebookresearch/xlm. We use the MLM model of 15
languages with tokenize + lowercase + no accent + BPE. to compute the
contextual representations.

Meanwhile, we also use these corpora to train the static word embeddings by
using fastText444https://github.com/facebookresearch/fastText/ to ensure that
both the contextual representations and the static word embeddings come from
the same data. We use the bilingual dictionaries released by Muse
project555https://github.com/facebookresearch/MUSE in our experiments. Note
that some words in these dictionaries do not necessarily appear in our
monolingual corpora, we have to recompose the training, validation, and test
sets such that all words in these sets are included in our monolingual
corpora. In the end, we have 5000 entries with unique source words in the
training set, and 1500 entries with unique source words in both the validation
set and the test set for all language pairs.

###  4.2 Baseline Systems

Baseline systems are divided into two tasks as below. We run the released code
of each baseline system in our experiments.

Supervised BLI task, which is allowed to use bilingual dictionaries for
training and validation. The baseline systems are:

  * •

Muse: Supervised Muse is set as the baseline in Conneau et al. (2017). It uses
iterative Procrustes alignment for supervised BLI.

  * •

VecMap666https://github.com/artetxem/vecmap: Artetxe et al. (2018a) use a
multi-step framework consisting of several steps: whitening, orthogonal
mapping, re-weighting, de-whitening, and dimensionality reduction.

  * •

RCSLS777https://github.com/facebookresearch/fastText/tree/master/alignment :
In addition to use CSLS during inference, Joulin et al. (2018) minimize a
convex relaxation of CSLS loss during training, and improve the supervised BLI
performance.

  * •

BLISS888https://github.com/joelmoniz/BLISS: Patra et al. (2019) use a semi-
supervised method that leverages both the bilingual dictionary and a larger
set of unaligned word embeddings.

Unsupervised BLI task, which is not allowed to use any parallel resources for
training and validation. The baseline systems are:

  * •

Muse: Unsupervised Muse (Conneau et al., 2017) uses adversarial training and
iterative Procrustes refinement.

  * •

VecMap: Artetxe et al. (2018b) use careful initialization, robust self-
learning procedure, and symmetric re-weighting to improve the unsupervised
mapping result.

  * •

Ad.999https://github.com/taasnim/unsup-word-translation/: Mohiuddin and Joty
(2019) include regularization terms for adversarial auto-encoder for the
unsupervised BLI.

###  4.3 Experimental Settings

We use fastText to train the word embeddings for BLI. The dimension of the
word embeddings is 300. The contextual representations are extracted from XLM,
and the dimension of the contextual representations is 1024. For each word
type, we randomly select ten sentences containing the word from the
monolingual corpora to do the averaging to get the contextual representation.
The influence of the number of selected sentences for each word type is
reported in section 4.5.2. Regarding the spring network, we use ten negative
pairs for each source word in the supervised contrastive training, and use one
negative pair for each source word in the unsupervised contrastive training.

All inferences in our experiments, including all baseline systems, use CSLS
which is introduced in Conneau et al. (2017). The results are evaluated by
Precision@1 (P@1).

| EN-ES | EN-AR | EN-ZH | EN-DE | EN-FR | avg
---|---|---|---|---|---|---
| $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$
Supervised BLI |
XLM-UnifiedRCSLS | 80.13 | 86.60 | 55.87 | 62.47 | 56.67 | 51.13 | 74.26 | 77.93 | 81.20 | 83.87 | 71.01
XLM-ContextualRCSLS | 46.27 | 51.40 | 3.67 | 7.13 | 19.47 | 7.93 | 45.67 | 47.80 | 58.00 | 65.20 | 32.05
XLM-InterpolatedRCSLS | 80.67 | 87.67 | 59.40 | 62.73 | 59.40 | 52.27 | 74.87 | 79.67 | 81.80 | 85.40 | 72.39
mBART-UnifiedRCSLS | 80.07 | 85.33 | 56.13 | 61.93 | 55.53 | 51.20 | 74.27 | 77.53 | 80.20 | 83.40 | 70.56
mBART-ContextualRCSLS | 46.33 | 49.70 | 3.53 | 8.24 | 18.80 | 8.03 | 45.33 | 48.33 | 60.10 | 66.53 | 32.26
mBART-InterpolatedRCSLS | 81.27 | 87.40 | 57.00 | 63.00 | 58.27 | 52.07 | 76.07 | 80.60 | 82.33 | 85.93 | 72.39
Unsupervised BLI |
XLM-Unified | 79.47 | 82.60 | 52.47 | 57.87 | 35.93 | 46.07 | 71.07 | 74.00 | 80.27 | 80.87 | 66.06
XLM-Contextual | 46.33 | 55.93 | 3.87 | 7.53 | 17.40 | 4.87 | 43.53 | 44.33 | 56.20 | 63.93 | 31.26
XLM-Interpolated | 79.93 | 85.33 | 52.73 | 58.47 | 37.07 | 46.27 | 72.53 | 78.73 | 81.80 | 84.13 | 67.70
mBART-Unified | 79.07 | 82.60 | 51.60 | 58.13 | 35.80 | 44.67 | 70.47 | 74.00 | 79.93 | 81.27 | 65.75
mBART-Contextual | 47.06 | 53.37 | 5.40 | 8.13 | 16.76 | 4.77 | 41.23 | 42.53 | 53.53 | 62.87 | 30.51
mBART-Interpolated | 79.93 | 84.90 | 52.73 | 59.67 | 36.60 | 45.27 | 71.87 | 76.33 | 81.40 | 84.53 | 67.32
Table 2:  Result comparison between using XLM and using mBART for the
contextual representations.

###  4.4 Main Results

Table 1 summarizes the main results of the supervised and the unsupervised BLI
tasks on all test sets. In both tasks, our proposed methods achieve
significant improvements, with average 3.2 points higher than the strongest
baseline RCSLS in the supervised task, and with average 3.1 points higher than
the strong baselines VecMap and Ad. in the unsupervised task.

In the supervised task, we have two independent bases to build our proposed
methods. One is VecMap (Artetxe et al., 2018a), the other is RCSLS (Joulin et
al., 2018). They are the preprocessing steps to align the static word
embeddings, and align the contextual representations in the two languages. Our
methods build upon these alignments, and further train the spring networks and
the unified word representations for the combination. The performances of our
methods with these two bases are reported in Table 1 with the corresponding
subscripts.

Table 1 shows that if we use VecMap as the basis of our method, we can improve
3.6 points over the corresponding VecMap baseline. If we use RCSLS as the
basis, we can improve 3.2 points over the corresponding RCSLS baseline. In our
methods, “Unified” can achieve around 2 points improvement over the
corresponding baselines. Although “Contextual” obtains inferior performances,
it is complementary to “Unified”. When “Contextual” is combined with “Unified”
through the interpolation, the performance is further improved, achieving the
best performance among all systems. It shows that our combination mechanism is
effective to utilize the merits of both the static word embeddings and the
contextual representations.

In the unsupervised task, we achieve the significant improvements over the
baselines. “Unified” is 1.5 points better than VecMap baseline. “Contextual”
is inferior to other methods, but it can provide useful complements to
“Unified”, resulting in the final 3.1 points improvement through
interpolation.

In summary, our combination mechanism consistently improves the performances
for both distant language pairs, such as EN-AR and EN-ZH, and closely-related
European language pairs.

###  4.5 Analyses

####  4.5.1 XLM v.s. mBART

Our results in Table 1 are based on using XLM for obtaining the contextual
representations. In this section, we also use mBART (Liu et al., 2020) to
compare with XLM. Table 2 shows the comparison result. XLM pre-trains the
Transformer encoder through the masking mechanism, while mBART pre-trains the
full Transformer encoder-decoder through multilingual denoising. Regarding BLI
task, we obtain the contextual representations from the encoder. Table 2 shows
that XLM and mBART get similar BLI performances since only encoder is used. In
some directions, mBART performs slightly better than XLM, while in other
directions, XLM is slightly better. According to the average performance, XLM
ties with mBART in the supervised task, and is slightly better in the
unsupervised task.

####  4.5.2 The Randomness of Contexts

| EN-DE | EN-FR
---|---|---
| $\to$ | $\leftarrow$ | $\to$ | $\leftarrow$
Supervised BLI |
RCSLS | 61.82 | 60.92 | 59.58 | 60.92
Unified | 62.93 | 62.35 | 61.87 | 62.40
Interpolation | 67.09 | 68.00 | 67.05 | 67.12
Unsupervised BLI |
VecMap | 55.29 | 56.96 | 57.18 | 59.33
Unified | 57.20 | 58.70 | 58.40 | 61.16
Interpolation | 61.00 | 61.52 | 62.70 | 63.40
Table 3:  P@1 of using WaCKy corpora.

The contextual representations are derived randomly from sentences of the
monolingual corpora. We study if this random derivation affects the
performances. Firstly, we run 5 times of randomly selecting 10 sentences to
gather contexts for each word type in the “Unified” setting. Figure
LABEL:fig:trials shows that the performance is stable in the 5 trials.
Secondly, we try randomly selecting 1-100 sentences to gather contexts for
each word type. Figure LABEL:fig:context shows that selecting 1 sentence will
drag the performance down to baseline, which indicates that 1 sentence is too
random to gather enough information for BLI.

We only list the studies on EN-ES due to space limit. Studies on other
language pairs can be found in the appendix.

####  4.5.3 The Influence of Selecting Encoder Layer

In the above experiments, we derive the contextual representations from the
first layer of the encoder of XML/mBART. In this section, we show how
different will be when we change the layer in the “Unified” setting. Figure
LABEL:fig:layers shows that as layer go higher, the performance drops. Please
refer to the appendix for performances of other language pairs.

####  4.5.4 Results of using WaCKy Corpora

WaCKy corpora is introduced in Dinu et al. (2014) for BLI, but only word
embeddings trained on WaCKy corpora are provided in their work. To obtain the
contextual representations, we find WaCKy corpora from
BUCC101010https://comparable.limsi.fr/bucc2020/bucc2020-task.html, and use the
corresponding dictionaries with the same training, validation, and test split.
We use mBART instead of XLM for computing the contextual representations in
this task.

Table 3 shows that our combination mechanism is robust on this dataset. Both
“Unified” and “Interpolation” perform better than the baselines.
“Interpolation” achieves significant improvements in the supervised setting.

##  5 Discussion

The static word embeddings in our paper are trained by skip-gram or CBOW,
while the word embeddings from XLM/mBART are trained by the pre-training
objectives. Different training objectives result in quite different word
embeddings. Actually, they show remarkably different behavior for BLI. By
using VecMap, the word embeddings from XLM/mBART perform averagely around 30
points lower than the static word embeddings used in our paper. We also test
fastText with 1024 dimension and word2vec with 300 dimension for fair
comparison. They all perform remarkably better than the word embeddings from
XLM/mBART. We plug in the word embeddings from XLM/mBART in place of the
fastText static embeddings in our combination approach, and obtain much worse
performance. This indicates that the static word embeddings trained by skip-
gram or CBOW are more suitable for BLI and our combination approach.

In addition, regarding the asymmetry in Figure LABEL:fig:framework, we used to
try the spring function that takes the static word embeddings as input, but
got much worse results. This indicates that the spring function conditioned on
the static space may be not helpful for BLI. Such observation may also explain
that symmetrizing Figure LABEL:fig:framework by yielding two unified spaces to
combine performs slightly worse than the asymmetry version of Figure 1 of this
paper. This is because that the spring function conditioned on the static
space is introduced to maintain the symmetry, while this introduced spring
function is not helpful for the combination.

##  6 Conclusion

Most BLI systems use either the static word embeddings or the contextual
representations, but there is no works to combine both. In this paper, we
propose a combination mechanism, which consists of the unified word
representations and the similarity interpolation. The unified word
representations use a spring network to pull the static word embeddings with
offsets produced by the contextual representations, and compose a unified
space such that parallel words are nearest neighbors to each other. The
similarity interpolation is applied afterward to interpolate the similarities
in the unified space and the contextual representation space. BLI experiments
on multiple language pairs show that our combination mechanism can utilize the
merits of both the static word embeddings and the contextual representations,
achieving significant improvements over robust baseline systems in both the
supervised and the unsupervised BLI tasks.

## Acknowledgments

The authors would like to thank the anonymous reviewers for the helpful
comments. This work was supported by National Natural Science Foundation of
China (Grant No. 62036004, 61673289)

## References

  * Aldarmaki and Diab (2019) Hanan Aldarmaki and Mona Diab. 2019.  Context-aware cross-lingual mapping.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 3906–3911.
  * Artetxe et al. (2016) Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2016.  Learning principled bilingual mappings of word embeddings while preserving monolingual invariance.  In _Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing_ , pages 2289–2294.
  * Artetxe et al. (2017) Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2017.  Learning bilingual word embeddings with (almost) no bilingual data.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 451–462.
  * Artetxe et al. (2018a) Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018a.  Generalizing and improving bilingual word embedding mappings with a multi-step framework of linear transformations.  In _Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence_ , pages 5012–5019.
  * Artetxe et al. (2018b) Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018b.  A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 789–798.
  * Artetxe et al. (2019) Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2019.  Bilingual lexicon induction through unsupervised machine translation.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 5002–5007.
  * Cao et al. (2020) Steven Cao, Nikita Kitaev, and Dan Klein. 2020.  Multilingual alignment of contextual word representations.  In _Proceedings of International Conference on Learning Representations_.
  * Conneau et al. (2017) Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2017.  Word translation without parallel data.  _arXiv preprint arXiv:1710.04087_.
  * Devlin et al. (2019) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019.  BERT: Pre-training of deep bidirectional transformers for language understanding.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 4171–4186.
  * Dinu et al. (2014) Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. 2014.  Improving zero-shot learning by mitigating the hubness problem.  _arXiv preprint arXiv:1412.6568_.
  * Fung and Yee (1998) Pascale Fung and Lo Yuen Yee. 1998.  An ir approach for translating new words from nonparallel, comparable texts.  In _36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Volume 1_ , pages 414–420.
  * Haghighi et al. (2008) Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008.  Learning bilingual lexicons from monolingual corpora.  In _Proceedings of the Annual Meeting of the Association for Computational Linguistics 2008: HLT_ , pages 771–779.
  * Irvine and Callison-Burch (2013) Ann Irvine and Chris Callison-Burch. 2013.  Combining bilingual and comparable corpora for low resource machine translation.  In _Proceedings of the eighth workshop on statistical machine translation_ , pages 262–270.
  * Joulin et al. (2018) Armand Joulin, Piotr Bojanowski, Tomas Mikolov, Hervé Jégou, and Edouard Grave. 2018.  Loss in translation: Learning bilingual word mapping with a retrieval criterion.  In _Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing_ , pages 2979–2984.
  * Koehn et al. (2007) Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007\.  Moses: Open source toolkit for statistical machine translation.  In _Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions_ , pages 177–180.
  * Kulshreshtha et al. (2020) Saurabh Kulshreshtha, José Luis Redondo-García, and Ching-Yun Chang. 2020\.  Cross-lingual alignment methods for multilingual bert: A comparative study.  _arXiv preprint arXiv:2009.14304_.
  * Lample and Conneau (2019) Guillaume Lample and Alexis Conneau. 2019.  Cross-lingual language model pretraining.  In _Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)_ , pages 7059–7069.
  * Liu et al. (2020) Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020.  Multilingual denoising pre-training for neural machine translation.  _arXiv preprint arXiv:2001.08210_.
  * Mikolov et al. (2013) Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013.  Exploiting similarities among languages for machine translation.  _arXiv preprint arXiv:1309.4168_.
  * Mohiuddin et al. (2020) Tasnim Mohiuddin, M Saiful Bari, and Shafiq Joty. 2020.  Lnmap departures from isomorphic assumption in bilingual lexicon induction through non-linear mapping in latent space.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing_ , pages 2712–2723.
  * Mohiuddin and Joty (2019) Tasnim Mohiuddin and Shafiq Joty. 2019.  Revisiting Adversarial Autoencoder for Unsupervised Word Translation with Cycle Consistency and Improved Training.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 3857–3867.
  * Patra et al. (2019) Barun Patra, Joel Ruben Antony Moniz, Sarthak Garg, Matthew R Gormley, and Graham Neubig. 2019.  Bilingual lexicon induction with semi-supervision in non-isometric embedding spaces.  In _Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics_ , pages 184–193.
  * Qi et al. (2018) Ye Qi, Devendra Singh Sachan, Matthieu Felix, Sarguna Janani Padmanabhan, and Graham Neubig. 2018.  When and why are pre-trained word embeddings useful for neural machine translation?  _arXiv preprint arXiv:1804.06323_.
  * Rapp (1999) Reinhard Rapp. 1999.  Automatic identification of word translations from unrelated english and german corpora.  In _Proceedings of the 37th annual meeting of the Association for Computational Linguistics_ , pages 519–526.
  * Ravi and Knight (2011) Sujith Ravi and Kevin Knight. 2011.  Deciphering foreign language.  In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_ , pages 12–21.
  * Ruder et al. (2019) Sebastian Ruder, Ivan Vulić, and Anders Søgaard. 2019.  A survey of cross-lingual word embedding models.  _Journal of Artificial Intelligence Research_ , 65:569–631.
  * Schuster et al. (2019) Tal Schuster, Ori Ram, Regina Barzilay, and Amir Globerson. 2019.  Cross-lingual alignment of contextual word embeddings, with applications to zero-shot dependency parsing.  In _Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)_ , pages 1599–1613.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.  Neural machine translation of rare words with subword units.  In _Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1715–1725.
  * Shi et al. (2021) Haoyue Shi, Luke Zettlemoyer, and Ikbal Sida. 2021.  Bilingual lexicon induction via unsupervised bitext construction and word alignment.  _arXiv preprint arXiv:2101.00148_.
  * Smith et al. (2017) Samuel L Smith, David HP Turban, Steven Hamblin, and Nils Y Hammerla. 2017.  Offline bilingual word vectors, orthogonal transformations and the inverted softmax.  _arXiv preprint arXiv:1702.03859_.
  * Søgaard et al. (2018) Anders Søgaard, Sebastian Ruder, and Ivan Vulić. 2018.  On the limitations of unsupervised bilingual dictionary induction.  In _Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 778–788.
  * Vulić and Moens (2013) Ivan Vulić and Marie Francine Moens. 2013.  A study on bootstrapping bilingual vector spaces from non-parallel data (and nothing else).  In _Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing_ , pages 1613–1624.
  * Wang et al. (2020) Zirui Wang, Jiateng Xie, Ruochen Xu, Yiming Yang, Graham Neubig, and Jaime Carbonell. 2020.  Cross-lingual alignment vs joint training: A comparative study and a simple unified framework.  In _Proceedings of International Conference on Learning Representations_.
  * Xiao and Guo (2014) Min Xiao and Yuhong Guo. 2014.  Distributed word representation learning for cross-lingual dependency parsing.  In _Proceedings of the Eighteenth Conference on Computational Natural Language Learning_ , pages 119–129.
  * Xing et al. (2015) Chao Xing, Dong Wang, Chao Liu, and Yiye Lin. 2015.  Normalized word embedding and orthogonal transform for bilingual word translation.  In _Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1006–1011.
  * Zhang et al. (2017a) Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017a.  Adversarial training for unsupervised bilingual lexicon induction.  In _Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)_ , pages 1959–1970.
  * Zhang et al. (2017b) Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017b.  Earth mover’s distance minimization for unsupervised bilingual lexicon induction.  In _Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing_ , pages 1934–1945.
  * Zhang et al. (2016) Yuan Zhang, David Gaddy, Regina Barzilay, and Tommi Jaakkola. 2016.  Ten pairs to tag – multilingual POS tagging via coarse mapping between embeddings.  In _Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies_ , pages 1307–1317.

## Appendix

##  Appendix A Experiment Environment and Settings

Our experiment is running on a Linux machine with GTX 1080Ti. The version of
cuDNN is 7.6.0 and the version of CUDA is 10.1. We also use the PyTorch deep
learning framework. The version of PyTorch is 1.6. The average runtime in our
experiment is 5 to 10 minutes for one language pair, excluding the time for
training word embeddings by fastText.

Regarding Wikipedia corpora used in our experiments, their download links are:
https://dumps.wikimedia.org/${lg}wiki/latest/${lg}wiki-latest-pages-
articles.xml.bz2,
where ${lg} should be replaced with the corresponding languages. ${lg} can be
set as “en”, “es”, “ar”, “zh”, “de”, and “fr”. We download them on Jan. 2,
2021. Regarding WaCKy corpora, we use WaCKy
corpora111111https://corpus.leeds.ac.uk/serge/bucc provided by BUCC2020 for
obtaining the contextual representations, and use the provided word embeddings
trained on this corpora. The BUCC2020 dictionaries are downloaded from the
same page. We concat the three training sets with high, mid, low frequency as
one training set, and concat the three test sets with high, mid, low frequency
as one test set. Because no translations are provided in the test set of
BUCC2020, we look up the Muse dictionaries to get the translations. All source
words in the test set can find translations in the Muse dictionaries.

The evaluation script for computing precision@1 (P@1) is:
https://github.com/artetxem/vecmap.git/eval_translation.py.

##  Appendix B Parameter Settings

The parameter size of the spring network is 796202. We use default parameter
settings of VecMap and RCSLS.

language | $\lambda$
---|---
EN$\to$ES | 0.11
ES$\to$EN | 0.05
EN$\to$AR | 0.10
AR$\to$EN | 0.30
EN$\to$ZH | 0.12
ZH$\to$EN | 0.10
EN$\to$DE | 0.11
DE$\to$EN | 0.25
EN$\to$FR | 0.13
FR$\to$EN | 0.11
Table 4: The optimal hyperparameter $\lambda$ in the unsupervised settings

Regarding the hyperparameter $\lambda$ in the similarity interpolation, we
search the optimal value in [0.05-0.3] with step size of 0.01. We found
$\lambda=0.1$ is superior on validation sets in all supervised settings. The
optimal value of $\lambda$ found by the unsupervised tuning procedure in the
unsupervised settings, which is introduced in section 3.2, is shown in Table
4. In the search range, the performances has low variance, and are better than
the baselines.

##  Appendix C The Influence of Selecting Encoder Layer in Other Language
Pairs

We report the influences on EN-AR, EN-ZH, EN-DE, EN-FR in Figure 5. It shows
that as we select higher layers for deriving the contextual representations,
the performances become lower. This trend exists in most language pairs,
except that the trend in EN-ZH is not significant.

(a) EN-AR

(b) EN-DE

(c) EN-FR

(d) EN-ZH

Figure 5: Performances of selecting different layer of XLM encoder to derive
the contextual representations.

##  Appendix D The Randomness of Contexts in Other Language Pairs

We report the randomness analyses on EN-AR, EN-ZH, EN-DE, EN-FR in Figure
LABEL:fig:randomContext1. It shows that trying 5 times of selecting 10 random
sentences for gathering contexts gets stable performances in all language
pairs. In most cases, using 1 sentence for computing the contextual
representation drags the performance down, which indicates the inadequacy of 1
sentence for gathering contexts.

##  Appendix E Performances on Words with Different Frequencies

We use WaCKy corpora and the dictionaries provided by
BUCC2020121212https://comparable.limsi.fr/bucc2020/bucc2020-task.html to study
the performances on words with different frequencies. The provided
dictionaries are divided into groups of high frequency words, mid frequency
words, and low frequency words. We test our combination mechanism on these
three groups respectively. The results are presented in Table 5 and Table 6.
We can see that our combination mechanism is effective on all three groups.

| EN$\to$DE | DE$\to$EN | EN$\to$FR | FR$\to$EN
---|---|---|---|---
High Frequency
VecMap | 67.60 | 75.27 | 79.00 | 79.27
Unified | 71.27 | 77.40 | 79.67 | 80.80
Interpolation | 72.53 | 79.67 | 80.47 | 82.13
Middle Frequency
VecMap | 67.20 | 71.47 | 77.73 | 80.27
Unified | 67.80 | 73.53 | 78.33 | 81.00
Interpolation | 70.60 | 78.13 | 81.13 | 83.73
Low Frequency
VecMap | 69.53 | 74.73 | 78.47 | 81.87
Unified | 70.00 | 75.53 | 79.27 | 82.67
Interpolation | 77.53 | 82.47 | 87.00 | 87.07
Table 5: Performances on words with different frequencies in the unsupervised setting | EN$\to$DE | DE$\to$EN | EN$\to$FR | FR$\to$EN
---|---|---|---|---
High Frequency
VecMap | 67.87 | 76.13 | 78.27 | 79.73
Unified | 70.40 | 78.07 | 78.4 | 81.4
Interpolation | 71.00 | 78.90 | 78.90 | 82.13
Middle Frequency
VecMap | 68.87 | 71.8 | 78.47 | 80.87
Unified | 69.93 | 75.20 | 79.53 | 82.20
Interpolation | 71.20 | 76.33 | 80.07 | 83.73
Low Frequency
VecMap | 70.60 | 76.13 | 79.67 | 82.93
Unified | 72.67 | 78.27 | 81.20 | 84.20
Interpolation | 74.13 | 79.97 | 82.30 | 84.73
Table 6: Performances on words with different frequencies in the supervised
setting