# Unification of quantum and classical correlations and quantumness measures

Kavan Modi<EMAIL_ADDRESS>Centre for Quantum Technologies, National
University of Singapore, Singapore  Vlatko Vedral  Centre for Quantum
Technologies, National University of Singapore, Singapore  Department of
Physics, National University of Singapore, Singapore  Clarendon Laboratory,
University of Oxford, Oxford UK

###### Abstract

We give a pedagogical introduction to quantum discord. We the discuss the
problem of separation of total correlations in a given quantum state into
entanglement, dissonance, and classical correlations using the concept of
relative entropy as a distance measure of correlations. This allows us to put
all correlations on an equal footing. Entanglement and dissonance, whose
definition is introduced here, jointly belong to what is known as quantum
discord. Our methods are completely applicable for multipartite systems of
arbitrary dimensions. We finally show, using relative entropy, how different
notions of quantum correlations are related to each other. This gives a single
theory that incorporates all correlations, quantum, classical, etc.

##  I Introduction

Quantum systems are correlated in ways inaccessible to classical objects. A
distinctive quantum feature of correlations is quantum entanglement (Einstein
_et al._ , 1935; Schrödinger, 1935; Bohr, 1935). Entangled states are
nonclassical as they cannot be prepared with the help of local operations and
classical communication (LOCC) Horodecki  _et al._ (2009). However, it is not
the only aspect of nonclassicality of correlations due to the nature of
operations allowed in the framework of LOCC. To illustrate this, one can
compare a classical bit with a quantum bit; in the case of full knowledge
about a classical bit, it is completely described by one of two locally
distinguishable states, and the only allowed operations on the classical bit
are to keep its value or flip it. To the contrary, quantum operations can
prepare quantum states that are indistinguishable for a given measurement.
Such operations and classical communication can lead to separable states
(those which can be prepared via LOCC) which are mixtures of locally
indistinguishable states. These states are nonclassical in the sense that they
cannot be prepared using classical operations on classical bits.

Recent measures of nonclassical correlations are motivated by different
notions of classicality and operational means to quantify nonclassicality
Henderson and Vedral (2001); Ollivier and Zurek (2001); Oppenheim  _et al._
(2002); Groisman  _et al._ (2005); Luo (2008); Wu  _et al._ (2009); Modi  _et
al._ (2010). Quantum discord has received much attention in studies involving
thermodynamics and correlations Zurek (2003); Horodecki  _et al._ (2005);
Rodriguez-Rosario  _et al._ (2008); Devi and Rendell (2008); Datta  _et al._
(2008); Piani  _et al._ (2008); mazzola; Dakić  _et al._ (2010); Chen  _et
al._ (2011); Cavalcanti  _et al._ (2010); Piani  _et al._ (2011). These works
are concerned with understanding the role of quantumness of correlations in a
variety of systems and tasks. In some of the studies, it is also desirable to
compare various notions of quantum correlations. It is well known that the
different measures of quantum correlation are not identical and conceptually
different. For example, the discord does not coincide with entanglement or
measurement induced disturbance and a direct comparison of any two of these
notions can be rather meaningless. Therefore, an unified classification of
correlations is in demand as well as a unification of different notions of
quantumness. In this article, using relative entropy, we resolve some of these
issues by introducing measures for classical and nonclassical correlations for
quantum states under a single theory. Our theory further allows us to connect
different notions of quantumness. This will allow us to generalize all
measures of quantumness for multipartite systems in symmetric and asymmetric
manners. We begin with a pedagogical introduction to quantumness of
correlations.

##  II Conditional entropy

Figure 1: _Conditional entropy._ The Venn diagram shows the joint entropy
$H(ab)$, marginal entropies $H(a)$ and $H(b)$, conditional entropies, $H(a|b)$
and $H(b|a)$, and mutual information $I(a:b)$ a joint classical probability
distribution for (correlated) random variables $a$ and $b$.

The story of quantumness of correlations beyond-entanglement begins with the
non-uniqueness of quantum conditional entropy. In classical probability
theory, conditional entropy is defined as

$\displaystyle H(b|a)=H(ab)-H(a).$ (1)

It is the measure ignorance of $b$ has given some knowledge of state of $a$.
Fig. 1) depicts this relationship in a graphical manner. Another way to
express the conditional entropy is as the ignorance of $b$ when the state of
$a$ is known to be in the $i$th state, weighted by the probability for $i$th
outcome as

$\displaystyle H(b|a)=\sum_{i}p^{a}_{i}H(b|a=i),$ (2)

It is the classical-equivalency of Eqs. 1 and 2 give rise to quantumness of
correlations and in specific quantum discord Datta (2008). This is due to the
fact that these two equations are not the same in quantum theory. While the
first simply takes difference in the joint ignorance and the ignorance of $a$,
the second formula depends on specific outcomes of $a$, which requires a
measurement. Measurements in quantum theory are basis dependent and change the
state of the system.

In generalizing the classical concepts above to quantum we replace joint
classical-probability distributions with density operators and Shannon’s with
von Neumann’s entropy. How do we deal with conditional entropy then? Clearly
there are two options: Eqs. 1 and 2. Let is deal with Eq. 1 first and define
quantum conditional entropy as

$\displaystyle S^{(1)}(B|A)=S(AB)-S(A).$ (3)

This is well known quantity in quantum information theory Schumacher and
Nielsen (1996) and negative of this quantity is known as coherent information.
However, this is a troubling quantity as it can be negative for entangled
states and for a long time there was no way to interpret the negativity. This
is in start contrast with the classical conditional entropy which has a clear
interpretation and is always positive.

On the other hand, we can give define the quantum version of Eq. 2 by making
measurements on party $A$. To put in the details, the joint state $\rho^{AB}$
is measured by $A$ giving $i$th outcome:

$\displaystyle\rho^{AB}\rightarrow\sum_{i}\Pi^{A}_{i}\rho^{AB}\Pi^{A}_{i}=\sum_{i}p_{i}|i\rangle\langle
i|\otimes\rho^{B}_{i},$ (4)

where $\Pi_{i}$ are rank one _positive operator values measures_ , $|i\rangle$
are classical flags on measuring apparatus indicating the measurement outcome,
$p_{i}=\mbox{Tr}[\Pi^{A}_{i}\rho^{AB}]$ is probability of $i$th outcome,
$\rho^{B}_{i}=\mbox{Tr}_{A}[\Pi^{A}_{i}\rho^{AB}]$. The conditional entropy of
$B$ is then clearly defined as

$\displaystyle S^{(2)}(B|A)=S(B|\\{\Pi_{i}\\})=\sum_{i}p_{i}S(\rho^{B}_{i}),$
(5)

This definition of conditional entropy is always positive. The obvious problem
with this definition is that the state $\rho^{AB}$ changes after the
measurement. Also note that this quantity is not symmetric under party swap.

A different approach to conditional entropy is taken in Cerf and Adami (1997,
1999), where a quantum _conditional amplitude_ (analogous to classical
conditional probability) is defined such that it satisfies Eq. 3. We only mean
to suggest that the two approaches above are not the only options available.
Different approaches give us different distinctions of quantum theory from the
classical theory. And in someway different notions of quantumness.

##  III Quantumness of correlations

Clearly the two definition of conditional entropies above are different in
quantum theory. The first one suffers from negativity and second one is
‘classicalization’ of a quantum state. Let us now derive quantum discord and
relate it to the preceding section. We start with the concept of mutual
information:

$\displaystyle I(a:b)=H(a)+H(b)-H(ab)$ (6)

and using Eq. 2

$\displaystyle J(b|a)=H(b)-\sum_{i}p^{a}_{i}H(b|a=i).$ (7)

Clearly the two classical mutual information above are the same, but not in
quantum theory. This is precisely what was noted by Ollivier and Zurek, and
they called the difference between $I$ and $J$ _quantum discord_ :

$\displaystyle\delta(B|A)=I(A:B)-J(B|A).$ (8)

Working out the details one finds that quantum discord is simply,

$\displaystyle\delta(B|A)=S^{(2)}(B|A)-S^{(1)}(B|A),$ (9)

the difference in two definition of conditional entropy.

Henderson and Vedral Henderson and Vedral (2001) had also looked at $J(B|A)$
called it classical correlations. In fact they advocated to that
$\max_{\\{\Pi_{i}\\}}J(B|A)$ to be the classical correlations. Which meant
that quantum discord is best defined as

$\displaystyle\delta(B|A)=\min_{\\{\Pi_{i}\\}}[I(A:B)-J(B|A)].$ (10)

Since conditional entropy in Eq. 5 is asymmetric under party swap, quantum
discord is also asymmetric under party swap.

A side note should be made at this point. The interpretation of negativity of
quantum conditional entropy in Eq. 3 was given in terms of task known as state
merging Horodecki  _et al._ (2005), and we will see shortly that a similar
task gives quantum discord an operational meaning. While the minimum of Eq. 5
over all POVM is related to entanglement of formation between $B$ and $C$, a
purification of $AB$: $E_{F}(BC)=\min_{\\{\Pi_{i}\\}}S^{(2)}(B|A)$ Koashi and
Winter (2004). Putting the two together leads to an task dependent operation
interpretation of quantum discord Cavalcanti  _et al._ (2010). Barring the
details, we can say that quantum discord between $A$ and $B$, as measured by
$A$ is equal to the consumption of entanglement in state preparation of $BC$
plus state merging in those two parties. Additionally, state merging and other
tasks that involve conditional entropies are asymmetric under party swap and a
natural interpretation of asymmetry of quantum discord arises.

The minimization over all POVM of quantum is not an easy problem to deal with
in general. A similar quantity called _measurement induced disturbance_ (MID)
was introduced to deal with this difficulty. MID is defined as the difference
in the mutual information of a joint state, $\rho^{AB}$ and it’s dephased
version $\chi^{AB}$. The dephasing takes place in the marginal basis, leaving
the marginal states unchanged:

$\displaystyle MID=I(\rho^{AB})-I(\chi^{AB})=S(\rho^{AB})-S(\chi^{AB}).$ (11)

We will come back to MID later in the article.

##  IV Unification of correlations

Figure 2: _Correlations as a distance._ The large ellipse represents the set
of all states with the set of separable states in the smaller ellipse. The
squares represent the set of classical states, and the dots within the squares
are the sets of product states. $\rho$ is an entangled state and $\sigma$ is
the closest separable state. The correlations are entanglement, $E$, discord,
$D$, and dissonance, $Q$.

Both of the measures above are defined in terms of mutual information and
therefore very difficult to generalize for multipartite case Chakrabarty  _et
al._ (2010). Below we will get over that hurdle by examining classical states,
states that have no quantum correlations. It is easy verify that a state has
zero discord and MID simultaneously. What that means is that such a state has
equal value for conditional entropies in Eqs. 3 and 5. Such a state is called
a _classical state_ and has the form

$\displaystyle\chi^{AB}=\sum_{i}p_{i}|i\rangle\langle i|\otimes\rho^{B}_{i}$
(12)

when measurements are made by $A$ and $\chi^{AB}=\sum_{j}\rho^{A}_{j}\otimes
p_{j}|j\rangle\langle j|$ when measurements are made by $B$. It is then easy
to see that a symmetric classical state must have the form

$\displaystyle\chi^{AB}=\sum_{ij}p_{ij}|ij\rangle\langle ij|.$ (13)

Further the conditional amplitude defined in Cerf and Adami (1997, 1999) is
not a density operator and may behave strangely. In Brodutch and Terno (2010)
it is shown that the conditional amplitude reduces to a density operator when
the state is classical.

Based on the definition of classical states we may now introduce a measure of
quantum correlations as a distance from a given state to the closest classical
state. The distance from a state to a state without the desired property (e.g.
entanglement or discord) is a measure of that property. For example, the
distance to the closest separable state is a meaningful measure of
entanglement. If the distance is measured with relative entropy, the resulting
measure of entanglement is the relative entropy of entanglement Vedral  _et
al._ (1997); Vedral and Plenio (1998). Using relative entropy we define
measures of nonclassical correlations as a distance to the closest classical
states Modi  _et al._ (2010), though many other distance measures can serve
just as well Dakić  _et al._ (2010). We call our measure of quantum
correlations _relative entropy of discord_.

Since all the distances are measured with relative entropy, this provides a
consistent way to compare different correlations, such as entanglement,
discord, classical correlations, and _quantum dissonance_ , a new quantum
correlation that may be present in separable states. Dissonance is a similar
notion to discord, but it excludes entanglement. Lastly, here we have to make
no mention of whether we want a symmetric discord measure or asymmetric, or
the number of parties to be involved. We simply have to choose the appropriate
classical state and no ambiguity is left. A graphical illustration is given in
Fig. 2.

Let us briefly define the types states discussed below. A product state of
$N$-partite system, a state with no correlations of any kind, has the form of
$\pi=\pi_{1}\otimes\dots\otimes\pi_{N}$, where $\pi_{n}$ is the reduced state
of the $n$th subsystem. The set of product states, $\mathcal{P}$, is not a
convex set in the sense a mixture of product states may not be another product
state. The set of classical states, $\mathcal{C}$, contains mixtures of
locally distinguishable states $\chi=\sum_{k_{n}}p_{k_{1}\dots
k_{N}}|k_{1}\dots k_{N}\rangle\langle k_{1}\dots
k_{N}|=\sum_{\vec{k}}p_{\vec{k}}|\vec{k}\rangle\langle\vec{k}|$, where
$p_{\vec{k}}$ is a joint probability distribution and local states
$|k_{n}\rangle$ span an orthonormal basis. The correlations of these states
are identified as classical correlations Henderson and Vedral (2001); Ollivier
and Zurek (2001); Oppenheim  _et al._ (2002); Modi  _et al._ (2010). Note that
$\mathcal{C}$ is not a convex set; mixing two classical states written in
different bases can give rise to a nonclassical state. The set of separable
states, $\mathcal{S}$, is convex and contains mixtures of the form
$\sigma=\sum_{i}p_{i}\pi_{1}^{(i)}\otimes\dots\otimes\pi_{N}^{(i)}$. These
states can be prepared using only local quantum operations and classical
communication Werner (1989) and can possess nonclassical features Henderson
and Vedral (2001); Ollivier and Zurek (2001). The set of product states is a
subset of the set of classical states which in turn is a subset of the set of
separable states. Finally, entangled states are all those which do not belong
to the set of separable states. The set of entangled states, $\mathcal{E}$, is
not a convex set either.

The relative entropy between two quantum states $x$ and $y$ is defined as
$S(x||y)\equiv-\mbox{tr}(x\log y)-S(x)$, where $S(x)\equiv-\mbox{tr}(x\log x)$
is the von Neumann entropy of $x$. The relative entropy is a non-negative
quantity and due to this property it often appears in the context of distance
measure though technically it is not a distance, e.g. it is not symmetric. In
Fig. 3, we present all possible types of correlations present in a quantum
state $\rho$. $T_{\rho}$ is the _total mutual information_ of $\rho$ given by
the distance to the closest product state. If $\rho$ is entangled, its
entanglement is measured by the relative entropy of entanglement, $E$, which
is the distance to the closest separable state $\sigma$. Having found
$\sigma$, one then finds the closest classical state, $\chi_{\sigma}$, to it.
This distance, denoted by $Q$, contains the rest of nonclassical correlations
(it is similar to discord Henderson and Vedral (2001); Ollivier and Zurek
(2001) but entanglement is excluded). We call this quantity _quantum
dissonance_. Alternatively, if we are interested in relative entropy of
discord, $D$, then we find the distance between $\rho$ and closest classical
state $\chi_{\rho}$. Summing up, we have the following nonclassical
correlations:

$\displaystyle E=$
$\displaystyle\min_{\sigma\in\mathcal{S}}S(\rho||\sigma)\quad\textrm{(entanglement)},$
(14) $\displaystyle D=$
$\displaystyle\min_{\chi\in\mathcal{C}}S(\rho||\chi)\quad\textrm{(quantum
discord)},$ (15) $\displaystyle Q=$
$\displaystyle\min_{\chi\in\mathcal{C}}S(\sigma||\chi)\quad\textrm{(quantum
dissonance)}.$ (16)

Next, we compute classical correlations as the minimal distance between a
classically correlated state, $\chi$, and a product state, $\pi$:
$C=\min_{\pi\in\mathcal{P}}S(\chi||\pi)$. Finally, we compute the quantities
labeled $L_{\rho}$ and $L_{\sigma}$ in Fig. 3, which give us additivity
conditions for correlations.

$\sigma$$T_{\sigma}$$Q$$\rho$$E$$D$$T_{\rho}$$\chi_{\sigma}$$C_{\sigma}$$\chi_{\rho}$$C_{\rho}$$\pi_{\sigma}$$L_{\sigma}$$\pi_{\chi_{\sigma}}$$\pi_{\chi_{\rho}}$$\pi_{\rho}$$L_{\rho}$
Figure 3: _Correlations in a quantum state._ An arrow from $x$ to $y$, $x\to
y$, indicates that $y$ is the closest state to $x$ as measured by the relative
entropy $S(x||y)$. The state $\rho\in\mathcal{E}$ (the set of entangled
states), $\sigma\in\mathcal{S}$ (the set of separable states),
$\chi\in\mathcal{C}$ (the set of classical states), and $\pi\in\mathcal{P}$
(the set of product states). The distances are entanglement, $E$, quantum
discord, $D$, quantum dissonance, $Q$, total mutual information, $T_{\rho}$
and $T_{\sigma}$, and classical correlations, $C_{\sigma}$ and $C_{\rho}$. All
relative entropies, except for entanglement, reduce to the differences in
entropies of $y$ and $x$, $S(x||y)=S(y)-S(x)$. With the aid of $L_{\rho}$ and
$L_{\sigma}$ the closed path are additive.

Skipping the details of the calculations (presented in Modi  _et al._ (2010))
we give the final results. The surprisingly simple results and is summarized
in Fig. 3. First, we find that the closest classical state is obtained by
making rank one POVM measurement on the quantum state

$\displaystyle\chi=\sum_{\vec{k}}|\vec{k}\rangle\langle\vec{k}|\rho|\vec{k}\rangle\langle\vec{k}|,$
(17)

where $|\vec{k}\rangle$ is projection in space at most of dimension $d^{2}$.
To find relative entropy of discord one has to optimize over all rank one
POVM.

$\displaystyle D=S(\rho||\chi)=\min_{\\{\Pi_{i}\\}}S(\chi)-S(\rho).$ (18)

Therefore finding the closest classical state is still a very difficult
problem and has the same challenged as faced in computing original discord.

We find that all correlations (except entanglement) in Fig. 3 are given by
simply taking the difference in entropy of the state at tail of the arrow from
the entropy of the state at the tip of the arrow, i.e. $S(x||y)=S(y)-S(x)$ for
all solid lines. Which means that a closed loop of (solid lines) yield
correlations that are additive, i.e. $D+C_{\rho}=T_{\rho}+L_{\rho}$ or
$T_{\rho}-C_{\rho}=D-L_{\rho}$, which is actually the original discord. We
show how these two measures are related to each other below. See Modi  _et
al._ (2010) for details presented in this section.

##  V Unifying quantumness measures

Next we note, there are four fundamental elements involved in study of
quantumness of correlations. The first of it is the quantum state, $\rho$.
Given a quantum state we immediately have its marginals, $\pi_{\rho}$. The
third element is the classical state $\chi$, obtained by dephasing $\rho$ in
some basis. And the final element is the marginals of $\chi_{\rho}$. It then
turns out that different measures of quantumness are different because they
put different constrain in the relationships these four elements have with
each other. We have illustrated this in Fig. 4.

In Fig. 4a the four fundamental elements are shown. Figs. 4b-d show how three
measures of quantumness are found using the four elements. The original
discord maximizes the distance from the classical state and its marginals.
This has the meaning that the classical state is least confusing from its
marginals. Quantum discord is the defined as the difference in confusion of a
quantum state with its marginals and the a classical state obtained from that
quantum state and it confusion with its marginals. Similarly MID attempts to
minimize the confusion between the classical state and marginals of the
original quantum state. This has the effect that the marginals of the quantum
state are the same as the marginals of the classical state. Finally relative
entropy of discord is defined as the distance between a quantum state and its
closest classical state.

$\rho$$T$$\pi\rho$$D$$\chi_{\rho}$$C$$L$$\pi_{\chi}$a. Elements of quantumness
$\rho$$\pi\rho=\pi_{\chi}$$\chi_{\rho}$minimizec.
MID$\rho$$\pi\rho$$\chi_{\rho}$$\pi_{\chi}$maximizeb.
Originaldiscord$\rho$$\pi\rho$minimize$\chi_{\rho}$$\pi_{\chi}$d. RED Figure
4: _Measures of quantumness._ a. Fundamental elements needed in defining a
measure of quantumness of correlations. b. Original quantum discord. c.
Measurement induced disturbance. d. Relative entropy of discord.

We now turn our attention to show that using relative entropy we can describe
(and generalize) other quantumness measures such as original _quantum discord_
Ollivier and Zurek (2001), _symmetric quantum discord_ Wu  _et al._ (2009),
and _measurement induced disturbance_. Luo (2008).

Vedral et al. Vedral  _et al._ (1997) show that quantum relative entropy has
the operational meaning of being able to confuse two quantum states. The
argument goes as the following: suppose you are given either $\rho$ or
$\sigma$ and you have to determine which by making $N$ measurements (POVM).
The probability of confusing the two states is

$\displaystyle P_{N}=e^{-NS(\rho||\sigma)}.$ (19)

Now suppose $\rho$ is an entangled state. Then for what separable state
$\sigma$ can be confused for $\rho$ the most? The answer is

$\displaystyle P_{N}=e^{-N\min_{\sigma\in\mathcal{S}}S(\rho||\sigma)},$ (20)

where $\mathcal{S}$ is the set of separable states. This is the meaning of
relative entropy of entanglement:

$\displaystyle E(\rho)=\min_{\sigma\in\mathcal{S}}S(\rho||\sigma).$ (21)

In similar manner we can give meaning to relative entropy of discord as

$\displaystyle D(\rho)=\min_{\chi\in\mathcal{C}}S(\rho||\chi)$ (22)

the classical state $\chi$ that imitates $\rho$ the most.

The great advantage of looking at these measures in this manner is that, now
they are no longer constrained to be bipartite measures. Nor they are
constrained to be symmetric or asymmetric under party exchanges. We can now
define quantum discord by $n-$partite systems with measurements on $m$
subsystems. Similarly, MID can be defined in such a manner as well. The other
advantage is that we know how these elements are related to each other and
that there are only finite number of relationships among them that make sense,
e.g. maximization of distance between a quantum state and a classical state
does not make sense as on may get infinity for the result.

##  VI Conclusions

We have given a pedagogical review of ideas behind quantum correlations beyond
entanglement. In doing so we were able to generalize the concepts of quantum
discord to multipartite case, with no ambiguity regarding the asymmetry of
quantum discord under party exchange. We have shown how three measures of
quantumness can be viewed under a single formalism using relative entropy.

###### Acknowledgements.

We acknowledge the financial support by the National Research Foundation and
the Ministry of Education of Singapore. We are grateful to the organizers of
the _75 years of entanglement_ in Kolkata for inviting our participation.

## References

  * Einstein  _et al._ (1935) A. Einstein, B. Podolsky, and N. Rosen, Phys. Rev., 47, 777 (1935).
  * Schrödinger (1935) E. Schrödinger, Naturwissenschaften, 23, 844 (1935).
  * Bohr (1935) N. Bohr, Phys. Rev., 48, 696 (1935).
  * Horodecki  _et al._ (2009) R. Horodecki, P. Horodecki, M. Horodecki, and K. Horodecki, Reviews of Modern Physics, 81, 865 (2009).
  * Henderson and Vedral (2001) L. Henderson and V. Vedral, J. Phys. A, 34, 6899 (2001).
  * Ollivier and Zurek (2001) H. Ollivier and W. H. Zurek, Phys. Rev. Lett., 88, 017901 (2001).
  * Oppenheim  _et al._ (2002) J. Oppenheim, M. Horodecki, P. Horodecki, and R. Horodecki, Phys. Rev. Lett., 89, 180402 (2002).
  * Groisman  _et al._ (2005) B. Groisman, S. Popescu, and A. Winter, Phys. Rev. A, 72, 032317 (2005).
  * Luo (2008) S. Luo, Phys. Rev. A, 77, 022301 (2008).
  * Wu  _et al._ (2009) S. Wu, U. V. Poulsen, and K. Mølmer, Phys. Rev. A, 80, 032319 (2009).
  * Modi  _et al._ (2010) K. Modi, T. Paterek, W. Son, V. Vedral, and M. Williamson, Phys. Rev. Lett., 104, 080501 (2010).
  * Zurek (2003) W. H. Zurek, Phys. Rev. A, 67, 012320 (2003).
  * Horodecki  _et al._ (2005) M. Horodecki, P. Horodecki, R. Horodecki, J. Oppenheim, A. Sen(De), U. Sen, and B. Synak-Radtke, Phys. Rev. A, 71, 062307 (2005a).
  * Rodriguez-Rosario  _et al._ (2008) C. A. Rodriguez-Rosario, K. Modi, A. Kuah, A. Shaji, and E. C. G. Sudarshan, J. Phys. A, 41, 205301 (2008).
  * Devi and Rendell (2008) A. R. U. Devi and R. W. Rendell, Phys. Rev. Lett., 100, 140502 (2008).
  * Datta  _et al._ (2008) A. Datta, A. Shaji, and C. Caves, Phys. Rev. Lett., 100, 050502 (2008).
  * Piani  _et al._ (2008) M. Piani, P. Horodecki, and R. Horodecki, Phys. Rev. Lett., 100, 090502 (2008).
  * Dakić  _et al._ (2010) B. Dakić, V. Vedral, and i. c. v. Brukner, Phys. Rev. Lett., 105, 190502 (2010).
  * Chen  _et al._ (2011) L. Chen, E. Chitambar, K. Modi, and G. Vacanti, Phys. Rev. A, 83, 020101 (2011).
  * Cavalcanti  _et al._ (2010) D. Cavalcanti, L. Aolita, S. Boixo, K. Modi, M. Piani, and A. Winter, arXiv:1008.3205 (2010).
  * Piani  _et al._ (2011) M. Piani, S. Gharibian, G. Adesso, J. Calsamiglia, P. Horodecki, and A. Winter, arXiv:1103.4032 (2011).
  * Datta (2008) A. Datta, arXiv:0807.4490v1 (2008).
  * Schumacher and Nielsen (1996) B. Schumacher and M. A. Nielsen, Phys. Rev. A, 54, 2629 (1996).
  * Cerf and Adami (1997) N. J. Cerf and C. Adami, Phys. Rev. Lett., 79, 5194 (1997).
  * Cerf and Adami (1999) N. J. Cerf and C. Adami, Phys. Rev. A, 60, 893 (1999).
  * Brodutch and Terno (2010) A. Brodutch and D. R. Terno, Phys. Rev. A, 81, 062103 (2010).
  * Horodecki  _et al._ (2005) M. Horodecki, J. Oppenheim, and A. Winter, Nature, 436 (2005b).
  * Koashi and Winter (2004) M. Koashi and A. Winter, Phys. Rev. A, 69, 022309 (2004).
  * Chakrabarty  _et al._ (2010) I. Chakrabarty, P. Agrawal, and A. K. Pati, arXiv:1006.5784 (2010).
  * Vedral  _et al._ (1997) V. Vedral, M. B. Plenio, M. A. Rippin, and P. L. Knight, Phys. Rev. Lett., 78, 2275 (1997a).
  * Vedral and Plenio (1998) V. Vedral and M. Plenio, Phys. Rev. A, 57, 1619 (1998).
  * Werner (1989) R. F. Werner, Phys. Rev. A, 40, 4277 (1989).
  * Note (1) In this Letter, when we speak of discord we mean the relative entropy of discord as defined by us in Eq. 15. When we speak of the original definition discord we will write it as such.
  * Vedral  _et al._ (1997) V. Vedral, M. B. Plenio, K. Jacobs, and P. L. Knight, Phys. Rev. A, 56, 4452 (1997b).


# Stability of Syzygy bundles corresponding to stable vector bundles on
algebraic surfaces

SURATNO BASU<EMAIL_ADDRESS><EMAIL_ADDRESS>Chennai Mathematical
Institute, Siruseri Sipcot IT Park, Kelambakkam, Chennai-603103, Tamilnadu,
India  and  SARBESWAR PAL<EMAIL_ADDRESS><EMAIL_ADDRESS>IISER
Thiruvananthapuram, Maruthamala P. O., Kerala 695551

(Date: 24 Nov, 2020)

###### Abstract.

Let $(X,H)$ be a polarized smooth projective algebraic surface and $E$ is
globally generated, stable vector bundle on $X$. Then the Syzygy bundle
$M_{E}$ associated to it is defined as the kernel bundle corresponding to the
evaluation map. In this article we will study the stability property of
$M_{E}$ with respect to $H$.

###### Key words and phrases:

Syzygy bundles, slope stable

###### 2010 Mathematics Subject Classification:

14J60

##  1\. Introduction

The purpose of this paper is to investigate the stability of Syzygy bundles
associated to a stable and globally generated vector bundles on a smooth
projective algebraic surface.

Let $X$ be a smooth, irreducible, projective algebraic variety defined over an
algebraically closed field $k$. We fix a very ample divisor $H$ on $X$. We
refer to the pair $(X,H)$ as a polarized algebraic variety. Let $E$ be a
globally generated vector bundle on $X$. Then the Syzygy bundle $M_{{}_{E}}$
is defined to be the kernel bundle corresponding to the evaluation map
$\text{ev}:H^{0}(E)\otimes\mathcal{O}_{{}_{X}}\to E\to 0$. Thus we have an
exact sequence

$0\to M_{{}_{E}}\to H^{0}(E)\otimes\mathcal{O}_{{}_{X}}\to E\to 0.$

These vector bundles (and some analogues) arise in a variety of geometric and
algebraic problems. For example what are the numerical conditions (optimal) on
vector bundles $E$ and $F$ on $X$ such that the natural product map

$H^{0}(X,E)\times H^{0}(X,F)\to H^{0}(X,E\otimes F),$

becomes surjective. The first initiative towards this question was taken by
D.C. Butler [2]. He gave an affirmative answer when $X$ is a smooth projective
curve. He approached the question via the vector bundle $M_{E}$ associated to
a bundle $E$\- generated by global sections. Consequently, there has been
considerable interest in trying to establish the stability of $M_{E}$ in
various settings. When $X$ is a smooth curve of genus $g\geqslant 1$, the
semistability was studied by Butler. He proved that when $E$ is semistable
with $\mu(E)\geqslant 2g$, then $M_{E}$ is semistable.

Our main aim in this paper is to study the (slope) stability of $M_{{}_{E}}$
with respect to $H$ when $E$ is stable with respect to $H$ and $X$ is a smooth
projective surface. In [1] the authors study the stability of $M_{{}_{E}}$
when $E$ is a very ample line bundle. In fact they showed that, if we take
sufficiently large power of $E$ then the kernel bundle is (slope) stable with
respect to $E$. In this short note we consider the case of $H$-stable vector
bundles. For any vector bundle $E$ on $X$ and $m>0$ let
$E(m):=E\otimes\mathcal{O}_{{}_{X}}(mH)$. We will prove the following

######  Theorem 1.1.

Let $E$ be a $H$-(slope) stable vector bundle on $X$. There exists $m>>0$ such
that the kernel bundle $M_{{}_{E(m)}}$ is stable with respect to $H$.

We follow the main proof strategy of [1]. Our method is suitable enhancement
of the arguments given in [1]. Two key ingredients of the proof are Mehta-
Ramanathan restriction theorem and Butler’s theorem on stability of kernel
bundles on curves. We note that our theorem as well as the theorem of [1] are
not effective in a sense there is no concrete lower bound of $m$.

##  2\. Proof Of the main theorem

This section is devoted to the proof of the main theorem. First let us fix up
some notations. As before, $X$ is a smooth, irreducible projective surface and
$H$ a very ample divisor. Let $E$ be a stable vector bundle with respect to
$H$ of rank $l$ on $X$. For the entire course of arguments we fix an integer
$n>>0$, sufficiently large, such that $nH-K_{{}_{X}}$ is very ample where
$K_{{}_{X}}$ is the canonical divisor and $H^{i}(X,E(n))=0$, $i=1,2$. For any
closed point $x$ let $m_{{}_{x}}$ denotes the ideal defining the the point
$x$.

We observe the following easy Lemma.

######  Lemma 2.1.

If $W\subset H^{0}(X,E(n))$ is a subspace which generates $E(n)$ then the
natural multiplication map

$H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes m_{{}_{x}}))\otimes W\to
H^{0}(X,E(m)\otimes m_{{}_{x}})$

is surjective for $m>>0$.

###### Proof.

Note that it is enough to prove that the map

$H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)))\otimes W\to H^{0}(X,E(m))$

is surjective for some $m>>0$. By definition of $W$ we have a surjection
$W\otimes\mathcal{O}_{{}_{X}}(-n)\to E$. Let $K$ be the kernel of this
surjection. Then we have the following exact sequence

$0\to K\to W\otimes\mathcal{O}_{{}_{X}}(-n)\to E\to 0$

We choose $m>>0$ such that $H^{1}(X,K(m))=0$. After tensoring the above exact
sequence by $\mathcal{O}_{{}_{X}}(m)$ and passing to the corresponding long
exact sequence we get

$H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)))\otimes W\to H^{0}(X,E(m))$

is surjective.

∎

We now choose a $m>>0$ independent of $n$ such that

(1) $\mathcal{O}_{{}_{X}}(mH)$ and $\mathcal{O}((m-n)H)$ are very ample.

(2) If $W\subset H^{0}(X,E(n))$ is a subspace which generates $E(n)$ then the
natural multiplication map

$H^{0}(X,\mathcal{O}(m-n)H\otimes m_{{}_{x}})\otimes W\to H^{0}(X,E(m)\otimes
m_{{}_{x}})$

is surjective.

We aim to show that there exists a $m>>0$ such that $M_{{}_{E(m)}}$ is
$H$-stable. We first analyze if for some $m>0$, $M_{{}_{E(m)}}$ is not
$H$-stable. In this case there exists a saturated locally free subsheaf
$F_{{}_{m}}\subset M_{{}_{E(m)}}$ such that

$\frac{c_{{}_{1}}(F_{{}_{m}}).H}{rk(F_{{}_{m}})}\geqslant\frac{c_{{}_{1}}(M_{{}_{E(m)}}).H}{rk(M_{{}_{E(m)}})}$

Our goal is to show that for sufficiently large $m>>0$ no such $F_{{}_{m}}$
can exists.

Pick a smooth and irreducible curve $C_{{}_{m}}\in|(m-n)H|$ through a fixed
point $x\in X$. We may also assume that $M_{{}_{E(m)}}/F_{{}_{m}}$ is locally
free along $C_{{}_{m}}$. Observe that

$\mu_{{}_{H}}(F_{{}_{m}})=\frac{c_{{}_{1}}(F_{{}_{m}}).H}{rk(F_{{}_{m}})}=\frac{1}{(m-n)}\mu(F_{{}_{m}}|_{{}_{C_{m}}}),$

Similarly,

$\mu_{{}_{H}}(M_{{}_{E(m)}})=\frac{1}{(m-n)}\mu(M_{{}_{E(m)}}|_{{}_{C_{{}_{m}}}})$

Thus we have

(2.1)
$\mu(F_{{}_{m}}|_{{}_{C_{m}}})\geqslant\mu(M_{{}_{E(m)}}|_{{}_{C_{{}_{m}}}})$

Since, $H^{1}(X,E(n))=0$ we have the following exact sequence

(2.2) $0\to\mathcal{O}_{{}_{C}}^{h^{0}(E(n))}\to
M_{{}_{E(m)}}|_{{}_{C_{{}_{m}}}}\to\overline{M}_{{}_{m}}\to 0$

where $\overline{M}_{{}_{m}}$ is the kernel bundle corresponding to
$E(m)|_{{}_{C_{m}}}$.

As $E$ is stable with respect to $H$, $E(m)$ is stable with respect to $H$.
Now if we choose $m$ so that $(m-n)>>0$ the by Mehta-Ramanathan restriction
theorem ([3, Theorem 7.2.8]) $E(m)|_{{}_{C_{m}}}$ is stable on $C_{{}_{m}}$.
Now we have

$\text{deg}(E(m)|_{{}_{C_{{}_{m}}}})=\text{deg}(E|_{{}_{C_{m}}})+r\text{deg}(\mathcal{O}_{{}_{X}}(m)|_{{}_{C_{m}}})$

Write

$mH=K_{{}_{X}}+mH-nH+Q$

where $Q=nH-K_{{}_{X}}$. By our assumption $Q$ is very ample. Since,
$C_{m}\in|(m-n)H|$ by adjunction formula we get
$K_{{}_{C_{m}}}=(K_{{}_{X}}+(m-n)H)|_{{}_{C_{m}}}$. Thus
$\text{deg}(\mathcal{O}_{{}_{X}}(m)|_{{}_{C_{m}}})=\text{deg}(K_{{}_{C_{m}}})+Q.C_{{}_{m}}$.
As $Q.C_{{}_{m}}\geqslant 3$

(2.3) $\text{deg}(K_{{}_{C_{m}}})+Q.C_{{}_{m}}\geqslant 2g_{{}_{C_{m}}}+1$

We have

$\text{deg}(E(m)|_{{}_{C_{{}_{m}}}})=\text{deg}(E|_{{}_{C_{m}}})+r\text{deg}(\mathcal{O}_{{}_{X}}(m)|_{{}_{C_{m}}})\geqslant\text{deg}(E|_{{}_{C_{m}}})+r(2g_{{}_{C_{m}}}+1)$

We assume that $c_{1}(E)$ is effective then $c_{1}(E).C_{{}_{m}}\geqslant 0$.
Then we have

$\text{deg}(E|_{{}_{C_{m}}})+r(2g_{{}_{C_{m}}}+1)\geqslant
r(2g_{{}_{C_{m}}}+1)$

Therefore, $\text{deg}(E(m)|_{{}_{C_{m}}})\geqslant r(2g_{{}_{C_{m}}}+1)$ and
hence $\mu(E(m)|_{{}_{C_{m}}})\geqslant(2g_{{}_{C_{m}}}+1)$. By Butler’s
theorem ([2, Theorem 1.2]) we have $\overline{M}_{{}_{m}}$ is stable.

Let $K_{{}_{m}}$ be the kernel of $F_{{}_{m}}\hookrightarrow
M_{{}_{E(m)}}|_{{}_{C_{{}_{m}}}}\to\overline{M}_{{}_{m}}$ and $N_{{}_{m}}$ be
the image. As $\overline{M}_{{}_{m}}$ is stable $K_{{}_{m}}\neq 0$. Then we
have the following commutative diagram

(2.4)
${0}$${K_{m}}$${F_{m}}$${N_{{}_{m}}}$${0}$${0}$${\mathcal{O}_{{}_{C_{m}}}^{h^{0}(E(n))}}$${M_{{}_{E(m)}}|_{{}_{C_{{}_{m}}}}}$${\overline{M}_{{}_{m}}}$${0}$

We complete the proof following two crucial lemmas. The proofs of these two
lemmas follow the exact line of arguments of [1, Lemma 1.1, Lemma 1.2] with
some modifications. However, for completeness sake we will provide the proofs.

######  Lemma 2.2.

For any $x\in X$, $\text{rank}(F_{{}_{m}})\geqslant
h^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}})=h^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H))-1$.

###### Proof.

For any vector space $V$ on $X$, $\mathbb{P}_{{}_{\text{sub}}}(V)$ denotes the
projective space associated to $V$ consisting $1$ dimensional linear subspace
of $V$. Multiplication of sections gives rise to a finite morphism

$\mu_{{}_{m}}:\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}})\times\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,E(n)))\to\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,E(m)\otimes
m_{x}))$

which sends

$([s_{1}],[s_{2}])\mapsto s_{1}\otimes s_{2}.$

Note that this morphism is composition of the Segre embedding

$\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}})\times\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,E(n)))\hookrightarrow\mathbb{P}_{{}_{\text{sub}}}(H^{0}((X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}}))\otimes H^{0}(X,E(n)),$

followed by the rational map

$\mathbb{P}_{{}_{\text{sub}}}(H^{0}((X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}}))\otimes
H^{0}(X,E(n))\dashrightarrow\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,E(m)\otimes
m_{x}))$

For any $x\in X$ we have $F_{{}_{m}}(x)\subseteq
M_{{}_{m}}(x)=H^{0}(X,E(m)\otimes m_{{}_{x}})$. Let
$Z:=\mu_{{}_{m}}^{-1}(\mathbb{P}_{{}_{\text{sub}}}(F_{{}_{m}}(x))$. Then
$\mu_{{}_{m}}:Z\to\mathbb{P}_{{}_{\text{sub}}}(F_{{}_{m}}(x)$ is finite
morphism. Therefore,

(2.5) $\text{dim}(Z)\leqslant\text{dim}(F_{{}_{m}}(x))-1$

Note that for any $s\in H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}})$ the map $H^{0}(X,E(n))\to H^{0}(X,E(m)\otimes m_{{}_{x}})$ which
sends $\varphi\mapsto s\otimes\varphi$ is injective. Thus for any
$[s]\in\mathbb{P}_{{}_{\text{sub}}}H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)$
$[s]\times K_{{}_{m}}(x)\subseteq Z$. Therefore,
$\pi_{{}_{1}}:Z\to\mathbb{P}_{{}_{\text{sub}}}H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)$
is dominant where
$\pi_{{}_{1}}:\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)\otimes
m_{{}_{x}})))\times\mathbb{P}_{{}_{\text{sub}}}(H^{0}(X,E(n)))\to\mathbb{P}_{{}_{\text{sub}}}H^{0}(X,\mathcal{O}_{{}_{X}}((m-n)H)$
is the first projection. Consequently, we have

(2.6) $\text{dim}(Z)\geqslant h^{0}(\mathcal{O}((m-n)H\otimes m_{{}_{x}}))-1$

Combining 2.5 and 2.6 we get the Lemma. ∎

As $F_{{}_{m}}\subsetneq M_{E(m)}$ and $\text{rank}(M_{E(m)})=h^{0}(E(m))-l$.
Therefore, $\text{rank}(F_{{}_{m}})<h^{0}(E(m))-l$. Note that by Riemann Roch
formula and the above lemma, we have
$\text{rank}(F_{m})\geqslant\frac{m^{2}}{2}+O(m)$, where $O(m)$ is a linear
function of $m$. Thus we have,

(2.7)
$\text{rank}(F_{{}_{m}})=f(m),~{}where~{}f(m)~{}is~{}a~{}function~{}of~{}m~{}such~{}that~{}\frac{f(m)}{m^{2}}=r,~{}\frac{1}{2}\leqslant
r\leqslant\frac{l}{2},~{}as~{}m\to\infty$

######  Lemma 2.3.

$\text{rank}(K_{{}_{m}})\geqslant rh^{0}(E(n))$ for large $m>>0$.

###### Proof.

From the equation 2.4 we get

(2.8)
$\mu(F_{{}_{m}}|_{{}_{C_{m}}})=\frac{\text{deg}(K_{{}_{m}})+\text{deg}(N_{{}_{m}})}{\text{rank}(F_{{}_{m}})}\leqslant\frac{\text{deg}(N_{{}_{m}})}{\text{rank}(F_{{}_{m}})}=\mu(N_{{}_{m}})\frac{\text{rank}(N_{{}_{m}})}{\text{rank}(F_{{}_{m}})}.$

Since $\overline{M_{{}_{m}}}$ is stable we get

(2.9)
$\mu(N_{{}_{m}})\frac{\text{rank}(N_{{}_{m}})}{\text{rank}(F_{{}_{m}})}<\mu(\overline{M_{{}_{m}}})\frac{\text{rank}(N_{{}_{m}})}{\text{rank}(F_{{}_{m}})}=\mu(\overline{M_{{}_{m}}})(1-\frac{\text{rank}(K_{{}_{m}})}{\text{rank}(F_{{}_{m}})}).$

Now from 2.4 we have
$\text{deg}(M_{{}_{m}}|_{{}_{C_{{}_{m}}}})=\text{deg}(\overline{M_{{}_{m}}})$,
and since

$\mu(M_{{}_{m}}|_{{}_{C_{m}}})\leqslant\mu(F_{{}_{m}}|_{{}_{C_{m}}})$

from equation 2.9 we get

(2.10)
$\frac{\text{deg}(M_{{}_{m}}|_{{}_{C_{m}}})}{h^{0}(E(n))+\text{rank}(\overline{M_{{}_{m}}})}<\frac{\text{deg}(M_{{}_{m}}|_{{}_{C_{m}}})}{\text{rank}(\overline{M_{{}_{m}}})}(1-\frac{\text{rank}(K_{{}_{m}})}{\text{rank}(F_{{}_{m}})}).$

Noting the fact $\text{deg}(M_{{}_{m}}|_{{}_{C_{m}}})<0$ we get

(2.11)
$\frac{1}{h^{0}(E(n))+\text{rank}(\overline{M_{{}_{m}}})}>\frac{1}{\text{rank}(\overline{M_{{}_{m}}})}(1-\frac{\text{rank}(K_{{}_{m}})}{\text{rank}(F_{{}_{m}})})$

i.e.,

$\frac{\text{rank}(\overline{M_{{}_{m}}})}{h^{0}(E(n))+\text{rank}(\overline{M_{{}_{m}}})}>(1-\frac{\text{rank}(K_{{}_{m}})}{\text{rank}(F_{{}_{m}})})$

Thus

$\frac{\text{rank}(K_{{}_{m}})}{\text{rank}(F_{{}_{m}})}>1-\frac{\text{rank}(\overline{M_{{}_{m}}})}{h^{0}(E(n))+\text{rank}(\overline{M_{{}_{m}}})}=\frac{h^{0}(E(n))}{h^{0}(E(n))+\text{rank}(\overline{M_{{}_{m}}})}$

Therefore,

$\text{rank}(K_{{}_{m}})>h^{0}(E(n))\frac{\text{rank}(F_{{}_{m}})}{\text{rank}(M_{{}_{m}})}$

By Lemma 2.2 and since $\text{rank}(M_{{}_{m}})=h^{0}(E(m))-l$ we have

$\text{rank}(K_{{}_{m}})>h^{0}(E(n))\frac{rm^{2}+q(m)}{h^{0}(E(m))-l}$

We have $\frac{rm^{2}+q(m)}{h^{0}(E(m))-l}=r-\epsilon(m)$ where
$\epsilon(m)\to 0$ as $m\to\infty$. Thus for large $m>>0$ we get the
inequality

$\text{rank}(K_{{}_{m}})\geqslant rh^{0}(E(n)).$

∎

Now we will proceed with the proof of Theorem $(1.1)$: Considering the fibres
of the various vector bundles appearing in the commutative diagram 2.4 we get

(2.12)
${K_{{}_{m}}(x)}$${F_{{}_{m}}(x)}$${H^{0}(E(n))}$${(M_{{}_{m}}|_{C_{m}})(x)=H^{0}(E(m)\otimes
m_{x})}$

Let $G$ be the subsheaf of $E(n)$ generated by $K_{{}_{m}}(x)$. We claim that
$\text{rank}(G)>1$. Note that as $G$ is generated by $K_{{}_{m}}(x)$ we have
$K_{{}_{m}}(x)\subseteq H^{0}(X,G)$ in other words

(2.13) $\text{rank}(K_{{}_{m}}(x))\leqslant h^{0}(G).$

As $E$ is rank $l$ slope stable vector bundle with respect to $H$, for any
proper subsheaf $F\subset E$ i.e., $0<\text{rank}(F)<l$, we have , for $d>>0$,

$\frac{\chi(F(d))}{rk(F(d))}<\frac{\chi(E(d))}{l}.$

where $\chi$ is the Euler characteristic of the respective sheaves.
Considering $F:=G(-n)$ we get, from the above equation,

$\frac{\chi(G)}{rk(G)}<\frac{\chi(E(n))}{l}$

Now we choose $n$, sufficiently large, such that $\chi(G)=h^{0}(G)$ and
$\chi(E(n))=h^{0}(E(n))$. Therefore, we have

(2.14) $\frac{h^{0}(G)}{rk(G)}<\frac{h^{0}(E(n))}{l}.$

Combining equation 2.13 and 2.14 we get
$\frac{\text{rank}(K_{{}_{m}}(x))}{rk(G)}<\frac{h^{0}(E(n))}{l}$. On the other
hand by Lemma 2.3
$\frac{rh^{0}(E(n))}{rk(G)}\leqslant\frac{\text{rank}(K_{{}_{m}}(x))}{rk(G)}$
where $\frac{1}{2}\leqslant r\leqslant\frac{l}{2}$. This immediately implies
that $rk(G)>lr\geqslant 1$ since $r\geqslant\frac{1}{2}$ and $l\geqslant 2$.
We have $h^{0}(G(m-n)\otimes
m_{{}_{x}})=h^{0}(G(m-n))-rk(G)=\frac{rk(G)}{2}m^{2}+p(m)$ where
$p\in\mathbb{Q}[n][m]$ and linear in $m$. Let $t=\frac{rk(G)}{2}$ then
$t\geqslant 1$. Now we consider two cases separately.
Case I : $r<1$.
As $t\geqslant 1$ by equation 2.7 for large $m>>0$

(2.15) $\text{rank}(F_{{}_{m}})<h^{0}(G(m-n)\otimes m_{{}_{x}}).$

For any $x\in X$ from the inclusion $G\hookrightarrow E(n)$ we obtain
$H^{0}(G((m-n)\otimes m_{{}_{x}})\hookrightarrow H^{0}(E(m)\otimes
m_{{}_{x}})$ and by 2.15 we get
$\text{Im}(F_{{}_{m}}(x))\neq\text{Im}(H^{0}(G((m-n)\otimes m_{{}_{x}}))$
inside $H^{0}(E(m)\otimes m_{{}_{x}})$. Let
$B_{{}_{m}}(x)=\text{Im}(F_{{}_{m}}(x))\cap\text{Im}(H^{0}(G((m-n)\otimes
m_{{}_{x}}))$. Then $B_{{}_{m}}(x)\subsetneq H^{0}(G((m-n)\otimes
m_{{}_{x}}))$ For any section $s\in H^{0}(\mathcal{O}_{{}_{X}}((m-n))\otimes
m_{{}_{x}})$ multiplication by $s$ maps

$H^{0}(G)\to H^{0}(G(m-n)\otimes m_{{}_{x}})$

and

$K_{{}_{m}}(x)\to B_{{}_{m}}(x).$

Thus from the commutative diagram 2.12 we get

(2.16) ${K_{{}_{m}}(x)}$${B_{{}_{m}}(x)}$${H^{0}(G)}$${H^{0}(G(m-n)\otimes
m_{x})}$

Since $K_{{}_{m}}(x)$ generates $G$ by $(2)$ we conclude that

$H^{0}(\mathcal{O}_{{}_{X}}((m-n))\otimes m_{{}_{x}})\otimes K_{{}_{m}}(x)\to
H^{0}(G(m-n)\otimes m_{{}_{x}})$

is surjective. This leads to a contradiction as we vary sections over an open
set of $H^{0}(\mathcal{O}((m-n)H))$ the images of $K_{{}_{m}}(x)$ span the
whole vector space $H^{0}(G(m-n)\otimes m_{{}_{x}})$. But from the above
commutative diagram every image lies on the proper fixed subspace
$B_{{}_{m}}(x)$. Therefore, $E(m)$ is $H$-stable.
Case II: $r\geqslant 1$.
In this case by Lemma 2.3 rank of $K_{m}=h^{0}(E(n))$. In other words,
$G=E(n)$. Thus the Theorem follows by similar arguments as in Case I.

## References

  * [1] Lawrence Ein; Lazarsfield R; and Mustopa Y. Stability of syzygy bundles on an algebraic surface. Mathematical Research Letters, Vol-20 (2013) no.1, 73-80.
  * [2] Butler D.C. Normal generation of vector bundles over a curve, Journal of Differential Geometry, 39(1994); no.1, 1-34.
  * [3] Huybrechts, D; Lehn, M. The geometry of moduli spaces of sheaves, Cambridge University Press, 2010.


# Multimodal Machine Translation with Embedding Prediction

Tosho Hirasawa  Hayahide Yamagishi  Yukio Matsumura  Mamoru Komachi
Tokyo Metropolitan University
<EMAIL_ADDRESS><EMAIL_ADDRESS>
<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

Multimodal machine translation is an attractive application of neural machine
translation (NMT). It helps computers to deeply understand visual objects and
their relations with natural languages. However, multimodal NMT systems suffer
from a shortage of available training data, resulting in poor performance for
translating rare words. In NMT, pretrained word embeddings have been shown to
improve NMT of low-resource domains, and a search-based approach is proposed
to address the rare word problem. In this study, we effectively combine these
two approaches in the context of multimodal NMT and explore how we can take
full advantage of pretrained word embeddings to better translate rare words.
We report overall performance improvements of 1.24 METEOR and 2.49 BLEU and
achieve an improvement of 7.67 F-score for rare word translation.

##  1 Introduction

In multimodal machine translation, a target sentence is translated from a
source sentence together with related nonlinguistic information such as visual
information. Recently, neural machine translation (NMT) has superseded
traditional statistical machine translation owing to the introduction of the
attentional encoder-decoder model, in which machine translation is treated as
a sequence-to-sequence learning problem and is trained to pay attention to the
source sentence while decoding Bahdanau et al. (2015).

Most previous studies on multimodal machine translation are classified into
two categories: visual feature adaptation and data augmentation. In visual
feature adaptation, multitask learning Elliott and Kádár (2017) and feature
integration architecture Caglayan et al. (2017a); Calixto et al. (2017) are
proposed to improve neural network models. Data augmentation aims to deal with
the fact that the size of available datasets for multimodal translation is
quite small. To alleviate this problem, parallel corpora without a visual
source Elliott and Kádár (2017); Grönroos et al. (2018) and pseudo-parallel
corpora obtained using back-translation Helcl et al. (2018) are used as
additional learning resources.

Due to the availability of parallel corpora for NMT, Qi et al. (2018)
suggested that initializing the encoder with pretrained word embedding
improves the translation performance in low-resource language pairs. Recently,
Kumar and Tsvetkov (2019) proposed an NMT model that predicts the embedding of
output words and searches for the output word instead of calculating the
probability using the softmax function. This model performed as well as
conventional NMT, and it significantly improved the translation accuracy for
rare words.

In this study, we introduce an NMT model with embedding prediction for
multimodal machine translation that fully uses pretrained embeddings to
improve the translation accuracy for rare words.

The main contributions of this study are as follows:

  1. 1.

We propose a novel multimodal machine translation model with embedding
prediction and explore various settings to take full advantage of word
embeddings.

  2. 2.

We show that pretrained word embeddings improve the model performance,
especially when translating rare words.

##  2 Multimodal Machine Translation with Embedding Prediction

We integrate an embedding prediction framework Kumar and Tsvetkov (2019) with
the multimodal machine translation model and take advantage of pretrained word
embeddings. To highlight the effect of pretrained word embeddings and
embedding prediction architecture, we adopt IMAGINATION Elliott and Kádár
(2017) as a simple multimodal baseline.

IMAGINATION jointly learns machine translation and visual latent space models.
It is based on a conventional NMT model for a machine translation task. In
latent space learning, a source sentence and the paired image are mapped
closely in the latent space. We use the latent space learning model as it is,
except for the preprocessing of images. The models for each task share the
same textual encoder in a multitask scenario.

The loss function for multitask learning is the linear interpolation of loss
functions for each task.

$J=\lambda
J_{\mathrm{T}}(\theta,\phi_{\mathrm{T}})+(1-\lambda)J_{\mathrm{V}}(\theta,\phi_{\mathrm{V}})$
(1)

where $\theta$ is the parameter of the shared encoder; $\phi_{\mathrm{T}}$ and
$\phi_{\mathrm{V}}$ are parameters of the machine translation model and latent
space model, respectively; and $\lambda$ is the interpolation coefficient111We
use $\lambda=0.01$ in the experiment..

###  2.1 Neural Machine Translation with Embedding Prediction

The machine translation part in our proposed model is an extension of Bahdanau
et al. (2015). However, instead of using the probability of each word in the
decoder, it searches for output words based on their similarity with word
embeddings. Once the model predicts a word embedding, its nearest neighbor in
the pretrained word embeddings is selected as the system output.

$\displaystyle\bm{\hat{e}_{j}}$ $\displaystyle=$
$\displaystyle\tanh(\bm{W_{\mathrm{o}}}\bm{s_{j}}+\bm{b_{\mathrm{o}}})$ (2)
$\displaystyle\hat{y}_{j}$ $\displaystyle=$ $\displaystyle\mathop{\rm
argmin}\limits_{w\in\mathcal{V}}\\{d(\bm{\hat{e}_{j}},\bm{\mathrm{e}}(w))\\}$
(3)

where $\bm{s_{j}}$, $\bm{\hat{e}_{j}}$, and $\hat{y}_{j}$ are the hidden state
of the decoder, predicted embedding, and system output, respectively, for each
timestep $j$ in the decoding process. $\bm{\mathrm{e}}(w)$ is the pretrained
word embedding for a target word $w$. $d$ is a distance function that is used
to calculate the word similarity. $\bm{W_{\mathrm{o}}}$ and
$\bm{b_{\mathrm{o}}}$ are parameters of the output layer.

We adopt margin-based ranking loss Lazaridou et al. (2015) as the loss
function of the machine translation model.

$\begin{split}J_{\mathrm{T}}(\theta,\phi_{\mathrm{T}})=\sum_{j}^{M}\max\\{0,\gamma+d(\bm{\hat{e}_{j}},\bm{\mathrm{e}}(w_{j}^{-}))\\\
\quad-d(\bm{\hat{e}_{j}},\bm{\mathrm{e}}(y_{j}))\\}\end{split}$ (4)
$w_{j}^{-}=\mathop{\rm
argmax}\limits_{w\in\mathcal{V}}\\{d(\bm{\hat{e}_{j}},\bm{\mathrm{e}}(w))-d(\bm{\hat{e}_{j}},\bm{\mathrm{e}}(y_{j}))$
(5)

where $M$ is the length of a target sentence and $\gamma$ is the margin222We
use $\gamma=0.5$ in the experiment.. $w_{j}^{-}$ is a negative sample that is
close to the predicted embedding and far from the gold embedding as measuring
using $d$.

Pretrained word embeddings are also used to initialize the embedding layers of
the encoder and decoder, and the output layer of the decoder. The embedding
layer of the encoder is updated during training, and the embedding layer of
the decoder is fixed to the initial value.

###  2.2 Visual Latent Space Learning

The decoder of this model calculates the average vector over the hidden states
$\bm{h_{i}}$ in the encoder and maps it to the final vector $\bm{\hat{v}}$ in
the latent space.

$\bm{\hat{v}}=\tanh(\bm{W_{\mathrm{v}}}\cdot\frac{1}{N}\sum_{i}^{N}\bm{h_{i}})$
(6)

where $N$ is the length of an input sentence and
$\bm{W_{\mathrm{v}}}\in\mathbb{R}^{N*M}$ is learned parameter of the model.

We use max margin loss as the loss function; it learns to make corresponding
latent vectors of a source sentence and the paired image closer.

$J_{\mathrm{V}}(\theta,\phi_{\mathrm{V}})=\sum_{\bm{v^{\prime}}\neq\bm{v}}\max\\{0,\alpha+d(\bm{\hat{v}},\bm{v^{\prime}})-d(\bm{\hat{v}},\bm{v})\\}$
(7)

where $\bm{v}$ is the latent vector of the paired image; $\bm{v^{\prime}}$,
the image vector for other examples; and $\alpha$, the margin that adjusts the
sparseness of each vector in the latent space333We use $\alpha=0.1$ in our
experiment..

##  3 Experiment

###  3.1 Dataset

We train, validate, and test our model with the Multi30k Elliott et al. (2016)
dataset published in the WMT17 Shared Task.

We choose French as the source language and English as the target one. The
vocabulary size of both the source and the target languages is 10,000.
Following Kumar and Tsvetkov (2019), byte pair encoding Sennrich et al. (2016)
is not applied. The source and target sentences are preprocessed with lower-
casing, tokenizing and normalizing the punctuation.

Visual features are extracted using pretrained ResNet He et al. (2016).
Specifically, we encode all images in Multi30k with ResNet-50 and pick out the
hidden state in the pool5 layer as a 2,048-dimension visual feature. We
calculate the centroid of visual features in the training dataset as the bias
vector and subtract the bias vector from all visual features in the training,
validation and test datasets.

###  3.2 Model

The model is implemented using nmtpytorch toolkit
v3.0.0444https://github.com/toshohirasawa/nmtpytorch-emb-pred Caglayan et al.
(2017b).

The shared encoder has 256 hidden dimensions, and therefore the bidirectional
GRU has 512 dimensions. The decoder in NMT model has 256 hidden dimension. The
input word embedding size and output vector size is 300 each. The latent space
vector size is 2,048.

We used the Adam optimizer with learning rate of 0.0004. The gradient norm is
clipped to 1.0. The dropout rate is 0.3.

BLEU Papineni et al. (2002) and METEOR Denkowski and Lavie (2014) are used as
performance metrics. We also evaluated the models using the F-score of each
word; this shows how accurately each word is translated into target sentences,
as was proposed in Kumar and Tsvetkov (2019). The F-score is calculated as the
harmonic mean of the precision (fraction of produced sentences with a word
that is in the references sentences) and the recall (fraction of reference
sentences with a word that is in model outputs). We ran the experiment three
times with different random seeds and obtained the mean and variance for each
model.

To clarify the effect of pretrained embeddings on machine translation, we also
initialized the encoder and decoder of our models with random values instead
of pretrained embeddings, and investigated the effect of fixing decoder
embeddings.

###  3.3 Word Embedding

We use publicly available pretrained FastText Bojanowski et al. (2017)
embeddings Grave et al. (2018). These word embeddings are trained on Wikipedia
and Common Crawl using the CBOW algorithm, and the dimension is 300.

The embedding for unknown words is calculated as the average embedding over
words that are a part of pretrained embeddings but are not included in the
vocabularies. Both the target and the source embeddings are preprocessed
according to Mu and Viswanath (2018), in which all embeddings are debiased to
make the average embedding into a zero vector and the top five principal
components are subtracted for each embedding.

##  4 Results

Table 1 shows the overall performance of the proposed and baseline models.
Compared with randomly initialized models, our model outperforms the text-only
baseline by +2.49 BLEU and +1.24 METEOR, and the multimodal baseline by +2.31
BLEU and +1.09 METEOR, respectively. While pretrained embeddings improve
NMT/IMAGINATION models as well, the improved models are still beyond our
model.

| val | test
---|---|---
Model | BLEU | BLEU | METEOR
NMT | 50.83 | 51.00$\pm$.37 | 42.65$\pm$.12
$+$ pretrained | 52.05 | 52.33$\pm$.66 | 43.42$\pm$.13
IMAG+ | 51.03 | 51.18$\pm$.16 | 42.80$\pm$.19
$+$ pretrained | 52.40 | 52.75$\pm$.25 | 43.56$\pm$.04
Ours | 53.14 | 53.49$\pm$.20 | 43.89$\pm$.14
Table 1:  Results on Multi30k validation and test dataset. NMT denotes the
text-only conventional NMT model Bahdanau et al. (2015) and IMAG+ denotes our
reimplementation of the IMAGINATION Elliott and Kádár (2017) model. “+
pretrained” models are initialized with pretrained embeddings.

Table 2 shows the results of ablation experiments of the initialization and
fine-tuning methods. The pretrained embedding models outperform other models
by up to +2.77 BLEU and +1.37 METEOR.

Encoder | Decoder | Fixed | BLEU | METEOR
---|---|---|---|---
fasttext | fasttext | Yes | 53.49 | 43.89
random | fasttext | Yes | 53.22 | 43.83
fasttext | random | No | 51.53 | 43.07
random | random | No | 51.42 | 42.77
fasttext | fasttext | No | 51.42 | 42.88
random | fasttext | No | 50.72 | 42.52
Table 2:  Results on test dataset with variations of model initialization and
fine-tuning in decoder.

##  5 Discussion

#### Rare Words

Our model shows a great improvement for low-frequency words. Figure 1 shows a
variety of F-score according to the word frequency in the training corpus.
Whereas IMAGINATION improves the translation accuracy uniformly, our model
shows substantial improvement for rare words.

Figure 1:  F-score of word prediction per frequency breakdown in training
corpus.

#### Word Embeddings

Furthermore, we found that decoder embeddings must be fixed to improve
multimodal machine translation with embedding prediction. When we allow fine-
tuning on the embedding layer, the performance drops below the baseline. It
seems that fine-tuning embeddings in NMT with embedding prediction makes the
model search for common words more than expected, thus preventing it from
predicting rare words.

More interestingly, using pretrained FastText embeddings on the decoder rather
than the encoder improves performance. This finding is different from Qi et
al. (2018), in which only the encoder benefits from pretrained embeddings.
Compared with the model initialized with a random value, initializing the
decoder with the embedding results in an increase of +1.80 BLEU; in contrast,
initializing the encoder results in an increase of only +0.11 BLEU. This is
caused by the multitask learning model that trains the encoder with images and
takes it away from what the embedding prediction model wants to learn from the
sentences.

#### Visual Feature

| val | test
---|---|---
Model | BLEU | BLEU | METEOR
Ours | 53.14 | 53.49 | 43.89
$-$ Debias | 52.65 | 53.27 | 43.91
$-$ Images | 52.97 | 53.25 | 43.91
Table 3:  Ablation experiments of visual features. “$-$ Debias” denotes the
result without subtracting the bias vector. “$-$ Images” shows the result of
text-only NMT with embedding prediction.

We also investigated the effect of images and its preprocessing in NMT with
embedding prediction (Table 3). The interesting result is that multitask
learning with raw images would not help the predictive model. Debiasing images
is an essential preprocessing for NMT with embedding prediction to use images
effectively in multitask learning scenario.

#### Translation Examples

Image |  |
---|---|---
Source |  un homme en vélo pédale devant une voûte . |  quatre hommes , dont trois portent des kippas , sont assis sur un tapis à motifs bleu et vert olive .
Reference |  a man on a bicycle pedals through an archway . |  four men , three of whom are wearing prayer caps , are sitting on a blue and olive green patterned mat .
NMT |  a man on a bicycle pedal past an arch . |  four men , three of whom are wearing aprons , are sitting on a blue and green speedo carpet .
IMAG+ |  a man on a bicycle pedals outside a monument . |  four men , three of them are wearing alaska , are sitting on a blue patterned carpet and green green seating .
Ours |  a man on a bicycle pedals in front of a archway . |  four men , three are wearing these are wearing these are sitting on a blue and green patterned mat .
Table 4:  French to English translation examples in the Multi30k test set.

In Table 4, we show French-English translations generated by different models.
In the left example, our proposed model correctly translates “voûte” into
“archway” (occurs five times in the training set), Although the baseline model
translates it to its synonym having higher frequency (nine times for “arch”
and 12 times for “monument”). At the same time, our outputs tend to be less
fluent for long sentences. The right example shows that our model translates
some words (“patterned” and “carpet”) more concisely; however, it generates a
less fluent sentence than the baseline.

##  6 Related Works

Most studies on multimodal machine translation are divided into two
categories: visual feature adaptation and data augmentation.

First, in visual feature adaptation, visual features are extracted using image
processing techniques and then integrated into a machine translation model. In
contrast, most multitask learning models use latent space learning as their
auxiliary task. Elliott and Kádár (2017) proposed the IMAGINATION model that
learns to construct the corresponding visual feature from the textual hidden
states of a source sentence. The visual model shares its encoder with the
machine translation model; this helps in improving the textual encoder.

Second, in data augmentation, parallel corpora without images are widely used
as additional training data. Grönroos et al. (2018) trained their multimodal
model with parallel corpora and achieved state-of-the-art performance in the
WMT 2018. However, the use of monolingual corpora has seldom been studied in
multimodal machine translation. Our study proposes using word embeddings that
are pretrained on monolingual corpora.

##  7 Conclusion

We have proposed a multimodal machine translation model with embedding
prediction and showed that pretrained word embeddings improve the performance
in multimodal translation tasks, especially when translating rare words.

In the future, we will tailor the training corpora for embedding learning,
especially for handling the embedding for unknown words in the context of
multimodal machine translation. We will also incorporate visual features into
contextualized word embeddings.

## References

  * Bahdanau et al. (2015) Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015.  Neural machine translation by jointly learning to align and translate.  In _ICLR_.
  * Bojanowski et al. (2017) Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017.  Enriching word vectors with subword information.  In _TACL_ , volume 5, pages 135–146.
  * Caglayan et al. (2017a) Ozan Caglayan, Walid Aransa, Adrien Bardet, Mercedes García-Martínez, Fethi Bougares, Loïc Barrault, Marc Masana, Luis Herranz, and Joost van de Weijer. 2017a.  LIUM-CVC submissions for WMT17 multimodal translation task.  In _WMT_ , pages 432–439.
  * Caglayan et al. (2017b) Ozan Caglayan, Mercedes García-Martínez, Adrien Bardet, Walid Aransa, Fethi Bougares, and Loïc Barrault. 2017b.  NMTPY: A flexible toolkit for advanced neural machine translation systems.  _Prague Bull. Math. Linguistics_ , 109:15–28.
  * Calixto et al. (2017) Iacer Calixto, Qun Liu, and Nick Campbell. 2017.  Doubly-attentive decoder for multi-modal neural machine translation.  In _ACL_ , pages 1913–1924.
  * Denkowski and Lavie (2014) Michael Denkowski and Alon Lavie. 2014.  Meteor universal: Language specific translation evaluation for any target language.  In _WMT_ , pages 376–380.
  * Elliott et al. (2016) Desmond Elliott, Stella Frank, Khalil Sima’an, and Lucia Specia. 2016.  Multi30k: Multilingual English-German image descriptions.  In _Proceedings of the 5th Workshop on Vision and Language_ , pages 70–74.
  * Elliott and Kádár (2017) Desmond Elliott and Àkos Kádár. 2017.  Imagination improves multimodal translation.  In _IJCNLP_ , volume 1, pages 130–141.
  * Grave et al. (2018) Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomas Mikolov. 2018.  Learning word vectors for 157 languages.  In _LREC_ , pages 3483–3487.
  * Grönroos et al. (2018) Stig-Arne Grönroos, Benoit Huet, Mikko Kurimo, Jorma Laaksonen, Bernard Merialdo, Phu Pham, Mats Sjöberg, Umut Sulubacak, Jörg Tiedemann, Raphael Troncy, and Raúl Vázquez. 2018.  The MeMAD submission to the WMT18 multimodal translation task.  In _WMT_ , pages 603–611.
  * He et al. (2016) Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016.  Deep residual learning for image recognition.  In _CVPR_ , pages 770–778.
  * Helcl et al. (2018) Jindřich Helcl, Jindřich Libovický, and Dusan Varis. 2018.  CUNI system for the WMT18 multimodal translation task.  In _WMT_ , pages 616–623.
  * Kumar and Tsvetkov (2019) Sachin Kumar and Yulia Tsvetkov. 2019.  Von Mises-Fisher loss for training sequence to sequence models with continuous outputs.  In _ICLR_.
  * Lazaridou et al. (2015) Angeliki Lazaridou, Georgiana Dinu, and Marco Baroni. 2015.  Hubness and pollution: Delving into cross-space mapping for zero-shot learning.  In _ACL-IJCNLP_ , pages 270–280.
  * Mu and Viswanath (2018) Jiaqi Mu and Pramod Viswanath. 2018.  All-but-the-Top: Simple and effective postprocessing for word representations.  In _ICLR_.
  * Papineni et al. (2002) Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002.  BLEU: a method for automatic evaluation of machine translation.  In _ACL_ , pages 311–318.
  * Qi et al. (2018) Ye Qi, Devendra Sachan, Matthieu Felix, Sarguna Padmanabhan, and Graham Neubig. 2018\.  When and why are pre-trained word embeddings useful for neural machine translation?  In _NAACL_ , pages 529–535.
  * Sennrich et al. (2016) Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016.  Neural machine translation of rare words with subword units.  In _ACL_ , pages 1715–1725.