# Normal-mode driven exploration of
protein domain motions

Yves-Henri<EMAIL_ADDRESS>

UFIP, UMR 6286 of CNRS,
Faculté des Sciences et des Techniques, Nantes, France

(March 22th, 2021)

## Abstract

Domain motions involved in the function of proteins can often be well
described as a combination of motions along a handfull of low-frequency modes,
that is, with the values of a few normal coordinates. This means that, when
the functional motion of a protein is unknown, it should prove possible to
predict it, since it amounts to guess a few values.

However, without the help of additional experimental data, using normal
coordinates for generating accurate conformers far away from the initial one
is not so straightforward. To do so, a new approach is proposed: instead of
building conformers directly with the values of a subset of normal
coordinates, they are built in two steps, the conformer built with normal
coordinates being just used for defining a set of distance constraints, the
final conformer being built so as to match them. Note that this approach
amounts to transform the problem of generating accurate protein conformers
using normal coordinates into a better known one: the distance-geometry
problem, which is herein solved with the help of the ROSETTA software.

In the present study, this approach allowed to rebuild accurately six large
amplitude conformational changes, using at most six low-frequency normal
coordinates. As a consequence of the low-dimensionality of the corresponding
subspace, random exploration also proved enough for generating low-energy
conformers close to the known end-point of the conformational change of the
LAO binding protein, lysozyme T4 and adenylate kinase.

Keywords: Conformers, Normal modes, Distance constraints, Elastic Network
Model, ROSETTA.

## Introduction

It has been known for a while that domain motions involved in the function of
a protein can be well described as a combination of motions along its lowest-
frequency normal modes [1, 2, 3, 4], and this even when the normal modes are
obtained using simplified protein models [1, 5, 6, 7, 8, 9], like the elastic
network model (ENM) [10, 11, 12].

These results suggest that it should prove possible to predict the functional
motion of a protein, since it amounts to guess the values of a few normal
coordinates. Indeed, relevant conformers can be obtained by optimizing these
values so as to match low-resolution X-ray crystallographic [13, 14], CryoEM
[15, 16, 17] or SAXS [18, 19] data, to solve molecular replacement problems
[17, 20] or to fulfill a set of experimentally determined distance constraints
[21].

On the other hand, when additional experimental data are missing, normal mode
coordinates can be used in order to explore the conformational landscape of a
protein in the vicinity of a known structure, for instance through single
mode-following techniques [22, 23, 24, 25]. They can also be used so as to
take advantage of the low-dimensionality of the subspace to be explored, in
conjunction with various sampling methods [26], such as random exploration
[27], energy minimization [28, 29, 30, 31], Monte Carlo Metropolis [32, 33],
molecular dynamics [34, 35, 36], path planning protocols [37] or geometric
simulations [38].

In principle, they could also be used to perform straight jumps, that is, to
go from a conformer to another one in a single step, noteworthy in the case of
pairs of conformers with a Cα-root-mean-square difference (RMSd) of several
Ångströms.

Indeed, it was shown in a previous study that, with the help of algorithms
implemented in the ROSETTA software [39], it is possible to perform such
jumps. Noteworthy, in the case of 14 large amplitude (RMSd $>$ 2 Å)
conformational changes, it proved possible to obtain conformers less than 1 Å
away from the target, that is, the known end-point of the conformational
change [40]. Though encouraging, such successes proved rare, namely, 15% of
the conformational changes considered, underlining the fact that using normal
coordinates for generating accurate conformers far away from the initial one
is not a straightforward task [41, 42, 43].

In the present study, a promising solution is proposed. Instead of building
conformers using directly a set of normal coordinates, they are built in two
steps, the conformer built with normal coordinates being just used for
defining a large set of distance constraints, the final conformer being built
so as to match them. Note that this approach amounts to transform the problem
of generating accurate protein conformers using normal coordinates into a
better known one: the distance-geometry problem [44, 45, 46].

## Methods

### Building conformers using normal coordinates

Conformers can be generated by displacing the $N$ atoms of a system along $n$
of its normal modes ($n\leq 3N$), $\Delta r_{i}$, the displacement of
coordinate $i$ being:

$\Delta r_{i}=\frac{1}{\sqrt{m_{i}}}\sum^{n}_{k=1}a_{ik}q_{k}$ (1)

where $q_{k}$ is the kth normal coordinate of the system, that is, the
amplitude of the motion along mode $k$, $a_{ik}$ being the weight of mode $k$
for coordinate $i$, while $m_{i}$ is the mass of the corresponding atom.

In the present study, the elastic energy (eqn 4) of the conformers obtained
using eqn 1 is usually minimized, under the following set of constraints:

$q_{k}=cste,\hskip 28.45274ptk=1\dots n$

In other words, during energy minimization, only the $3N-n$ other coordinates
are allowed to vary, as done in a number of previous studies [23, 24, 40].

In practice, conformer generation was performed with the dq_modes software,
which is freely available on HAL (https://hal.archives-
ouvertes.fr/hal-03144518), via the Software Heritage program [47].

The conformers thus obtained were not considered as is. Instead, they were
used to specify NOE-like restraints between all pairs of Cα atoms of the
protein considered, except for pairs of Cα atoms belonging to consecutive
residues along the sequence, the corresponding Cα-Cα distances being not
expected to vary significantly during a conformational change.

Conformers consistent with this set of distance restraints were then
determined with the ROSETTA software [39], version 2018.12, using the RELAX
protocol, a 1 Å distance between the lower and upper NOE boundaries being
assumed in order to take into account the lack of precision of the distances
obtained above. Finally, all restraints were removed and an energy
minimization was performed, with the parameters of the MINIMIZE protocol set
to their default values.

Note that hereafter, for the sake of comparison between proteins of different
sizes, minimum energies thus obtained are given per residue.

### Rebuilding protein conformational changes

Like any motion, a conformational change of a protein, as for instance known
by comparing structures A and B, can be described as a combination of motions
along the normal modes of structure A, that is, with values of the
corresponding normal coordinates obtained as follows:

$q_{k}=\sum^{3N}_{i=1}a_{ik}\sqrt{m_{i}}\Delta r_{i}$ (2)

$\Delta r_{i}$ being the difference between the values of the ith coordinate
of the protein for structures A and B, structure B being first fitted onto
structure A so as to have $q_{k}=0$ for the six zero-frequency modes
associated to the overall translation and rotation of the protein. Note that,
because the normal modes of a system form a basis set [48, 49], $I_{k}$, the
percentage of involvement of mode $k$ in a given motion can be quantified as
follows [50]:

$I_{k}=100\frac{q_{k}^{2}}{\lVert\bf{\Delta r}\rVert}$ (3)

with $\sum\limits^{3N}I_{k}=$ 100%. Hereafter, it is assumed that mode $k$ is
significantly involved in a conformational change if $I_{k}>5$%.

Reciprocally, using eqn 1 and the values of $n$ normal coordinates obtained
through eqn 2, the conformational change of a protein can be partially
rebuilt, the rebuilding being perfect when $n=3N$.

Hereafter, like in our previous study [40], the rebuilding of protein
conformational changes is performed using their $n$ lowest-frequency modes.
Note however that conformers thus obtained are then optimized with ROSETTA, as
described in the previous section, ten independant optimizations being
performed for each conformer built using eqn 1.

### Elastic network models

Protein normal modes were calculated with a standard Elastic Network Model
[10, 12, 51] (ENM), that is, a model where $V$, the (elastic) energy of the
protein, is given by:

$V=\frac{1}{2}k_{enm}\sum_{d_{ij}^{0}<R_{c}}(d_{ij}-d_{ij}^{0})^{2}$ (4)

where $d_{ij}$ is the actual distance between atoms $i$ and $j$, $d_{ij}^{0}$
being their distance in the initial structure. $R_{c}$, a distance cutoff, is
the only parameter of this ENM. Indeed, when, as assumed herein, $m_{i}=cste$,
$k_{enm}$ is just an unit factor allowing for instance to specify the value of
the lowest (non-zero) normal mode frequency of the system.

Hereafter, two flavors of this kind of ENM are considered, namely, the popular
Cα-ENM [6, 12, 51], which is built using only Cα atoms (with Rc = 12 Å), and
the all-atom ENM noteworthy used by the Elnémo webserver [52] (with Rc = 5 Å),
where atoms belonging to a given residue are assumed to behave like a rigid-
body —the so-called RTB approximation [5, 53, 54, 55].

In practice, normal mode calculations were performed with the enm_modes
software, which is freely available on HAL (https://hal.archives-
ouvertes.fr/hal-03142851), via the Software Heritage program [47].

## Results

Figure 1:  Maximum distance between consecutive Cα atoms along the sequence,
as a function of the number of Cα-ENM modes used to rebuild the conformational
change of adenylate kinase. Plain line: straight rebuilding. Dotted line: when
the elastic energy of the conformer is minimized, keeping the normal
coordinates used for the rebuilding fixed.

### The main bottleneck

The main problem encountered when using only eqn 1 for generating protein
conformers is illustrated in Figure 1, where the maximum distance between
consecutive Cα atoms along the sequence, observed during the rebuilding of the
conformational change of adenylate kinase, as known by comparing the open (PDB
4AKE [56]) and closed (PDB 1AKE [57]) forms, is shown.

With a Cα-ENM of the open form of adenylate kinase, using eqn 1 and the value
of the lowest-frequency normal coordinate (eqn 2) of the closed form, without
any minimization of the elastic energy and no optimization by ROSETTA of the
conformer thus obtained, the maximum distance between consecutive Cα atoms is
5.2 Å, far from standard values, like the average one observed in the case of
the open form, namely, 3.8 $\pm$ 0.1 Å.

With the 102 lowest-frequency normal coordinates of the closed form, the
maximum distance between consecutive Cα atoms of the conformer becomes as high
as 7.2 Å. However, its RMSd from the closed form is as low as 1.2 Å, in line
with previous works showing that the conformational change of adenylate kinase
can be well described as a combination of motions along its low-frequency
modes [6, 5]. So, while the overall conformational change of adenylate kinase
is indeed very well described with the values of 102 normal coordinates, as
illustrated in Figure 1, chemical details look then quite ugly. As a matter of
fact, more than $\approx$ 450 normal coordinates (70 % of them) are needed in
order not to have any distance between consecutive Cα atoms above 5 Å.

On the other hand, Figure 1 shows that minimizing the energy of the elastic
energy of the conformer thus obtained, while keeping the considered normal
coordinates fixed, as done in our previous study [40], does not improve
significantly the chemical accuracy of the conformers, distances between
consecutive Cα atoms around 6 Å being still observed.

In our previous study [40], in order to cope with this issue, all conformers
were further optimized with ROSETTA [39], using harmonic restraints on the
protein backbone. Hereafter, as mentioned in the Methods section, optimized
conformers are obtained so as to match a large set of distance restraints.
Interestingly, with such an approach, the chemical accuracy of the generated
conformers is guaranteed, since it only depends upon the quality of the
initial structure, that is, the structure used for the normal mode analysis.

While working with internal coordinates [58, 59, 60, 61] also secures the
chemical accuracy of the generated conformers [32, 43], note that variations
of a backbone internal coordinate can yield steric clashes far away along the
sequence, a problem which is the counterpart of the problem with chemical
accuracy occurring when working with cartesian coordinates.

### Choice of the kind of ENM

Figure 2:  Cα root-mean-square deviation from adenylate kinase closed form
(PDB 4AKE), as a function of the number of ENM modes used to rebuild the
conformational change of adenylate kinase, starting from the open form (PDB
1AKE). a) Minimum and b) median values obtained after ten ROSETTA
optimizations, using modes obtained with a Cα-ENM, with (open diamonds) or
without (stars) minimization of the elastic energy, keeping the normal
coordinates used for the rebuilding fixed; filled diamonds: using an all-atom
ENM together with elastic energy minimization. Dotted line: corresponds to 1.0
Å.

As shown in Figure 2, in the case of the rebuilding of the conformational
change of adenylate kinase, starting from the open form, using the values of
less than 5–8 low-frequency normal coordinates of the closed form proves
enough for obtaining optimized conformers less than 2 Å away from the target.
However, with the Cα-ENM and no preliminary minimization of the elastic energy
of the conformers, even with 20 low-frequency normal coordinates, no optimized
conformer less than 1.5 Å away is obtained, while otherwise conformers
$\approx$ 1.0 Å away from the closed form are found (Fig. 2a).

Noteworthy, with the all-atom ENM, a median RMSd from the closed form of 1.5 Å
is observed with as little as six low-frequency normal coordinates (Fig. 2b).
Moreover, with this model, the quality of the conformers obtained seems to
depend less upon the exact number of low-frequency normal coordinates taken
into account. This is the main reason why it was chosen for performing the
rest of this work.

### Rebuilding conformational changes

Figure 3:  Minimum Cα root-mean-square deviation from the target, as a
function of the number of all-atom ENM modes used to rebuild the
conformational change of: a) Adenylate kinase (4AKE $\,\to\,$ 1AKE); b) LAO
binding protein (2LAO $\,\to\,$ 1LST); c) Lysozyme T4 (177L $\,\to\,$ 178L);
d) Glutamin binding protein (1GGG $\,\to\,$ 1WDN); e) NF-$\kappa$B (1LEI
$\,\to\,$ 1RAM); f) Maltose binding protein (1OMP $\,\to\,$ 1ANF). Dotted
line: corresponds to 1.0 Å.

Figure 3 shows the results obtained when rebuilding the conformational change
of six proteins, including classic cases like adenylate kinase [5, 6, 31, 62],
lysozyme T4 [63, 64, 65], the lysine/arginine/ornithine (LAO) [6, 60, 64] or
the maltose [6, 10, 60] binding proteins.

The more spectacular result is obtained with the LAO binding protein (Fig.
3b). Indeed, providing the value of a single normal coordinate, namely, the
lowest-frequency one, proves enough for building, starting from the open form
(PDB 2LAO [66]), a conformer less than 1 Å away from the closed form (PDB 1LST
[66]). However, such a success is not surprising since the percentage of
involvement (eqn 3) of the lowest-frequency normal mode of the open form in
the conformational change of the LAO binding protein is very high ($I_{7}=$
86%).

A result almost as convincing is obtained in the case of the glutamin binding
protein (Fig. 3d), since providing the value of the lowest-frequency normal
coordinate allows to build a conformer $\approx$ 1.5 Å away from the closed
form (PDB 1WDN [67]). Note that, in this case, the lowest-frequency mode is
less involved in the conformational change ($I_{7}=$ 76%), with another one
being also significantly, though marginally, involved, namely, the second
lowest-frequency one ($I_{8}=$ 5%).

While three of them are needed in the case of Lysozyme T4 (Fig. 3c), note
that, in all six cases, conformers $\approx$ 1 Å away from the target are
obtained by specifying the values of no more than six low-frequency normal
coordinates.

Figure 4:  Minimum Cα root-mean-square deviation from the LAO binding protein
closed form (PDB 1LST), as a function of the value of the lowest-frequency
normal coordinate used to build the conformers, starting from the open form
(PDB 2LAO). The arrow pinpoints the value observed for the closed form. Dotted
line: corresponds to 1.0 Å.  Table 1: Average and lowest energy of the
conformers found close to the target (in kcal/mole per residue). Conformers
were randomly generated using between one and five low-frequency modes of the
initial ones.

Protein | $\langle$Energy$\rangle^{a}$ | Initial$\,\to\,$Target | RMSd | Modes | Number of | $\langle$Energy$\rangle$ | Lowest
---|---|---|---|---|---|---|---
(Å) | usedb | conformers | energy
LAO bp | -3.27 $\pm$ 0.03 | 2LAO $\,\to\,$ 1LST | 3.4 | 7 | 38c | -3.2 $\pm$ 0.1 | -3.31
Lysozyme T4 | -2.78 $\pm$ 0.05 | 177L $\,\to\,$ 178L | 4.7 | 7–9 | 43d | -2.6 $\pm$ 0.2 | -2.84
Adenylate kin. | -3.02 $\pm$ 0.03 | 4AKE $\,\to\,$ 1AKE | 7.1 | 7–11 | 29e | -2.7 $\pm$ 0.2 | -2.94

a Of the initial conformer, the average being over 100 relaxed conformations.
b Modes 1–6 correspond to overall translation and rotation motions.
c With a RMSd from target $\leq$ 1.0 Å.
d With a RMSd from target $\leq$ 1.5 Å.
e With a RMSd from target $\leq$ 2.0 Å.

In this respect, NF-$\kappa$B is an interesting case. Indeed, providing less
than six values does not allow to build conformers significantly less than 2 Å
away from the target (PDB 1RAM [68]). Adding the sixth value means that the
highest-frequency mode found significantly involved in the conformational
change is taken into account. Doing so, conformers $\approx$ 1 Å away from the
target can be built (Fig. 3e). So, though the involvement of mode 12 may seem
low ($I_{12}$ = 12%), the information brought by this mode proves important
for the accurate rebuilding of the conformational change of NF-$\kappa$B.

### Single mode exploration

If the value of a single normal coordinate is enough for rebuilding the
conformational change of a protein, it is then straightforward, starting from
the initial conformer, to build conformers close to the target when the
structure of the latter is unknown, for instance by randomly picking values
for the relevant normal coordinate.

This is illustrated in Figure 4, where 50 randomly chosen negative values of
the lowest-frequency normal coordinate were considered, conformers being
generated starting from the open form of the LAO binding protein. As above,
for a given value of the normal coordinate, ten ROSETTA optimizations were
performed. Among the 500 conformers thus obtained, 38 (8%) were found less
than 1 Å away from the closed form. Moreover, compared to the energy of the
open form, their average energy is similar, the lowest conformer energy
observed being even lower than the average energy of the open form (see Table
1). Note that the values of the lowest-frequency normal coordinate used for
generating such highly accurate conformers span a rather wide range, namely,
between -245 and -196 mass-weighted Å (Fig. 4).

### Subspace exploration

Random exploration of a subspace with less than ten dimensions can also prove
efficient, as illustrated by the two following examples, where the sampling
was however limited by choosing the right sign for the normal coordinates,
that is, the sign observed for the target, as well as maximum values twice as
large as the value known for the target.

In the case of Lysozyme T4, such random sampling of the values of the three
lowest-frequency normal coordinates of a mutant with a wild-type conformation
(PDB 177L [69]) allowed to obtain 43 conformers (out of 1,000 trials) less
than 1.5 Å away from the target (PDB 178L [69]). Moreover, like in the case of
the LAO binding protein, their average energy is rather low, the lowest
conformer energy observed being also lower than the average energy of the
initial one (see Table 1).

In the case of adenylate kinase, random sampling of the values of the five
lowest-frequency normal coordinates of the open form allowed to obtain 22
conformers (out of 5,000) less than 2.0 Å away from the closed form.

### Exploring with large jumps

Using eqn 1, atomic displacements can be scaled, so as to generate conformers
a given RMSd away from the initial one.

In the case of adenylate kinase, conformers 6–7 Å away from the open form (PDB
4AKE) were thus built, the scaling being performed after a random choice of
the values of the five lowest-frequency normal coordinates. Note that, in this
case, their sign was also chosen randomly.

First, 10,000 conformers were generated using eqn 1, and their elastic energy
was minimized, keeping the five normal coordinates considered fixed. The 71
conformers found less than 3.5 Å away from the closed form were then optimized
with ROSETTA, as done above, allowing to obtain 7 other conformers less than
2.0 Å away from the target.

Though the average energy of the 29 conformers found close to the closed form
of adenylate kinase is also low, at variance with both previous cases, the
lowest conformer energy observed is significantly higher than the average
energy of the open form (see Table 1). This is not due to the energy gap
expected between open and closed forms since, according to ROSETTA, in spite
of the fact that ligands are missing, the closed form has the lowest average
energy, namely, -3.17 $\pm$ 0.04 kcal/mole per residue. It may mean instead
that the sampling performed was not extensive enough (the generated conformer
the closest to the closed form being still 1.6 Å away), or that more than five
normal coordinates should have been taken into account during the conformer
generation process.

Figure 5:  Optimized value of the lowest-frequency normal coordinate of
adenylate kinase, as a function of its value before ROSETTA optimization, for
conformers found less than 2.0 Å away from the closed form, during a random
exploration along the five lowest-frequency modes of the open form. (0,0)
corresponds to the open form. The dotted line, which indicates equal values,
is a guide for the eye.

## The contribution of ROSETTA

As illustrated in Figure 5 for the case of adenylate kinase, probably as a
consequence of the large number of restraints taken into account, the value of
the lowest-frequency normal coordinate does not change much upon optimization
by ROSETTA. This suggests that the approach introduced herein should work with
various optimization methods, noteworthy those developed specifically for
solving the distance geometry problem [44, 45, 46].

## Conclusion

The approach proposed in the present study allowed to rebuild accurately (RMSd
$\approx$ 1 Å) six large amplitude (RMSd $>$ 3 Å) conformational changes,
using the values of at most six low-frequency normal coordinates (Fig. 3).

This result noteworthy means that, in a predictive context, as a consequence
of the low-dimensionality of the corresponding subspace, conformers close to
the target can be obtained in a straightforward way, for instance by random
sampling, as done herein for three classic cases, namely, the conformational
changes of the LAO binding protein (Fig. 4), lysozyme T4 and adenylate kinase
(Table 1).

More efficient algorithms should significantly speed up the conformer
generation process. Note however that it is already a fairly quick one,
generating an optimized conformer taking a few minutes only, on a single
standard CPU.

So, a timely task left is to delineate the field of application of this
approach. In the present study, a handful of proteins with clear-cut large
amplitude domain motions were scrutinized. Proteins experiencing motions more
arbitrary or of a more subtle nature need now to be considered.

## Acknowledgments

This study is a sequel of the work undertaken with Dr Swapnil Mahajan during
the BIP:BIP Projet d’Avenir (grant ANR-10-BINF-03-04).

## References

  * [1] Harrison, R. (1984).  Variational calculation of the normal modes of a large macromolecule: Methods and some initial results.  _Biopolymers_ 23, 2943–2949.
  * [2] Brooks, B.R. & Karplus, M. (1985).  Normal modes for specific motions of macromolecules: application to the hinge-bending mode of lysozyme.  _Proc. Natl. Acad. Sci. USA_ 82, 4995–4999.
  * [3] Marques, O. & Sanejouand, Y.H. (1995).  Hinge-bending motion in citrate synthase arising from normal mode calculations.  _Proteins_ 23, 557–560.
  * [4] Perahia, D. & Mouawad, L. (1995).  Computation of low-frequency normal modes in macromolecules: improvements to the method of diagonalization in a mixed basis and application to hemoglobin.  _Comput. Chem._ 19(3), 241–246.
  * [5] Tama, F., Gadea, F.X., Marques, O. & Sanejouand, Y.H. (2000).  Building-block approach for determining low-frequency normal modes of macromolecules.  _Proteins_ 41(1), 1–7.
  * [6] Tama, F. & Sanejouand, Y.H. (2001).  Conformational change of proteins arising from normal mode calculations.  _Protein Eng., Des. Sel._ 14(1), 1–6.
  * [7] Delarue, M. & Sanejouand, Y.H. (2002).  Simplified normal modes analysis of conformational transitions in DNA-dependant polymerases: the Elastic Network Model.  _J. Mol. Biol._ 320, 1011–1024.
  * [8] Krebs, W.G., Alexandrov, V., Wilson, C.A., Echols, N., Yu, H. & Gerstein, M. (2002).  Normal mode analysis of macromolecular motions in a database framework: Developing mode concentration as a useful classifying statistic.  _Proteins_ 48(4), 682–695.
  * [9] Mahajan, S. & Sanejouand, Y.H. (2015).  On the relationship between low-frequency normal modes and the large-scale conformational changes of proteins.  _Arch. Biochem. Biophys._ 567, 59–65.
  * [10] Tirion, M. (1996).  Low-amplitude elastic motions in proteins from a single-parameter atomic analysis.  _Phys. Rev. Lett._ 77, 1905–1908.
  * [11] Bahar, I., Atilgan, A.R. & Erman, B. (1997).  Direct evaluation of thermal fluctuations in proteins using a single-parameter harmonic potential.  _Fold. Des._ 2, 173–181.
  * [12] Hinsen, K. (1998).  Analysis of domain motions by approximate normal mode calculations.  _Proteins_ 33, 417–429.
  * [13] Tirion, M., ben Avraham, D., Lorenz, M. & Holmes, K. (1995).  Normal modes as refinement parameters for the F-actin model.  _Biophys. J._ 68(1), 5–12.
  * [14] Chen, X., Wang, Q., Ni, F. & Ma, J. (2010).  Structure of the full-length Shaker potassium channel Kv1.2 by normal-mode-based X-ray crystallographic refinement.  _Proc. Natl. Acad. Sci. USA_ 107(25), 11352–11357.
  * [15] Tama, F., Valle, M., Frank, J. & Brooks III, C.L. (2003).  Dynamic reorganization of the functionally active ribosome explored by normal mode analysis and cryo-electron microscopy.  _Proc. Natl. Acad. Sci. USA_ 100(16), 9319–9323.
  * [16] Suhre, K., Navaza, J. & Sanejouand, Y.H. (2006).  Norma: a tool for flexible fitting of high resolution protein structures into low resolution electron microscopy derived density maps.  _Act. Cryst. D_ 62(9), 1098–1100.
  * [17] Delarue, M. & Dumas, P. (2004).  On the use of low-frequency normal modes to enforce collective movements in refining macromolecular structural models.  _Proc. Natl. Acad. Sci. USA_ 101(18), 6957–6962.
  * [18] Gorba, C. & Tama, F. (2010).  Normal mode flexible fitting of high-resolution structures of biological molecules toward SAXS data.  _Bioinformatics and biology insights_ 4, 43.
  * [19] Panjkovich, A. & Svergun, D.I. (2016).  Deciphering conformational transitions of proteins by small angle X-ray scattering and normal mode analysis.  _Physical Chemistry Chem. Phys._ 18(8), 5707–5719.
  * [20] Suhre, K. & Sanejouand, Y.H. (2004).  On the potential of normal mode analysis for solving difficult molecular replacement problems.  _Act. Cryst. D_ 60, 796–799.
  * [21] Zheng, W. & Brooks, B.R. (2005).  Normal mode based prediction of protein conformational changes guided by distance constraints.  _Biophys. J._ 88(5), 3109–3117.
  * [22] Wei, G., Mousseau, N. & Derreumaux, P. (2002).  Exploring the energy landscape of proteins: A characterization of the activation-relaxation technique.  _J. Chem. Phys._ 117(24), 11379–11387.
  * [23] Miloshevsky, G.V. & Jordan, P.C. (2006).  The open state gating mechanism of gramicidin A requires relative opposed monomer rotation and simultaneous lateral displacement.  _Structure_ 14(8), 1241–1249.
  * [24] Floquet, N., Marechal, J.D., Badet-Denisot, M.A., Robert, C.H., Dauchez, M. & Perahia, D. (2006).  Normal mode analysis as a prerequisite for drug design: application to matrix metalloproteinases inhibitors.  _FEBS Lett._ 580(22), 5130–5136.
  * [25] Batista, P.R., Pandey, G., Pascutti, P.G., Bisch, P.M., Perahia, D. & Robert, C.H. (2011).  Free energy profiles along consensus normal modes provide insight into HIV-1 protease flap opening.  _J. Chem. Theory Comput._ 7(8), 2348–2352.
  * [26] Krieger, J.M., Doruker, P., Scott, A.L., Perahia, D. & Bahar, I. (2020).  Towards gaining sight of multiscale events: utilizing network models and normal modes in hybrid methods.  _Curr. Opin. Struct. Biol._ 64, 34–41.
  * [27] Venkatraman, V. & Ritchie, D.W. (2012).  Flexible protein docking refinement using pose-dependent normal mode analysis.  _Proteins: Struct., Funct., Bioinf._ 80(9), 2262–2274.
  * [28] Cavasotto, C.N., Kovacs, J.A. & Abagyan, R.A. (2005).  Representing receptor flexibility in ligand docking through relevant normal modes.  _J. Am. Chem. Soc._ 127(26), 9632–9640.
  * [29] May, A. & Zacharias, M. (2008).  Energy minimization in low-frequency normal modes to efficiently allow for global flexibility during systematic protein–protein docking.  _Proteins: Struct., Funct., Bioinf._ 70(3), 794–809.
  * [30] Moal, I.H. & Bates, P.A. (2010).  Swarmdock and the use of normal modes in protein-protein docking.  _International journal of molecular sciences_ 11, 3623–3648.
  * [31] Kurkcuoglu, Z., Bahar, I. & Doruker, P. (2016).  ClustENM: ENM-based sampling of essential conformational space at full atomic resolution.  _J. Chem. Theory Comput._ .
  * [32] Noguti, T. & Gō, N. (1985).  Efficient monte carlo method for simulation of fluctuating conformations of native proteins.  _Biopolymers: Original Research on Biomolecules_ 24(3), 527–546.
  * [33] Miloshevsky, G.V. & Jordan, P.C. (2007).  Open-state conformation of the KcsA K+ channel: Monte Carlo normal mode following simulations.  _Structure_ 15(12), 1654–1662.
  * [34] Durup, J. (1991).  Protein molecular dynamics constrained to slow modes: theoretical approach based on a hierarchy of local modes with a set of holonomic constraints; the method and its tests on citrate synthase.  _J. Phys. Chem._ 95(4), 1817–1829.
  * [35] Elezgaray, J. & Sanejouand, Y.H. (1998).  Modeling large-scale dynamics of proteins.  _Biopolymers_ 46, 493–501.
  * [36] Costa, M.G., Batista, P.R., Bisch, P.M. & Perahia, D. (2015).  Exploring free energy landscapes of large conformational changes: molecular dynamics with excited normal modes.  _J. Chem. Theory Comput._ 11(6), 2755–2767.
  * [37] Kirillova, S., Cortes, J., Stefaniu, A. & Simeon, T. (2008).  An NMA-guided path planning approach for computing large-amplitude conformational changes in proteins.  _Proteins_ 70(1), 131–143.
  * [38] Jimenez-Roldan, J., Freedman, R., Römer, R. & Wells, S.A. (2012).  Rapid simulation of protein motion: merging flexibility, rigidity and normal mode analyses.  _Phys. Biol._ 9(1), 016008.
  * [39] O’Meara, M.J., Leaver-Fay, A., Tyka, M.D., Stein, A., Houlihan, K., DiMaio, F., Bradley, P., Kortemme, T., Baker, D., Snoeyink, J. _et al._ (2015).  Combined covalent-electrostatic model of hydrogen bonding improves structure prediction with Rosetta.  _J. Chem. Theory Comput._ 11(2), 609–622.
  * [40] Mahajan, S. & Sanejouand, Y.H. (2017).  Jumping between protein conformers using normal modes.  _J. Comput. Chem._ 38, 1622–1630.
  * [41] Yang, Q. & Sharp, K.A. (2009).  Building alternate protein structures using the elastic network model.  _Proteins: Struct., Funct., Bioinf._ 74(3), 682–700.
  * [42] Ahmed, A., Rippmann, F., Barnickel, G. & Gohlke, H. (2011).  A normal mode-based geometric simulation approach for exploring biologically relevant conformational transitions in proteins.  _J. Chem. Inf. Model._ 51(7), 1604–1622.
  * [43] Frezza, E. & Lavery, R. (2019).  Internal coordinate normal mode analysis: A strategy to predict protein conformational transitions.  _J. Phys. Chem. B_ 123(6), 1294–1301.
  * [44] Wagner, G., Braun, W., Havel, T.F., Schaumann, T., Gō, N. & Wüthrich, K. (1987).  Protein structures in solution by nuclear magnetic resonance and distance geometry: the polypeptide fold of the basic pancreatic trypsin inhibitor determined using two different algorithms, DISGEO and DISMAN.  _J. Mol. Biol._ 196(3), 611–639.
  * [45] Moré, J.J. & Wu, Z. (1999).  Distance geometry optimization for protein structures.  _Journal of Global Optimization_ 15(3), 219–234.
  * [46] Malliavin, T.E., Mucherino, A., Lavor, C. & Liberti, L. (2019).  Systematic exploration of protein conformational space using a distance geometry approach.  _J. Chem. Inf. Model._ 59(10), 4486–4503.
  * [47] Di Cosmo, R. & Zacchiroli, S. (2017).  Software heritage: Why and how to preserve software source code.  In _iPRES 2017-14th International Conference on Digital Preservation_ , pages 1–10.
  * [48] Wilson, E., Decius, J. & Cross, P. (1955).  _Molecular Vibrations_.  McGraw-Hill, New York.
  * [49] Goldstein, H. (1956).  _Classical Mechanics_.  Addison-Wesley, Reading, MA.
  * [50] Ma, J. & Karplus, M. (1997).  Ligand-induced conformational changes in ras p21: a normal mode and energy minimization analysis.  _J. Mol. Biol._ 274, 114–131.
  * [51] Doruker, P., Atilgan, A.R. & Bahar, I. (2000).  Dynamics of proteins predicted by molecular dynamics simulations and analytical approaches: Application to $\alpha$-amylase inhibitor.  _Proteins: Struct., Funct., Bioinf._ 40(3), 512–524.
  * [52] Suhre, K. & Sanejouand, Y.H. (2004).  Elnémo: a normal mode server for protein movement analysis and the generation of templates for molecular replacement.  _Nucl. Ac. Res._ 32, W610–W614.
  * [53] Durand, P., Trinquier, G. & Sanejouand, Y.H. (1994).  A new approach for determining low-frequency normal modes in macromolecules.  _Biopolymers_ 34, 759–771.
  * [54] Li, G. & Cui, Q. (2002).  A coarse-grained normal mode approach for macromolecules: an efficient implementation and application to Ca2+-ATPase.  _Biophys. J._ 83(5), 2457–2474.
  * [55] Hoffmann, A. & Grudinin, S. (2017).  Nolb: Nonlinear rigid block normal-mode analysis method.  _J. Chem. Theory Comput._ 13(5), 2123–2134.
  * [56] Müller, C., Schlauderer, G., Reinstein, J. & Schulz, G.E. (1996).  Adenylate kinase motions during catalysis: an energetic counterweight balancing substrate binding.  _Structure_ 4(2), 147–156.
  * [57] Müller, C.W. & Schulz, G.E. (1992).  Structure of the complex between adenylate kinase from Escherichia coli and the inhibitor Ap5A refined at 1.9 Å resolution: A model for a catalytic transition state.  _J. Mol. Biol._ 224(1), 159–177.
  * [58] Noguti, T. & Go, N. (1982).  Collective variable description of small-amplitude conformational fluctuations in a globular protein.  _Nature_ 296, 776–778.
  * [59] Levitt, M., Sander, C. & Stern, P. (1983).  Normal-mode dynamics of a protein: Bovine pancreatic trypsin inhibitor.  _Int. J. Quant. Chem._ 10, 181–199.
  * [60] López-Blanco, J.R., Aliaga, J.I., Quintana-Ortí, E.S. & Chacón, P. (2014).  iMODS: internal coordinates normal mode analysis server.  _Nucleic Acids Res._ 42(W1), W271–W276.
  * [61] Tirion, M.M. & ben Avraham, D. (2015).  Atomic torsional modal analysis for high-resolution proteins.  _Phys. Rev. E_ 91(3), 032712.
  * [62] Maragakis, P. & Karplus, M. (2005).  Large amplitude conformational change in proteins explored with a plastic network model: adenylate kinase.  _J. Mol. Biol._ 352(4), 807–822.
  * [63] De Groot, B., Hayward, S., Van Aalten, D., Amadei, A. & Berendsen, H. (1998).  Domain motions in bacteriophage T4 lysozyme: a comparison between molecular dynamics and crystallographic data.  _Proteins: Struct., Funct., Bioinf._ 31(2), 116–127.
  * [64] Nicolay, S. & Sanejouand, Y.H. (2006).  Functional modes of proteins are among the most robust.  _Phys. Rev. Lett._ 96(7), 078104.
  * [65] Zacharias, M. (2008).  Combining elastic network analysis and molecular dynamics simulations by hamiltonian replica exchange.  _J. Chem. Theory Comput._ 4(3), 477–487.
  * [66] Oh, B., Pandit, J., Kang, C., Nikaido, K., Gokcen, S., Ames, G. & Kim, S. (1993).  Three-dimensional structures of the periplasmic lysine/arginine/ornithine-binding protein with and without a ligand.  _J. Biol. Chem._ 268, 11348–11355.
  * [67] Sun, Y.J., Rose, J., Wang, B.C. & Hsiao, C.D. (1998).  The structure of glutamine-binding protein complexed with glutamine at 1.94 Å resolution: comparisons with other amino acid binding proteins.  _J. Mol. Biol._ 278(1), 219–229.
  * [68] Chen, Y.Q., Ghosh, S. & Ghosh, G. (1998).  A novel DNA recognition mode by the NF-$\kappa$B p65 homodimer.  _Nature structural biology_ 5(1), 67–73.
  * [69] Faber, H.R. & Matthews, B.W. (1990).  A mutant T4 lysozyme displays five different crystal conformations.  _Nature_ 348, 263–266.


# A Large Parallel Corpus of Full-Text Scientific Articles

###### Abstract

The Scielo database is an important source of scientific information in Latin
America, containing articles from several research domains. A striking
characteristic of Scielo is that many of its full-text contents are presented
in more than one language, thus being a potential source of parallel corpora.
In this article, we present the development of a parallel corpus from Scielo
in three languages: English, Portuguese, and Spanish. Sentences were
automatically aligned using the Hunalign algorithm for all language pairs, and
for a subset of trilingual articles also. We demonstrate the capabilities of
our corpus by training a Statistical Machine Translation system (Moses) for
each language pair, which outperformed related works on scientific articles.
Sentence alignment was also manually evaluated, presenting an average of 98.8%
correctly aligned sentences across all languages. Our parallel corpus is
freely available in the TMX format, with complementary information regarding
article metadata.

Keywords: parallel corpus, scientific articles, Scielo

languageresourceLanguage Resources

A Large Parallel Corpus of Full-Text Scientific Articles

Felipe Soares, Viviane Pereira Moreira, Karin Becker
---
Institute of Informatics, Universidade Federal do Rio Grande do Sul
Porto Alegre - RS, Brazil
{felipe.soares, viviane<EMAIL_ADDRESS>

Abstract content

##  1\. Introduction

Cross-language corpora is one of the basis of Statistical Machine Translation
(SMT) systems. The acquisition of quality corpora is not a trivial task, as it
may demand considerable use of expert human curating, especially for parallel
corpora. In that sense, the automated building of parallel corpora from open
resources is of great interest in Natural Language Processing (NLP). Europarl
[Koehn, 2005], is one of the largest parallel corpora availabe, including up
to 21 European languages. Similarly, the United Nations (UN) parallel corpus
[Ziemski et al., 2016] makes use of UN’s official documents and records,
providing aligned sentenced in six languages. Other parallel corpora
initiatives have been reported in distinct domains, such as patents [Utiyama
and Isahara, 2007], movie subtitles [Zhang et al., 2014], and books [Skadins
et al., 2014].

Parallel corpora based on scientific articles can be a valuable language
resource. Several text mining tasks may benefit from the availability of
parallel corpora of scientific articles. The most straightforward example is
cross language plagiarism detection, when an original text is translated in
another language and presented as novel. Other possible applications are
related to article indexing or classification, as well as the development and
extension of Named Entity Recognition (NER) tools for multiple languages. The
latter is of particular interest to the increasingly active biomedical
sciences field, which has already standardized vocabularies and ontologies,
thus favoring NER initiatives.

The construction of parallel corpora of scientific texts has been addressed by
different authors. ?) constructed a parallel corpus of biomedical article
titles from PUBMED in six languages. Based on this corpus, the authors built
SMT systems and achieved higher BLEU scores than Google Translator. With a
different intent, ?) produced a gold-standard annotated parallel corpus for
biomedical concept recognition in five languages. They used Medline abstracts,
drug labels, and patent claims as sources. Recently, ?) used the Scielo
scientific database to produce a parallel corpus of biomedical abstracts in
three language pairs: Portuguese-English, Spanish-English, and French-English.

The Scielo database is a Latin American and Caribbean initiative developed to
meet the needs of developing countries regarding scientific communications,
increasing the visibility and access to scientific literature [Packer, 2000].
Another interesting aspect of Scielo is that several journals publish full-
text of scientific articles in more than one language, a feature commonly
limited to the abstracts. Therefore, the Scielo database can be a valuable
source for parallel corpora for various scientific domains.

In this work, we developed a parallel corpus of full-text scientific articles
collected from the Scielo database in English (EN), Portuguese (PT) and,
Spanish (ES). The corpus is sentence aligned for all language pairs. We also
made available trilingual alignments for a subset of sentences. The main
differences with regard to a previous initiative of ?) are: (i) our corpus
contains full-text articles, providing a larger resource; (ii) we collected
data from various domains, not just biomedical; (iii) whenever possible, we
presented the articles in a structured way using sections and paragraphs,
favoring other NLP tasks such as summarization; and (iv) we included metadata,
such as journal name and subject category, which can be used for text
classification.

##  2\. License and Availability

Most articles in the Scielo database are licensed under the Creative Commons
copyright, with different types of licenses. In order to be able to distribute
the contents of the gathered articles, we filtered only those licensed under
terms that allow derivatives, since ND (No Derivatives) licenses require the
content to be distributed without any modification. As we removed some parts
from the articles (e.g. images, tables, references), we would be infringing
such copyright rules. All articles distributed in our dataset contain the
corresponding license, authorship, and unique identifiers of original sources,
as detailed in Section 4.1.

##  3\. Material and Methods

In this section, we detail the information gathered from Scielo, the filtering
process, as well as our method for article parsing and alignment. An overview
of the workflow employed in this article is shown in Figure 1

Figure 1: Steps employed in the development of the parallel corpora.

###  3.1. Document retrieval

Scielo’s website111http://www.scielo.org provides unified access to a series
of regional databases (such as from Argentina, Brazil, South Africa), offering
simple and advanced search capabilities. We iteratively queried the database
to retrieve all lists of results, which were then parsed and all relevant
contents stored, such as URLs for all available languages of each article,
authorship, licensing, title, and abstract. We adopted the MongoDB database
system, as it is document-oriented, and allows for the easy querying and
storage of this type of data.

We queried the results in MongoDB to filter only the articles meeting the
following constraints: a) articles with full-text available in at least two of
three languages of interest (i.e. English, Portuguese, and Spanish); and b)
type of licensing is non ND terms. The full-text of all articles meeting these
two criteria were downloaded from the Scielo database in HTML format.

###  3.2. Document parsing

The HTML contents of all articles were parsed using an in-house Python script
tailored to the Scielo format. First, all non-textual elements, such as
images, tables, references, citations, and footnotes were removed. Our
algorithm was designed to preserve the hierarchical and paragraph structure of
the article across the different languages in order to produce results aligned
at paragraph and section levels. This could help achieving good sentence level
alignment.

The main challenges in parsing Scielo HTML contents are heterogeneity issues
concerning HTML structure and formatting over different years. More recent
articles are well-formated and contain specific tags for paragraphs, sections,
subsections, and titles. We concentrated efforts in developing rules to tackle
all ill-formated HTML issues identified, so as to cover as much content as
possible, but to reduce the risk of misalignment, we discarded all documents
that presented very different structures across the languages.

Each parsed full-text translation was stored in MongoDB aiming at preserving
the structure of the articles. When our parsing algorithm failed at
identifying the document structure, its content was stored as a unstructured
list of paragraphs, as we assume that if two translations of the same article
present the same number of parsed paragraphs, it is likely they can be simply
aligned according to their order.

###  3.3. Sentence alignment

Once all articles were parsed, we employed a pre-processing step to ensure a
better alignment. We deleted all parentheses from the texts (mainly used for
citations), as well as newline/carriage return characters (i.e \n and \r), as
they would interfere with the sentence alignment tool.

For sentence alignment, we used the LF aligner
tool222https://sourceforge.net/projects/aligner/, a wrapper around the
Hunalign algorithm [Varga et al., 2007], which provides an easy to use and
complete solution for sentence alignment, including pre-loaded dictionaries
for several languages.

Hunalign uses Gale-Church sentence-length information to first automatically
build a dictionary based on this alignment. Once the dictionary is built, the
algorithm realigns the input text in a second iteration, this time combining
sentence-length information with the dictionary. When a dictionary is supplied
to the algorithm, the first step is skipped. A drawback of Hunalign is that it
is not designed to handle large corpora (above 10 thousand sentences), causing
large memory consumption. In these cases, the algorithm cuts the large corpus
in smaller manageable chunks, which may affect dictionary building.

For articles with the same structure across the languages, pairs of parallel
paragraphs were input to the sentence aligner at a time, aiming at reducing
the risk of misalignment. For the other cases, all paragraphs were passed to
the aligner together. Aligned sentences were stored as text files for post-
processing.

After sentence alignment, the following post-processing steps were performed:
(i) removal of all non-aligned sentences; (ii) removal of all sentences with
fewer than three characters, since they are likely to be noise from ill-
formatted HTML; (iii) removal of all sentences written in the same language
using a language detector333https://github.com/Mimino666/langdetect. This last
step was performed since abstracts in different languages could be present in
a full-text HTML, which could produce same-language alignments.

###  3.4. Machine translation evaluation

To evaluate the usefulness of our corpus for SMT purposes, we used it to train
an automatic translator with Moses moses_lang. The produced translations were
evaluated according to the BLEU score[Papineni et al., 2002], using the
evaluation tool multi-bleu in Moses.

###  3.5. Manual evaluation

Although the Hunalign algorithm usually presents a good alignment between
sentences, we also conducted a manual validation to evaluate the quality of
the aligned sentences. We randomly selected 300 pairs of sentences, 100 for
each language pair, and 100 trilingual sentences. If the pair was correctly
aligned, we marked it as ”correct”, otherwise, as ”no alignment”.

##  4\. Results and Discussion

In this section, we present statistics about the corpus and a quality
evaluation in terms of SMT and sentence alignment.

###  4.1. Corpus statistics

Table 1 shows the overall corpus statistics for all language pairs and for the
set of trilingual aligned documents. One may notice that EN-PT documents are
predominant over other language pairs. This may be explained by the fact that
almost all Brazilian journals are published through Scielo, thus favoring
Portuguese-English translations.

The datasets are available444https://figshare.com/s/091fcaf8ad66a3304e90 in
the TMX format [Rawat et al., 2016], since it is the standard format for
translation memories. Besides the aligned sentences, we included the following
metadata for each document: aligned title, authors, copyright license, DOI (if
available), journal name, Scielo’s unique identifier, and subject area. This
information was included either to fully comply with Creative Commons
requisites, or to provide additional information for other possible
applications, such as text classification or clustering.

An example of trilingual sentence is shown below:

> English: Among its objectives, it aims to defend the interests of society
> and Nursing in the context of Public Policies and the Unified Health System
> with emphasis on Mental Health.
>
> Spanish: Entre sus objetivos está defender los intereses de la sociedad y de
> la Enfermería en el contexto de las Políticas Públicas y del Sistema Único
> de Salud con énfasis en el área de la Salud Mental.
>
> Portuguse: Entre seus objetivos, visa defender os interesses da sociedade e
> da Enfermagem no contexto das Políticas Públicas e do Sistema Único de Saúde
> com ênfase na área de Saúde Mental.

Table 1: Corpus statistics for all language pairs and the trilingual set. ”Docs” refers to the number of documents, ”Sents” to the number of aligned sentences, and ”Tokens” is the number of tokens in each language. Languages | Docs | Sents | Tokens
---|---|---|---
EN-ES | 2,029 | 177,781 | 5.2M
5.7M
PT-ES | 76 | 4,987 | 140,434
151,148
EN-PT | 29,609 | 2.9M | 76.0M
77.3M
EN-PT-ES | 3,142 | 255,914 | 7.0M
7.8M
7.2M

###  4.2. SMT experiments

Prior to the SMT experiments, all sentences were randomly split in three
disjoint datasets for each language pair: training, tuning and test.
Approximately 85% of the aligned sentences were kept for training, 5% for
tuning and 10% for test. The translation models were built following Moses’
baseline system steps555http://www.statmt.org/moses/?n=moses.baseline.

Table 2 presents the BLEU scores for each language pair for the test set. We
included the best results from related work by ?) for the sake of comparison.
We highlight that their study was focused on titles and abstracts from
biomedical articles, while our corpus is focused on full-text content of
scientific articles in general.

Table 2: BLEU scores for translation using Moses. Previous related work by Neves et al.(2016) is also presented for comparison in the right-hand column. Language Pairs |  BLEU Current Work |  BLEU (Neves et al.2016)
---|---|---
EN-ES | EN$\rightarrow$ES | 36.88 | 32.75
ES$\rightarrow$EN | 37.93 | 30.53
PT-ES | PT$\rightarrow$ES | 62.63 | -
ES$\rightarrow$PT | 62.96 | -
EN-PT | EN$\rightarrow$PT | 48.51 | 33.37
PT$\rightarrow$EN | 49.24 | 31.78

Our EN-ES models presented considerably higher BLEU scores, despite dealing
with several domains. Regarding EN$\rightarrow$ES translation, our results are
approximately 4.13 percentage points (pp) higher, while about 7.4 pp better
for ES$\rightarrow$EN. Considering EN-PT, our results are more expressive.
BLEU score for EN$\rightarrow$PT is 15.48 pp higher, and 17.46 pp for
PT$\rightarrow$EN.

Another source of comparison is the EuroMatrix
project666http://matrix.statmt.org/matrix, which is an European funded
initiative to promote machine translation research for all pairs of the
European Union (EU) languages. When comparing our results to the reported
benchmarking BLEU scores available on the project’s website, our results are
similar to the reported BLEU scores for the Europarl corpus (scores between
29.4 and 39.4) and overall lower than the ones for the JRC-Acquis corpus
(between 55.1 and 60.7). The main differences among these three corpora are
domain related, since JRC-Acquis is comprised of EU laws applicable in its
state members, Europarl is derived from the EU Parliament proceedings, and our
corpus is from scientific articles. Thus, a variation in achieved BLEU scores
is expected. As stated by ?), JRC-Acquis corpus is of considerable size within
its very specific and well-defined domain of legal text, therefore presenting
good translation performance. On the other hand, Europarl corpus is a
transcription of speeches, thus inducing a greater linguistic variability.

###  4.3. Sentence alignment quality

We manually validated the alignment quality for 400 sentences randomly sampled
from the parsed corpus. Figure 2 depicts the rate of correct alignments for
each subset of parallel languages. All language combinations presented at
least 98% of correct alignments, with the language pair ES - PT achieving
100%. Different factors may have contributed to this high alignment quality.
The use of Hunalign [Varga et al., 2007] with a dictionary is perhaps the most
probable reason, as it combines a dictionary with sentence-length information
to boost alignments. The input of articles segmented by parallel paragraphs
also contributed to quality enhancement, since this can reduce the probability
of misalignment.

Figure 2: Alignment accuracy for the four language subsets.

During alignment validation, some translation incoherences were identified,
specially regarding nomenclatures, which may have affected the SMT
performance. An example of incorrect scientific nomenclature translation is:

> Portuguese: […]movimentos coreoatetosicos nos membros ipsilaterais ao lado
> comprometido

which was translated to:

> English: […] and ipsilateral coreoatetosis

while the correct translation would be using ”choreoathetosis”:

> English: […] and ipsilateral choreoathetosis.

Another nomenclature translation problem, but related to context, is:

> Spanish: […]estudiantes en el internado de la Escuela Superior de Medicina

which was translated to:

> English: […]boarding school students at the School of Medicine

while a better translation would be:

> English: […]medical intern students at the School of Medicine

since boarding schools are mainly associated to primary and secondary
education.

##  5\. Conclusion and Future Work

We developed a parallel corpus of scientific articles in three languages:
English, Portuguese and Spanish. Additionally to the language pairs, we
provided a subset of trilingual aligned sentences. Our corpus is based on
full-text contents from the Scielo database, which is available under open-
access licenses, thus favoring distribution.

We evaluated our corpus using an SMT experiment with Moses and by manual
evaluation of sentence alignment. Our translation experiment presented
superior performance regarding BLEU score than a previous related work on
Scielo database. We highlight the high translation scores achieved for PT-EN
language pairs, boosted by the large number of sentences (almost 3M). Hunalign
also presented remarkable alignment quality, with over 98% sentences correctly
aligned. Other important features of our corpus are the availability of
trilingual sentences, and the additional subset of articles aligned according
their hierarchical structures, which can be useful for automatic building of
structured abstracts.

Regarding future work, we foresee the use of this corpus in text mining
applications, such as classification and clustering. In addition, the corpus
could be used in cross-language plagiarism detection, as we provide at least
two versions of the same article in multiple languages. New Neural Machine
Translation (NMT) systems could be trained to provide a comparison against the
tested SMT system. An interesting approach would be use an approach similar to
Google’s multilingual NMT [Johnson et al., 2017] to perform zero-shot
translation based on an additional parallel corpus.

##  6\. Bibliographical References

## References

  * Johnson et al., 2017 Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al.  (2017).  Google’s multilingual neural machine translation system: Enabling zero-shot translation.  Transactions of the Association for Computational Linguistics, 5:339–351.
  * Koehn et al., 2009 Koehn, P., Birch, A., and Steinberger, R., (2009).  462 Machine Translation Systems for Europe, pages 65–72.  Association for Machine Translation in the Americas, AMTA.
  * Koehn, 2005 Koehn, P.  (2005).  Europarl: A parallel corpus for statistical machine translation.  In MT summit, volume 5, pages 79–86.
  * Kors et al., 2015 Kors, J. A., Clematide, S., Akhondi, S. A., Van Mulligen, E. M., and Rebholz-Schuhmann, D.  (2015).  A multilingual gold-standard corpus for biomedical concept recognition: the mantra gsc.  Journal of the American Medical Informatics Association, 22(5):948–956.
  * Neves et al., 2016 Neves, M., Yepes, A. J., and Névéol, A.  (2016).  The scielo corpus: a parallel corpus of scientific publications for biomedicine.  In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France, may. European Language Resources Association (ELRA).
  * Packer, 2000 Packer, A. L.  (2000).  Scielo-a model for cooperative electronic publishing in developing countries.  D-Lib Magazine, 6(10).
  * Papineni et al., 2002 Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J.  (2002).  Bleu: a method for automatic evaluation of machine translation.  In Proceedings of the 40th annual meeting on association for computational linguistics, pages 311–318. Association for Computational Linguistics.
  * Rawat et al., 2016 Rawat, S., Chandak, M. B., and Chauhan, N., (2016).  An Approach for Efficient Machine Translation Using Translation Memory, pages 285–291.  Springer Singapore, Singapore.
  * Skadins et al., 2014 Skadins, R., Tiedemann, J., Rozis, R., and Deksne, D.  (2014).  Billions of parallel words for free: Building and using the eu bookshop corpus.  In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2014), pages 1850–1855.
  * Utiyama and Isahara, 2007 Utiyama, M. and Isahara, H.  (2007).  A japanese-english patent parallel corpus.  MT summit XI, pages 475–482.
  * Varga et al., 2007 Varga, D., Halácsy, P., Kornai, A., Nagy, V., Németh, L., and Trón, V.  (2007).  Parallel corpora for medium density languages.  Amsterdam studies in the theory and history of Linguistic Science series 4, 292:247.
  * Wu et al., 2011 Wu, C., Xia, F., Deleger, L., and Solti, I.  (2011).  Statistical machine translation for biomedical text: are we there yet?  In AMIA Annual Symposium Proceedings, volume 2011, page 1290. American Medical Informatics Association.
  * Zhang et al., 2014 Zhang, S., Ling, W., and Dyer, C.  (2014).  Dual subtitles as parallel corpora.
  * Ziemski et al., 2016 Ziemski, M., Junczys-Dowmunt, M., and Pouliquen, B.  (2016).  The united nations parallel corpus v1.0.  In Nicoletta Calzolari (Conference Chair), et al., editors, Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France, may. European Language Resources Association (ELRA).

##  7\. Language Resource References

lrec xample