#  CERN-LPCC-2019-02
Proposal for the validation of Monte Carlo implementations
of the standard model effective field theory

Gauthier Durieux${}^{\ref{aff:Technion}}$ (ed.), Ilaria
Brivio${}^{\ref{aff:NBIA},\ref{aff:Heidelberg}}$ (ed.), Fabio
Maltoni${}^{\ref{aff:UCLouvain},\ref{aff:Bologna}}$ (ed. ex officio), Michael
Trott${}^{\ref{aff:NBIA}}$ (ed. ex officio), Simone
Alioli,${}^{\ref{aff:Bicocca-INFN}}$ Andy Buckley,${}^{\ref{aff:Glasgow}}$
Mauro Chiesa,${}^{\ref{aff:Wuerzburg}}$ Jorge de
Blas,${}^{\ref{aff:Padova},\ref{aff:INFN-Padova}}$ Athanasios
Dedes,${}^{\ref{aff:ioamnina}}$ Céline Degrande,${}^{\ref{aff:UCLouvain}}$
Ansgar Denner,${}^{\ref{aff:Wuerzburg}}$ Christoph
Englert,${}^{\ref{aff:Glasgow}}$ James Ferrando,${}^{\ref{aff:DESY}}$ Benjamin
Fuks,${}^{\ref{aff:LPTHE},\ref{aff:IUF}}$ Peter
Galler,${}^{\ref{aff:Glasgow}}$ Admir Greljo,${}^{\ref{aff:CERNTH}}$ Valentin
Hirschi,${}^{\ref{aff:ETH}}$ Gino Isidori,${}^{\ref{aff:Zurich}}$ Wolfgang
Kilian,${}^{\ref{aff:Siegen}}$ Frank Krauss,${}^{\ref{aff:Durham}}$ Jean-
Nicolas Lang,${}^{\ref{aff:Zurich}}$ Jonas Lindert,${}^{\ref{aff:Durham}}$
Michelangelo Mangano,${}^{\ref{aff:CERNTH}}$ David
Marzocca,${}^{\ref{aff:Trieste-INFN}}$ Olivier
Mattelaer,${}^{\ref{aff:UCLouvain}}$ Kentarou Mawatari,${}^{\ref{aff:Osaka}}$
Emanuele Mereghetti,${}^{\ref{aff:LosAlamos}}$ David J.
Miller,${}^{\ref{aff:Glasgow}}$ Ken Mimasu,${}^{\ref{aff:UCLouvain}}$ Michael
Paraskevas,${}^{\ref{aff:Warsaw}}$ Tilman Plehn,${}^{\ref{aff:Heidelberg}}$
Laura Reina,${}^{\ref{aff:fsu}}$ Janusz Rosiek,${}^{\ref{aff:Warsaw}}$ Jürgen
Reuter,${}^{\ref{aff:DESY}}$ José Santiago,${}^{\ref{aff:Granada}}$ Kristaq
Suxho,${}^{\ref{aff:ioamnina}}$ Lampros Trifyllis,${}^{\ref{aff:ioamnina}}$
Eleni Vryonidou,${}^{\ref{aff:CERNTH}}$ Christopher
White,${}^{\ref{aff:QueenMary}}$ Cen
Zhang,${}^{\ref{aff:IHEP},\ref{aff:Beijing}}$ Hantian
Zhang${}^{\ref{aff:Zurich}}$ 1 Physics Department, Technion-Israel Institute
of Technology, Haifa 32000, Israel
2 Niels Bohr International Academy and Discovery Center, Niels Bohr Institute,
University of Copenhagen, 2100 Copenhagen, Denmark
3 Institut für Theoretische Physik Universität Heidelberg, 69120 Heidelberg,
Germany
4 Centre for Cosmology, Particle Physics and Phenomenology (CP3), Université
catholique de Louvain, 1348 Louvain-la-Neuve, Belgium
5 Dipartimento di Fisica e Astronomia, Università di Bologna and INFN, Sezione
di Bologna, via Irnerio 46, 40126 Bologna, Italy
6 Università degli Studi di Milano-Bicocca and INFN, Sezione di Milano-
Bicocca, 20126, Milano, Italy
7 School of Physics and Astronomy, University of Glasgow, Glasgow, G12 8QQ, UK
8 Julius-Maximilians-Universität Würzburg, Institut für Theoretische Physik
und Astrophysik, Emil-Hilb-Weg 22, 97074 Würzburg, Germany
9 Dipartimento di Fisica e Astronomia “Galileo Galilei”, Università di Padova,
Via Marzolo 8, 35131 Padova, Italy
10 INFN, Sezione di Padova, Via Marzolo 8, 35131 Padova, Italy
11 Department of Physics, Division of Theoretical Physics, University of
Ioannina, 45110, Greece
12 DESY, Notkestraße 85, 22607 Hamburg, Germany
13 Laboratoire de Physique Théorique et Hautes Energies (LPTHE), UMR 7589,
Sorbonne Université et CNRS, 4 place Jussieu, 75252 Paris Cedex 05, France
14 Institut Universitaire de France, 103 boulevard Saint-Michel, 75005 Paris,
France
15 CERN, TH Department, Geneva, Switzerland
16 ETH Zürich, Rämistrasse 101, 8092 Zürich, Switzerland
17 Physik-Institut, Universität Zürich, 8057 Zürich, Switzerland
18 University of Siegen, Department of Physics, 57068 Siegen, Germany
19 Institute for Particle Physics Phenomenology, Department of Physics, Durham
University, Durham DH1 3LE, UK
20 INFN - Sezione di Trieste, Via Bonomea 265, 34136, Trieste, Italy
21 Department of Physics, Osaka University, Toyonaka, Osaka 560-0043, Japan
22 Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545,
USA
23 Faculty of Physics, University of Warsaw, Pasteura 5, 02-093 Warsaw, Poland
24 Department of Physics, Florida State University, 510 Keen Building,
Tallahassee, FL 32306-4350
25 CAFPE and Departamento de Física Teórica y del Cosmos, Universidad de
Granada, 18071 Granada, Spain
26 INFN, Sezione di Roma, Piazzale Aldo Moro 2, 00185 Roma, Italy
27 Centre for Research in String Theory, School of Physics and Astronomy,
Queen Mary University of London, 327 Mile End Road, London E1 4NS, UK
28 Institute of High Energy Physics, Chinese Academy of Sciences, Beijing
100049, China
29 School of Physical Sciences, University of Chinese Academy of Sciences,
Beijing 100049, China

###### Abstract

We propose a procedure to cross-validate Monte Carlo implementations of the
standard model effective field theory. It is based on the numerical comparison
of squared amplitudes computed at specific phase-space and parameter points in
pairs of implementations. Interactions are fully linearised in the effective
field theory expansion. The squares of linear effective field theory
amplitudes and their interference with standard-model contributions are
compared separately. Such pairwise comparisons are primarily performed at tree
level and a possible extension to the one-loop level is also briefly
considered. We list the current standard model effective field theory
implementations and the comparisons performed to date.

##  1 Introduction

This note proposes a procedure to validate Monte Carlo (MC) implementations
—more or less partial— of the standard model effective field theory (SMEFT).
The underlying agreement was achieved under the auspices of the LHC Top and
Electroweak Working Groups, and of the LHC Higgs Cross Section Working Group.

Most of the existing MC implementations are able to compute leading-order
predictions. Therefore, we primarily consider tree-level calculations. Many
implementations are based on Lagrangians written in codes such as Sarah [1] or
FeynRules [2] and then exported through standard format to matrix-element-
based MC generators. A systematic procedure for comparing MC implementations
is outlined through the comparison of squared amplitudes.

##  2 Existing implementations

A brief list of public SMEFT implementations with links and references is
provided below. It shall be updated as new implementations become available.

  * •

dim6top is a UFO implementation of top-quark interactions following the
conventions of the LHC top Working Group [3]. It is available at this
url11endnote: 1https://feynrules.irmp.ucl.ac.be/wiki/dim6top.

  * •

SMEFTsim is a complete UFO implementation of the Warsaw basis [4] of
dimension-six operators [5]. It is available at this url22endnote:
2https://feynrules.irmp.ucl.ac.be/wiki/SMEFT.

  * •

SMEFT@NLO is a UFO implementation, to next-to-leading order in QCD, of of CP-
and $U(2)_{q}\times U(2)_{u}\times U(3)_{d}\times U(3)_{l}\times
U(3)_{e}$-conserving dimension-six interactions, available at this
url33endnote: 3https://feynrules.irmp.ucl.ac.be/wiki/SMEFTatNLO.

  * •

SmeftFR [6, 7] is a package generating Feynman rules, in FeynRules and UFO
formats, for the dimension-six operators of the Warsaw basis [4] (or any
subset), in unitary or linear $R_{\xi}$ gauges, in terms of physical fields
(mass eigenstates), for general flavour structures. It is available at this
url44endnote: 4http://www.fuw.edu.pl/smeft.

  * •

HEL [8] is an implementation of dimension-six operators in the SILH basis [9]
available at this url55endnote: 5https://feynrules.irmp.ucl.ac.be/wiki/HEL.

  * •

BSMC [10] is an implementation of dimension-six operators in the Higgs basis
[11] associated with the Rosetta package (here66endnote:
6https://rosetta.hepforge.org). It is available at this url77endnote:
7http://feynrules.irmp.ucl.ac.be/wiki/BSMCharacterisation.

  * •

Powheg SM-EFT [12] is an implementation of SMEFT corrections to neutral and
charged current Drell-Yan and electroweak Higgs boson production, including
next-to-leading order QCD corrections and interfaced with parton showering and
hadronization according to the POWHEG method. It is available at this
url88endnote: 8http://powhegbox.mib.infn.it/.

  * •

Recola2 is a tree-level and one-loop matrix-element provider. While Recola
[13, 14] was specifically designed for the SM, Recola2 [15, 16] can be linked
against any user-provided model file. For the present comparison, a tree-level
model containing the full set of dimension-six operators in the Warsaw basis
was implemented. A preliminary version of the model was used in ref. [17] to
study the impact of the dimension-six operators contributing to anomalous
triple-gauge-boson interactions in vector-boson pair production at the LHC.

  * •

SFitter is a general purpose fitting tool that combines a large number of
experimental channels in a global fit to constrain dimension-six operators in
the HISZ basis. The basis is implemented in a UFO file and processed by
MadGraph5, yielding parametrizations of experimental observables in terms of
the Wilson coefficients. This implementation, extensively validated
internally, is not public.

  * •

HiggsPO is a UFO implementation of the Higgs pseudo-observables framework at
next-to-leading order in QCD [18, 19, 20], available at this url99endnote:
9https://www.physik.uzh.ch/data/HiggsPO.

##  3 Validation procedure

The cross-validation of MC implementations of new physics models is in general
a very time-consuming exercise, especially in the case of EFTs which feature
large number of new interactions and parameters. In addition, as existing
implementations rely on different operator bases, conventions and tools, a
flexible, robust and scalable approach needs to be employed. The main idea of
our proposal is to use physical predictions as terms of comparison. These are
by definition independent of a certain number of conventions, and can be
uniquely defined once the input parameters, the scheme employed and the order
of the computations are set.

A first possibility is to compare total cross sections for producing given
final states. Cross sections have the advantage of being easily obtainable in
any MC framework. Such procedure was for example applied in
ite]cite.AguilarSaavedra:2018nenref. [3] for cross-validating the
implementations of the top-quark sectors in SMEFTsim and dim6top frameworks.
However, total cross sections are intrinsically limited by the numerical
precision of the phase-space integration, which in some cases can entail
considerable computing time. For example, to test operators involving four top
quarks, processes such as $pp\to t\bar{t}t\bar{t}$ have to be considered and
corresponding cross sections computed. We therefore adopt here a much leaner
strategy that has been employed successfully in the past to compare models in
FeynRules [2] with native ones in generators such as MadGraph5 [21], Sherpa
[22], Whizard [23], Herwig++ [24], CalcHEP [25] and FeynArts/FormCalc [26,
27].

We propose the following validation procedure:

  1. 1.

Comparisons are performed between pairs of models, at a series of parameter
points.

One may not aim at a fully connected network of $n$ model with $n^{2}$
comparisons, a sparser one may be sufficient. It is expected that families of
similar implementations that are easier to compare among themselves will form.

  2. 2.

SM and EFT predictions are primarily considered at the tree level (see section
4 for an extension to the one-loop level).

  3. 3.

The same set of electroweak input parameters is to be used in both models when
comparing operators that shift them. Masses ($m_{Z}$ and $m_{W}$) are included
as input parameters to avoid SMEFT corrections to propagators that are not
easy to treat in MC programs.

Standard numerical values are input parameters for instance
$\\{G_{F}=1.16637\cdot 10^{-5}\,\text{GeV}^{-2},\linebreak
m_{Z}=91.1876\,\text{GeV},m_{W}=80.379\,\text{GeV},m_{h}=125\,\text{GeV},m_{t}=172\,\text{GeV},\alpha_{S}(m_{Z})=0.1184\\}$.

We also suggest to fix the CKM matrix to unity, unless another scheme is
adopted between the two models compared.

  4. 4.

Particle widths are explicitly set to zero in the comparison, to avoid
differences due to gauge choices in the two models. SMEFT corrections to
propagators, that are not easy to treat for MC programs, would otherwise again
be generated.

The dependence of decay amplitude squared on EFT parameters could also be
explicitly examined through one-to-$n$ squared amplitudes. One may also adopt
the complex mass scheme in the future.

  5. 5.

The couplings of EFT vertices are linearised in the $C/\Lambda^{2}$ expansion.
A linearisation of all internal parameters (such as coupling constants, the
Higgs vev, electroweak inputs, etc.) and of their functions (denominators,
square roots, power, etc.) is in particular required. This allows for the
isolation by MC programs of amplitudes themselves linear in this expansion.
Equivalent parameter points in different implementations are then also
obtained through a linear mapping.

  6. 6.

The computation of a set of two-to-$n$ (and one-to-two) squared amplitudes,
averaged over initial colours and helicities, summed over final ones, is
performed at given phase space points (which may not necessarily physically
representative). Numerical values are quoted in units of GeV to the
appropriate power.

In the future, one could consider comparing helicity amplitudes squared.

When targetting Higgs or top-quark EFT interactions, the set of all SM two-to-
two processes (excluding crossing-symmetric and conjugates) involving at least
one of these particles can for instance be considered for comparisons,
together with selected two-to-three processes like `g g > t t~ h` and `q q~ >
q q~ h/a/z/w` processes (see table 2).

  7. 7.

In addition to the SM contribution $|A_{\text{SM}}|^{2}$, the interference
$2\operatorname{Re}\\{A_{\text{SM}}^{*}A_{\text{EFT}}\\}$ and squared
$|A_{\text{EFT}}|^{2}$ EFT contributions are evaluated for equivalent
parameter points in the two models.

Although the $|A_{\text{EFT}}|^{2}$ contributions do not constitute complete
EFT results to $1/\Lambda^{4}$ order, they are unambiguously defined (since
they arise from the square of leading EFT contributions at the amplitude level
which are themselves unambiguously defined) and useful for the comparison of
implementations, in particular in cases where the
$2\operatorname{Re}\\{A_{\text{SM}}^{*}A_{\text{EFT}}\\}$ interference
vanishes.

Considered parameter points may for instance correspond to one single operator
coefficient set to a non-vanishing value in one of the two models compared.
One should then remember to check the relative sign of operators having
vanishing interferences with the SM model by considering parameter points
where two of them are non vanishing. Establishing sets of equivalent parameter
points in implementations relying on different operator bases may require some
effort.

  8. 8.

Information about the implementations compared, parameter points, phase-space
points, and averaged squared amplitudes are stored in LHE format (see v1 [28],
v21010endnote:
10http://www.lpthe.jussieu.fr/LesHouches09Wiki/index.php/LHEF_for_Matching
[29] and v31111endnote: 11https://phystev.cnrs.fr/wiki/2013:groups:tools:lhef3
descriptions).

Information about the generators, model versions, processes and parameter
points used to compute the amplitude squared corresponding to a given id are
provided under the standard <weight> header (see table 1 for an example). The
generator version used is specified in a <weight_generator> sub-header, the
model version in a <weight_model_version> one. The date at which a UFO model
has been generated can for instance be specified here to be as specific as
possible. Information about the generated processes (numbered), the coupling
order restrictions, etc. are specified in a generator-dependent format in a
<weight_card> sub-header. The parameter point is provided in SLHA format in a
<weight_slha> sub-header.

The <event> field is used to store the phase-space point at which squared
amplitudes are evaluated. The values of the latter for every id specified
earlier are stored in a <wgt> field. The `id` label can be chosen to be human-
readable and to include, separated by hyphens, information about the
implementation, the parameter point, and a `sq` or `int` label specifying
whether it corresponds to a squared or interference term (see again table 1
for an example).

        <LesHouchesEvents version="3.0">
      <header>
        ...
        <initrwgt>
          <weightgroup name=’mg_reweighting’ weight_name_strategy=’includeIdInWeightName’>
            <weight id=’mod1-sm-sq’>
<EMAIL_ADDRESS>              <weight_model-version>dim6top_LO_UFO v1.0 Sat 8 Dec 2018 14:17:45</weight_model_version>
              <weight_card>
                change model SMEFTsim_A_general_MwScheme_UFO_v2-sm
                change process  g g > t~ t FCNC=0 QED<=99 QCD<=99 NP=0 @1
                change process  g a > t~ t FCNC=0 QED<=99 QCD<=99 NP=0 @2
                change process  g t > z  t FCNC=0 QED<=99 QCD<=99 NP=0 @3
                ...
              </weight_card>
              <weight_slha>
                Block EFT
                   22 0. # ctB
                   23 0. # ctW
                ...
              </weight_slha>
            </weight>
            <weight id=’mod1-ctW-int’>
<EMAIL_ADDRESS>              <weight_model_version>dim6top_LO_UFO v1.0 Sat 8 Dec 2018 14:17:45</weight_model_version>
              <weight_card>
                change model SMEFTsim_A_general_MwScheme_UFO_v2-ctW
                change process  g g > t~ t FCNC=0 QED<=99 QCD<=99 NP<=1 NP^2==1 @1
                change process  g a > t~ t FCNC=0 QED<=99 QCD<=99 NP<=1 NP^2==1 @2
                change process  g t > z  t FCNC=0 QED<=99 QCD<=99 NP<=1 NP^2==1 @3
                ...
              </weight_card>
              <weight_slha>
                Block EFT
                   22 0. # ctB
                   23 1. # ctW
                ...
              </weight_slha>
            </weight>
            ...
          </weightgroup>
        </initrwgt>
      </header>
      ...
      <event>
         4      2 +5.7925728e-01 9.11180000e+01 0.00000000e+00 1.18400000e-01
               21 -1    0    0    0    0 +0.00e+00 +0.00e+00 +5.00e+02 5.00e+02 0.00e+00 0.00e+00 9.00e+00
               22 -1    0    0    0    0 +0.00e+00 +0.00e+00 -5.00e+02 5.00e+02 0.00e+00 0.00e+00 9.00e+00
               -6  1    1    2    0    0 +1.66e+02 -3.90e+02 +2.01e+02 5.00e+02 1.72e+02 0.00e+00 9.00e+00
                6  1    1    2    0    0 -1.66e+02 +3.90e+02 -2.01e+02 5.00e+02 1.72e+02 0.00e+00 9.00e+00
        <rwgt>
        <wgt id=’mod1-sm-sq’> +2.1166989e-01 </wgt>
        <wgt id=’mod1-ctW-int’> +3.6769585e-01 </wgt>
        <wgt id=’mod1-ctZ-int’> -3.2187585e-01 </wgt>
        <wgt id=’mod1-ctG-int’> +3.0451935e-02 </wgt>
        <wgt id=’mod1-ctW-sq’> +1.5438777e+00 </wgt>
        <wgt id=’mod1-ctZ-sq’> +1.1830749e+00 </wgt>
        <wgt id=’mod1-ctG-sq’> +1.0589251e-02 </wgt>
        <wgt id=’mod2-sm-sq’> +2.1166988e-01 </wgt>
        <wgt id=’mod2-ctW-int’> +3.6769591e-01 </wgt>
        <wgt id=’mod2-ctZ-int’> -3.2187587e-01 </wgt>
        <wgt id=’mod2-ctG-int’> +3.0451934e-02 </wgt>
        <wgt id=’mod2-ctW-sq’> +1.5438783e+00 </wgt>
        <wgt id=’mod2-ctZ-sq’> +1.1830750e+00 </wgt>
        <wgt id=’mod2-ctG-sq’> +1.0589251e-02 </wgt>
        ...
        </rwgt>
      </event>
      ...
    </LesHouchesEvents>

Table 1: Example of LHE format (based on the SLHA3 convention) including
headers specifying the EFT implementations, processes (numbered with @i in
MadGraph syntax), parameter points. The event record relative to a given
process and parameter point specifies the phase-space point and the averaged
squared amplitudes obtained for the specified implementations, parameter
points and EFT order.

  9. 9.

Cross-validations realized by model authors will be referenced in section 5 of
this note. Existing LHE files can serve as basis of comparison for additional
models and be complemented with the numerical values they give rise to.

  10. 10.

Comparison authors are encouraged to implement the basis translation(s)
established in the public tools Rosetta [10] (available at this
url1212endnote: 12https://rosetta.hepforge.org) and wcxf-python [30]
(available at this url1313endnote: 13https://wcxf.github.io/python.html). Such
tools can then be used to generate SLHA inputs for equivalent parameter
points. Note that basis translations should be kept linear in the EFT
expansion to comply with item 5.

Higgs
---
`g g > g h`
`g g > h h`
`g u > u h`
`g c > c h`
`g d > d h`
`g s > s h`
`g b > b h`
`g t > t h`
`u u~ > a h`
`u u~ > z h`
`u d~ > w+ h`
`c c~ > a h`
`c c~ > z h`
`c s~ > w+ h`
`d u~ > w- h`
`d d~ > a h`
`d d~ > z h`
`s c~ > w- h`
`s s~ > a h`
`s s~ > z h`
`b b~ > a h`
`b b~ > z h`
`b t~ > w- h`
`b~ t > w+ h`
`a a > h h`
`a ve > ve h`
`a vm > vm h`
`a vt > vt h`
`a e- > e- h`
`a mu- > mu- h`
`a ta- > ta- h`
`a t > t h`
`a z > h h`
`a w+ > w+ h`
`ve ve~ > z h`
`ve e+ > w+ h`
`vm vm~ > z h`
`vm mu+ > w+ h`
`vt vt~ > z h`
`vt ta+ > w+ h`
`e- ve~ > w- h`
`e- e+ > z h`
`mu- vm~ > w- h`
`mu- mu+ > z h`
`ta- vt~ > w- h`
`ta- ta+ > z h`
`t t~ > z h`
`t t~ > h h`
`z z > h h`
`z w+ > w+ h`
`w+ h > w+ h`
`h h > h h`

top
---
`g g > t~ t`
`g b > w- t`
`g a > t~ t`
`g t > z t`
`u b > d t`
`u u~ > t~ t`
`c b > s t`
`c c~ > t~ t`
`d d~ > t~ t`
`s s~ > t~ t`
`b b~ > t~ t`
`b a > w- t`
`b ve > e- t`
`b vm > mu- t`
`b vt > ta- t`
`b z > w- t`
`a a > t~ t`
`a t > z t`
`ve ve~ > t~ t`
`vm vm~ > t~ t`
`vt vt~ > t~ t`
`e- e+ > t~ t`
`mu- mu+ > t~ t`
`ta- ta+ > t~ t`
`t t > t t`
`t z > z t`
`t w+ > w+ t`

two-to-three
---
`g g > t t~ h`
`u u~ > u u~ g`
`u u~ > u u~ a`
`u u~ > u u~ z`
`u u~ > u u~ h`
`u u~ > u d~ w-`
`u u~ > c c~ g`
`u u~ > c c~ a`
`u u~ > c c~ z`
`u u~ > c c~ h`
`u u~ > c s~ w-`
`u u~ > d u~ w+`
`u u~ > d d~ g`
`u u~ > d d~ a`
`u u~ > d d~ z`
`u u~ > d d~ h`
`u u~ > s c~ w+`
`u u~ > s s~ g`
`u u~ > s s~ a`
`u u~ > s s~ z`
`u u~ > s s~ h`
`u u~ > b b~ g`
`u u~ > b b~ a`
`u u~ > b b~ z`
`u u~ > b b~ h`
`u u~ > b t~ w+`
`u u~ > t b~ w-`
`u u~ > t t~ g`
`u u~ > t t~ a`
`u u~ > t t~ z`
`u u~ > t t~ h`
`u c~ > d c~ w+`
`u c~ > d s~ g`
`u c~ > d s~ a`
`u c~ > d s~ z`
`u c~ > d s~ h`
`u d~ > d d~ w+`
`u d~ > s s~ w+`
`u d~ > b b~ w+`
`u d~ > t b~ g`
`u d~ > t b~ a`
`u d~ > t b~ z`
`u d~ > t b~ h`
`u d~ > t t~ w+`

`c u~ > c d~ w-`
---
`c u~ > s d~ g`
`c u~ > s d~ a`
`c u~ > s d~ z`
`c u~ > s d~ h`
`c c~ > c c~ g`
`c c~ > c c~ a`
`c c~ > c c~ z`
`c c~ > c c~ h`
`c c~ > c s~ w-`
`c c~ > d d~ g`
`c c~ > d d~ a`
`c c~ > d d~ z`
`c c~ > d d~ h`
`c c~ > s c~ w+`
`c c~ > s s~ g`
`c c~ > s s~ a`
`c c~ > s s~ z`
`c c~ > s s~ h`
`c c~ > b b~ g`
`c c~ > b b~ a`
`c c~ > b b~ z`
`c c~ > b b~ h`
`c c~ > b t~ w+`
`c c~ > t b~ w-`
`c c~ > t t~ g`
`c c~ > t t~ a`
`c c~ > t t~ z`
`c c~ > t t~ h`
`c d~ > s d~ w+`
`c s~ > s s~ w+`
`c s~ > b b~ w+`
`c s~ > t b~ g`
`c s~ > t b~ a`
`c s~ > t b~ z`
`c s~ > t b~ h`
`c s~ > t t~ w+`
`d u~ > d d~ w-`
`d u~ > s s~ w-`
`d u~ > b b~ w-`
`d u~ > b t~ g`
`d u~ > b t~ a`
`d u~ > b t~ z`
`d u~ > b t~ h`
`d u~ > t t~ w-`

`d c~ > d s~ w-`
---
`d d~ > d d~ g`
`d d~ > d d~ a`
`d d~ > d d~ z`
`d d~ > d d~ h`
`d d~ > s s~ g`
`d d~ > s s~ a`
`d d~ > s s~ z`
`d d~ > s s~ h`
`d d~ > b b~ g`
`d d~ > b b~ a`
`d d~ > b b~ z`
`d d~ > b b~ h`
`d d~ > b t~ w+`
`d d~ > t b~ w-`
`d d~ > t t~ g`
`d d~ > t t~ a`
`d d~ > t t~ z`
`d d~ > t t~ h`
`s c~ > s s~ w-`
`s c~ > b b~ w-`
`s c~ > b t~ g`
`s c~ > b t~ a`
`s c~ > b t~ z`
`s c~ > b t~ h`
`s c~ > t t~ w-`
`s s~ > s s~ g`
`s s~ > s s~ a`
`s s~ > s s~ z`
`s s~ > s s~ h`
`s s~ > b b~ g`
`s s~ > b b~ a`
`s s~ > b b~ z`
`s s~ > b b~ h`
`s s~ > b t~ w+`
`s s~ > t b~ w-`
`s s~ > t t~ g`
`s s~ > t t~ a`
`s s~ > t t~ z`
`s s~ > t t~ h`
`b b~ > b b~ g`
`b b~ > b b~ a`
`b b~ > b b~ z`
`b b~ > b b~ h`
`b b~ > b t~ w+`

`b b~ > t b~ w-`
---
`b b~ > t t~ g`
`b b~ > t t~ a`
`b b~ > t t~ z`
`b b~ > t t~ h`
`b t~ > t t~ w-`
`t b~ > t t~ w+`
`t t~ > t t~ g`
`t t~ > t t~ a`
`t t~ > t t~ z`
`t t~ > t t~ h`

Table 2: Illustrative sets of two-to-two SM processes involving at least a
Higgs or a top-quark which can be used as basis for a comparison. Examples of
two-to-three processes including $gg\to t\bar{t}h$ and $q\bar{q}\to
q\bar{q}\;g/\gamma/Z/W/h$ ones.

##  4 Extension to the one-loop level

Relying on the BLHA [31, 32] conventions for loop amplitude providers, one can
also extend the comparison of SMEFT amplitudes to the loop level.

A setup block denoted as <weight_blha> is added to the <weight> header. In
addition to BLHA2 parameters the value of the renormalization scale `mu` is
also specified. Following, for instance, the example provided in figure 6 of
ite]cite.Alioli:2013ndaref. [32], one would obtain a format like that of table
3.

Colour- and helicity-summed pole residues would then be specified in a
(PoleCoeff0,PoleCoeff1, PoleCoeff2) triplet stored in <wgt> fields for each
implementation, process, phase-space point, parameter point, and EFT order.
The squared terms correspond here to loop amplitudes with one EFT operator
insertion squared against tree amplitudes featuring also one EFT operator
insertion.

As mentioned in item 6, one could consider comparing squared helicity
amplitudes in the future.

    <LesHouchesEvents version="3.0">
      <header>
        <initrwgt>
          <weightgroup name=’...’ weight_name_strategy=’...’>
            <weight id=’...’>
              ...
              <weight_blha>
                InterfaceVersion        BLHA2
                MatrixElementSquareType CHsummed
                CorrectionType          QCD
                IRregularisation        DREG
                WidthScheme             ComplexMass
                EWScheme                alphaGF
                AccuracyTarget          0.0001
                CouplingPower QCD       2
                CouplingPower QED       0
                mu                      91.188         # renormalization scale
              </weight_blha>
            </weight>
            ...
          </weightgroup>
        </initrwgt>
      </header>
      ...
      <event>
         4      2 +5.7925728e-01 9.11180000e+01 0.00000000e+00 1.18400000e-01
               21 -1    0    0    0    0 +0.00e+00 +0.00e+00 +5.00e+02 5.00e+02 0.00e+00 0.00e+00 9.00e+00
               22 -1    0    0    0    0 +0.00e+00 +0.00e+00 -5.00e+02 5.00e+02 0.00e+00 0.00e+00 9.00e+00
               -6  1    1    2    0    0 +1.66e+02 -3.90e+02 +2.01e+02 5.00e+02 1.72e+02 0.00e+00 9.00e+00
                6  1    1    2    0    0 -1.66e+02 +3.90e+02 -2.01e+02 5.00e+02 1.72e+02 0.00e+00 9.00e+00
        <rwgt>
        <wgt id=’mod1-sm-sq’> (+2.1166989e-01, +3.6769585e-01, -3.2187585e-01) </wgt>
        <wgt id=’mod1-ctW-int’> (+3.6769585e-01, +1.5438777e+00, +2.1166989e-01) </wgt>
        <wgt id=’mod1-ctW-sq’> (+1.5438777e+00, -3.2187585e-01, +1.5438777e+00) </wgt>
        ...
        </rwgt>
      </event>
      ...
    </LesHouchesEvents>

Table 3: Extension of the LHE formatting for one-loop processes. A
<weight_blha> sub-header is added with BLHA information and the residues of
$\epsilon$ poles are quoted in <wgt> fields as (PoleCoeff0, PoleCoeff1,
PoleCoeff2) triplets.

##  5 Comparisons performed

A list of the comparisons performed between different implementations,
according to the guidelines of this note, is provided here with references to
their detailed discussion. It shall be updated as new comparisons become
available.

  * •

A comparison of top-quark interactions in dim6top, SMEFTsim and SMEFT@NLO UFO
models obtained with a dedicated MadGraph plugin are provided at this
url1414endnote:
14https://bazaar.launchpad.net/~rwgtdim6/mg5amcnlo/plugin_eft_contrib/revision.
More explanations are provided in this file1515endnote:
15https://bazaar.launchpad.net/~rwgtdim6/mg5amcnlo/plugin_eft_contrib/view/head:/example/compare_models_top.py.

  * •

The Recola2 model file including the dimension-six operators in the Warsaw
basis has been cross-checked against the SMEFTsim_A_U35_MwScheme_UFO_v2_1 UFO
model file for the subset of operators that only involve gauge bosons and/or
scalars. The numerical results corresponding to the former model have been
obtained with the Recola2 matrix-element provider, while Madgraph5_aMC@NLO was
used for the latter model. A sample output file in the LHE format described
above can be found at the url1616endnote:
16https://recola.hepforge.org/modelfiles/smeftcmp.html for a few selected
processes. As can be seen from the LHE file, the only differences between the
predictions of the two models are some interference matrix-elements that
however correspond to numerical zeros.

  * •

A comparison of HEL and BSMC implementations is ongoing.

###### Links

  1. 1 https://feynrules.irmp.ucl.ac.be/wiki/dim6top
  2. 2 https://feynrules.irmp.ucl.ac.be/wiki/SMEFT
  3. 3 https://feynrules.irmp.ucl.ac.be/wiki/SMEFTatNLO
  4. 4 http://www.fuw.edu.pl/smeft
  5. 5 https://feynrules.irmp.ucl.ac.be/wiki/HEL
  6. 6 https://rosetta.hepforge.org
  7. 7 http://feynrules.irmp.ucl.ac.be/wiki/BSMCharacterisation
  8. 8 http://powhegbox.mib.infn.it/
  9. 9 https://www.physik.uzh.ch/data/HiggsPO
  10. 10 http://www.lpthe.jussieu.fr/LesHouches09Wiki/index.php/LHEF_for_Matching
  11. 11 https://phystev.cnrs.fr/wiki/2013:groups:tools:lhef3
  12. 12 https://rosetta.hepforge.org
  13. 13 https://wcxf.github.io/python.html
  14. 14 https://bazaar.launchpad.net/~rwgtdim6/mg5amcnlo/plugin_eft_contrib/revision
  15. 15 https://bazaar.launchpad.net/~rwgtdim6/mg5amcnlo/plugin_eft_contrib/view/head:/example/compare_models_top.py
  16. 16 https://recola.hepforge.org/modelfiles/smeftcmp.html

## References

  * Staub [2015] F. Staub, _Exploring new models in all detail with SARAH_ , Adv. High Energy Phys. 2015 (2015) 840780, arXiv:1503.04200 [hep-ph].
  * Alloul _et al._ [2014a] A. Alloul, N. D. Christensen, C. Degrande, C. Duhr, and B. Fuks, _FeynRules 2.0 - A complete toolbox for tree-level phenomenology_ , Comput. Phys. Commun. 185 (2014a) 2250, arXiv:1310.1921 [hep-ph].
  * [3] J. A. Aguilar-Saavedra, C. Degrande, G. Durieux, F. Maltoni, E. Vryonidou, and C. Zhang, _Interpreting top-quark LHC measurements in the standard-model effective field theory_ , arXiv:1802.07237 [hep-ph].
  * Grzadkowski _et al._ [2010] B. Grzadkowski, M. Iskrzynski, M. Misiak, and J. Rosiek, _Dimension-Six Terms in the Standard Model Lagrangian_ , JHEP 10 (2010) 085, arXiv:1008.4884 [hep-ph].
  * Brivio _et al._ [2017] I. Brivio, Y. Jiang, and M. Trott, _The SMEFTsim package, theory and tools_ , JHEP 12 (2017) 070, arXiv:1709.06492 [hep-ph].
  * Dedes _et al._ [2017] A. Dedes, W. Materkowska, M. Paraskevas, J. Rosiek, and K. Suxho, _Feynman rules for the Standard Model Effective Field Theory in R ξ -gauges_, JHEP 06 (2017) 143, arXiv:1704.03888 [hep-ph].
  * Dedes _et al._ [2019] A. Dedes, M. Paraskevas, J. Rosiek, K. Suxho, and L. Trifyllis, _SmeftFR - Feynman rules generator for the Standard Model Effective Field Theory_ , (2019), arXiv:1904.03204 [hep-ph].
  * Alloul _et al._ [2014b] A. Alloul, B. Fuks, and V. Sanz, _Phenomenology of the Higgs Effective Lagrangian via FEYNRULES_ , JHEP 04 (2014b) 110, arXiv:1310.5150 [hep-ph].
  * Giudice _et al._ [2007] G. F. Giudice, C. Grojean, A. Pomarol, and R. Rattazzi, _The Strongly-Interacting Light Higgs_ , JHEP 06 (2007) 045, arXiv:hep-ph/0703164.
  * Falkowski _et al._ [2015] A. Falkowski, B. Fuks, K. Mawatari, K. Mimasu, F. Riva, and V. Sanz, _Rosetta: an operator basis translator for Standard Model effective field theory_ , Eur. Phys. J. C75 (2015) 583, arXiv:1508.05895 [hep-ph].
  * Falkowski [2015] A. Falkowski, _Higgs Basis: Proposal for an EFT basis choice for LHC HXSWG_ , (2015).
  * Alioli _et al._ [2018] S. Alioli, W. Dekens, M. Girard, and E. Mereghetti, _NLO QCD corrections to SM-EFT dilepton and electroweak Higgs boson production, matched to parton shower in POWHEG_ , JHEP 08 (2018) 205, arXiv:1804.07407 [hep-ph].
  * Actis _et al._ [2013] S. Actis, A. Denner, L. Hofer, A. Scharf, and S. Uccirati, _Recursive generation of one-loop amplitudes in the Standard Model_ , JHEP 04 (2013) 037, arXiv:1211.6316 [hep-ph].
  * Actis _et al._ [2017] S. Actis, A. Denner, L. Hofer, J.-N. Lang, A. Scharf, and S. Uccirati, _RECOLA: REcursive Computation of One-Loop Amplitudes_ , Comput. Phys. Commun. 214 (2017) 140, arXiv:1605.01090 [hep-ph].
  * Denner _et al._ [2017] A. Denner, J.-N. Lang, and S. Uccirati, _NLO electroweak corrections in extended Higgs Sectors with RECOLA2_ , JHEP 07 (2017) 087, arXiv:1705.06053 [hep-ph].
  * Denner _et al._ [2018] A. Denner, J.-N. Lang, and S. Uccirati, _Recola2: REcursive Computation of One-Loop Amplitudes 2_ , Comput. Phys. Commun. 224 (2018) 346, arXiv:1711.07388 [hep-ph].
  * Chiesa _et al._ [2018] M. Chiesa, A. Denner, and J.-N. Lang, _Anomalous triple-gauge-boson interactions in vector-boson pair production with RECOLA2_ , Eur. Phys. J. C78 (2018) 467, arXiv:1804.01477 [hep-ph].
  * Gonzalez-Alonso _et al._ [2015] M. Gonzalez-Alonso, A. Greljo, G. Isidori, and D. Marzocca, _Pseudo-observables in Higgs decays_ , Eur. Phys. J. C75 (2015) 128, arXiv:1412.6038 [hep-ph].
  * Greljo _et al._ [2016] A. Greljo, G. Isidori, J. M. Lindert, and D. Marzocca, _Pseudo-observables in electroweak Higgs production_ , Eur. Phys. J. C76 (2016) 158, arXiv:1512.06135 [hep-ph].
  * Greljo _et al._ [2017] A. Greljo, G. Isidori, J. M. Lindert, D. Marzocca, and H. Zhang, _Electroweak Higgs production with HiggsPO at NLO QCD_ , Eur. Phys. J. C77 (2017) 838, arXiv:1710.04143 [hep-ph].
  * Alwall _et al._ [2014] J. Alwall, R. Frederix, S. Frixione, V. Hirschi, F. Maltoni, O. Mattelaer, H. S. Shao, T. Stelzer, P. Torrielli, and M. Zaro, _The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations_ , JHEP 07 (2014) 079, arXiv:1405.0301 [hep-ph].
  * Gleisberg _et al._ [2009] T. Gleisberg, S. Hoeche, F. Krauss, M. Schonherr, S. Schumann, F. Siegert, and J. Winter, _Event generation with SHERPA 1.1_ , JHEP 02 (2009) 007, arXiv:0811.4622 [hep-ph].
  * Kilian _et al._ [2011] W. Kilian, T. Ohl, and J. Reuter, _WHIZARD: Simulating Multi-Particle Processes at LHC and ILC_ , Eur. Phys. J. C71 (2011) 1742, arXiv:0708.4233 [hep-ph].
  * Bahr _et al._ [2008] M. Bahr _et al._ , _Herwig++ Physics and Manual_ , Eur. Phys. J. C58 (2008) 639, arXiv:0803.0883 [hep-ph].
  * Belyaev _et al._ [2013] A. Belyaev, N. D. Christensen, and A. Pukhov, _CalcHEP 3.4 for collider physics within and beyond the Standard Model_ , Comput. Phys. Commun. 184 (2013) 1729, arXiv:1207.6082 [hep-ph].
  * Hahn [2001] T. Hahn, _Generating Feynman diagrams and amplitudes with FeynArts 3_ , Comput. Phys. Commun. 140 (2001) 418, arXiv:hep-ph/0012260.
  * Hahn _et al._ [2016] T. Hahn, S. Paßehr, and C. Schappacher, _FormCalc 9 and Extensions_ , _Proceedings, 13th DESY Workshop on Elementary Particle Physics: Loops and Legs in Quantum Field Theory (LL2016): Leipzig, Germany, April 24-29, 2016_ , PoS LL2016 (2016) 068, [J. Phys. Conf. Ser.762,no.1,012065(2016)], arXiv:1604.04611 [hep-ph].
  * Alwall _et al._ [2007] J. Alwall _et al._ , _A Standard format for Les Houches event files_ , _Monte Carlos for the LHC: A Workshop on the Tools for LHC Event Simulation (MC4LHC) Geneva, Switzerland, July 17-16, 2006_ , Comput. Phys. Commun. 176 (2007) 300, arXiv:hep-ph/0609017.
  * Butterworth _et al._ [2010] J. M. Butterworth _et al._ , _THE TOOLS AND MONTE CARLO WORKING GROUP Summary Report from the Les Houches 2009 Workshop on TeV Colliders_ , in _Physics at TeV colliders. Proceedings, 6th Workshop, dedicated to Thomas Binoth, Les Houches, France, June 8-26, 2009_ (2010) arXiv:1003.1643 [hep-ph].
  * Aebischer _et al._ [2018] J. Aebischer _et al._ , _WCxf: an exchange format for Wilson coefficients beyond the Standard Model_ , Comput. Phys. Commun. 232 (2018) 71, arXiv:1712.05298 [hep-ph].
  * Binoth _et al._ [2010] T. Binoth _et al._ , _A Proposal for a Standard Interface between Monte Carlo Tools and One-Loop Programs_ , _Physics at TeV colliders. Proceedings, 6th Workshop, dedicated to Thomas Binoth, Les Houches, France, June 8-26, 2009_ , Comput. Phys. Commun. 181 (2010) 1612, [,1(2010)], arXiv:1001.1307 [hep-ph].
  * Alioli _et al._ [2014] S. Alioli _et al._ , _Update of the Binoth Les Houches Accord for a standard interface between Monte Carlo tools and one-loop programs_ , Comput. Phys. Commun. 185 (2014) 560, arXiv:1308.3462 [hep-ph].


# Phase transition at 350 K in the Ti3C2Tx MXene: possible sliding (moiré)
ferroelectricity

Francesco Cordero<EMAIL_ADDRESS>Istituto di Struttura della
Materia-CNR (ISM-CNR), Area della Ricerca di Roma - Tor Vergata, Via del Fosso
del Cavaliere 100, I-00133 Roma, Italy  Hanna Pazniak  Université Grenoble
Alpes, CNRS, Grenoble INP, LMGP, 3 parvis Luois Néel, F-38000 Grenoble, France
Thierry Ouisse  Université Grenoble Alpes, CNRS, Grenoble INP, LMGP, 3 parvis
Luois Néel, F-38000 Grenoble, France  Jesus Gonzalez-Julian  Department of
Ceramics, Institute of Mineral Engineering, RWTH Aachen University, D-52074
Aachen, Germany  Aldo Di Carlo  Istituto di Struttura della Materia-CNR (ISM-
CNR), Area della Ricerca di Roma - Tor Vergata, Via del Fosso del Cavaliere
100, I-00133 Roma, Italy  Viktor Soprunyuk  Faculty of Physics, University of
Vienna, Boltzmanngasse 5, 1090 Wien, Austria  Wilfried Schranz  Faculty of
Physics, University of Vienna, Boltzmanngasse 5, 1090 Wien, Austria

###### Abstract

A phase transition is found in Ti3C2Tx MXene at 350 K, by measuring the
complex Young’s modulus of self-standing thick films. A step-like softening
and increase of the mechanical losses is found below 350 K, indicative of a
phase transition, where the square of the order parameter is coupled to
strain. It is argued that it should be a ferroelectric transition, most likely
of the sliding (moiré) type, due to charge transfer between facing flakes
sliding with respect to each other. If the transition will be confirmed to be
ferroelectric, Ti3C2Tx will be added to the class of metallic ferroelectrics
and open new perspectives of applications, in addition to the numerous already
studied.

###### pacs:

77.80.B-, 62.40.+i, 77.84.Cg, 77.22.Ch

Up to recent times it was believed that ferroelectric transitions cannot occur
in materials with sizes below a certain limit. Indeed, the critical thickness
for a film to sustain ferroelectricity (FE) is found to be of the order of
tens of nanometers in ferroelectrics with perovskite or fluorite structure
PKS23 . After the prediction of FE with giant piezoelectricity in the
monolayer chalcogenides SnSe, SnS, GeSe, and GeS FKY16  and the experimental
verifications that CuInP2S6 has a FE transition with $T_{\mathrm{C}}$ = 320 K
BHD15b ; LYS16b , the number of layered van der Waals (vdW) ferroelectrics has
increased and includes $\alpha$-In2Se3 WLL18 , $\gamma$-InSe SJZ23 , SnS HKL20
, SnSe, SnTe CLL16 , 1T′-MoTe2 YLC19 , 1T′-MoS2 LCG22 , and 1T′-WTe2 FZP18 ;
SXS19 . It has also been ascertained that in BN SWC21 , MoS2 WCE22  WTe2 and
few other systems a new mechanism, other than lattice instability, is
responsible for FE, namely the relative sliding of the vdW stacked layers,
called sliding or moiré FE YWL18 ; WL21 ; LWW23 ; WGH23  and can even be
observed in multilayers of monoatomic graphene YDG23 . The phenomenon was
first predicted for BN and mixed BN/graphene bilayers and other vdW bilayers
LW17  and is based on the charge transfer between the facing atoms in the two
layers, which induces an out-of-plane polarization. The interest in these
ferroelectric mono and bi-layers is great, in view of possible applications in
nanoelectronics PG22 ; LWW23 , for example more miniaturized FeRAMs integrable
with Si or other materials thanks to the vdW coupling. Especially the sliding
or Moiré FE offers unique advantages of robust out-of-plane polarization WL21
.

MXenes are other 2D layered materials with chemical formula Mn+1CnTx (M
standing for a transition metal, X standing for C or N, and Tx is surface
functional groups). In the MXene structure, the surface groups Tx (usually –F,
=O, –OH), inherited from the etching environment, are covalently bound to the
transition metal and weakly bound via van der Waals bonds to terminations of
neighboring sheets. Such structures are being extensively studied for a
multitude of possible applications in important sectors BAB21 , such as
electrodes in Li/Na batteries LLW23 , intermediate layers of perovskite solar
cells APP19 , flexible and nano-electronics HG18 , soft robotics WGT23 ,
catalysis and many others BAB21 ; LLW23 . However, no ferroelectricity has
been found up to now in MXenes, but only theoretically predicted in Sc2CO2
CMS17 . On the other hand, Ti3C2Tx monolayers have been demonstrated to be
piezoelectric TJS21 ; TSC22 . Piezoelectricity, namely an induced strain
proportional to an applied electric field or polarization induced by stress,
is a consequence of FE, but is also possible in non-ferroelectrics lacking
spontaneous polarization, if their lattice belongs to certain non-
centrosymmetric classes. This is the case of bi-atomic hexagonal layers and of
Ti3C2Tx TJS21 .

It must be said that the theoretical predictions of 2D FE are more numerous
than the experimental verifications, which are rather challenging. In fact,
these types of FE are probed on mono-, bi- or tri-layer nanoflakes with
techniques such as Piezo-Force Microscopy (PFM) or conductive AFM.

We will show that these types of FE can be probed also with the simple
macroscopic method of measuring the elastic anomaly associated with the FE
transition on thick films of the layered material. We present the first
measurements of the complex Young’s modulus versus temperature of self-
standing thick films of the Ti3C2Tx MXene, which exhibit a clear and robust
nearly second order phase transition at $\simeq 350$ K, and it is argued that,
in spite of the good electrical conductivity, the transition should be
ferroelectric, likely of the sliding type.

Ti3C2Tx MXenes were synthesized by selective chemical etching of the Al layer
from a parent Ti3AlC2 powder precursor using the minimally intensive layer
delamination approach LAL16 . To produce a thick Ti3C2Tx film, a suspension of
well-delaminated flakes was vacuum filtered through a nitrocellulose membrane
with an average pore size of 0.22 $\mu$m. After drying, deposited layers were
detached from the membrane, and a rigid thick free-standing MXene film was
obtained. The film thickness was controlled by the concentration and the
amount of filtered suspension.

The complex Young’s modulus $E=$ $E^{\prime}+iE^{\prime\prime}$ was measured
in an apparatus where a bar-shaped sample is suspended on thin thermocouple
wires in high vacuum ($<10^{-5}$ mbar), and electrostatically excited on its
free flexural resonant modes CDC09 . The resonance frequencies are
$f\propto\sqrt{E}$ NB72 , so that the temperature dependence of $E$ is
evaluated as $E\left(T\right)/E_{0}=$
$\left[f\left(T\right)/f_{0}\right]^{2}$. The elastic energy loss coefficient,
$Q^{-1}=$ $E^{\prime\prime}/E^{\prime}$, is measured from the decay of the
free oscillations or the width of the resonance peak. The measurements are
made well within the linear elastic limit. Details on some of the experimental
difficulties encountered and their solutions can be found in the Supplemental
Material (SM). The measurements reported here were made on a strip 25.5 mm
long, 3 mm wide and $\left(73\pm 4\right)$ $\mu$m thick, with the first three
free flexural resonances clearly visible. In order to consolidate the film
avoiding warping and bubbles (see SM), the sample was put between two alumina
slabs and heated at 1.5 ∘C/min up to 700 ∘C in $10^{-6}$ mbar, kept 1 h and
furnace cooled. The film became flat and more brittle. Thermal annealing has
been reported to improve electrical conductivity of MXene films and we
observed the same trend in the change in Young’s modulus: after annealing it
increased from 11.9 GPa to 44 GPa, presumably due to the loss of intercalated
species.

The anelastic spectra were measured also using a Perkin-Elmer Diamond DMA in
the three point bending mode, where flexural vibrations are forced at fixed
frequencies in the range 0.1–10 Hz in Ar atmosphere (see SM). The sample were
also characterized before and after the anelastic measurements by XRD, and
Raman spectroscopy (see SM).

Figure 1 presents the Young’s modulus $E$ and elastic energy loss $Q^{-1}$
measured during the 1st and 3rd cooling runs (empty symbols), and subsequent
heating (filled symbols). In the last case, in addition to the fundamental
also the 2nd and 3rd flexural modes were excited at higher frequencies. Until
the sample was maintained in vacuum, the measurements were well repeatable,
with some hysteresis between heating and cooling above room temperature. The
main result is the nearly frequency independent negative step of $E$ and
positive step of $Q^{-1}$ below $T_{\mathrm{C}}\simeq$ 350 K. The step in $E$
is preceded by a precursor softening extending tens of kelvin above
$T_{\mathrm{C}}$. This is the typical signature of a phase transition whose
squared order parameter is linearly coupled with strain Reh73 ; CS98 .

Figure 1: Normalized Young’s modulus and elastic energy loss measured during
the 4th heating (filled symbols), exciting the 1st, 2nd and 3rd flexural
modes. The empty symbols are the 1st and 3rd coolings.

Above $T_{\mathrm{C}}$ the Young’s modulus exhibits some frequency dispersion,
that may be at least partly accounted for by the intense relaxation process
labeled P2. In fact, this broad peak shifts to higher temperature with
increase of frequency, indicating thermal activation of the spectrum of
characteristic times and must be accompanied by a decrease of the real part,
according to the Debye formula $\Delta
E\propto\left(1+i\omega\tau\right)^{-1}$ in the case of a single relaxation
time $\tau$ ($\omega=2\pi f$). There is also a minor peak in $Q^{-1}$, labeled
P1, too small to produce visible effects on $E$.

These measurements have been repeated several times, to check their
reproducibility also after exposure to air or immersion in water, as described
in the SM. In brief, it was found that the anelastic spectrum was reproduced
even when measured in 700 mbar water vapor up to 430 K, with little or no
intercalation of H2O. One day immersion in water resulted in intercalation of
$\sim 0.3$ H2O per formula unit, softening the modulus and strongly depressing
the elastic anomaly at $T_{\mathrm{C}}$. Outdiffusion of water in vacuum was
fast above 430 K. The cycle of experiments was concluded with heating up to
890 K, after which the original $Q^{-1}$ curve was completely recovered, but
$E$ remained $\sim 4\%$ softer and the step amplitude was smaller than
originally (curve 12 in Figs. 2 and S1). This was probably also due to surface
oxidation of the film, revealed by Raman spectroscopy after completing the
anelastic measurements.

The DMA measurements substantially confirm all the previous ones with the
resonance method, and provide additional information on the relatively slow
dynamics characterizing the transition. A strip of as-prepared material had at
room temperature a Young’s modulus as low as 4 GPa, which increased up to 25
GPa after warming up to 495 K (not shown), confirming the consolidation effect
of heating the as-deposited MXene sheet in vacuum or inert atmosphere. Figure
2 presents a DMA run from 200 to 810 K at 5 K/min and 0.1, 1, 10 Hz on the
same sample that had been measured with the resonance method. Two months
passed between the two sets of measurements. For comparison, the first and
last runs with the resonance method are also shown, with the same labeling of
Fig. S1. The major result is the confirmation of the steplike softening below
350 K. The magnitude of the modulus measured with the two methods is also in
fair agreement, considering that the $E\left(T\right)$ curves are quite
reproducible until the sample is measured and continuously kept in HV, but
otherwise they can vary considerably.

Figure 2: Young’s modulus and elastic energy loss measured with DMA at three
frequencies heating at 5 K/min. The previous first and last runs with the
resonance method are also shown, numbering the temperature scans in the same
manner as in Fig. S1.

The DMA measurements put in evidence a progressive stiffening of $E$ with
increase of frequency above $T_{\mathrm{C}}$ and a broad step around 700 K,
that had appeared also in the last resonance heating (curve 11).

Figure 3: Frequency dependence of the step in the Young’s modulus at the phase
transition. The resonance data are multiplied by 1.097.

More interesting is the frequency dispersion at the phase transition, put in
evidence in Fig. 3. The last heating with the resonance method (curve 11 of
Fig. 2) is multiplied by 1.097 for clarity. A 10% difference in the magnitude
of the Young’s moduli measured with the two methods may be due to imperfect
shape and homogeneity of the sample and is well within the 20% estimated
absolute accuracy of the DMA method SKT12 . In addition, it may be partly due
to the variability of the modulus following exposure to air and high
temperature cycles.

XRD and Raman spectra were measured again after the above experiments, as
described in the SM. The XRD spectra did not show any new peak, excluding
decomposition and formation of new phases within the bulk of the film, but
were shifted, indicating a decrease of the $c$ lattice parameter from 26.35 to
20.22 Å . This is due to the loss of the initial intercalants. The Raman
spectra, probing only the surface, exhibited new peaks attributable to TiO2
(atanase), absent in the XRD spectrum, indicating that surface oxidation
occurred.

The major result of these anelastic measurements is the presence of a
previously unnoticed nearly second order phase transition at
$T_{\mathrm{C}}\simeq 350$ K.

The elastic anomaly consists of a negative step of $\sim 4\%$ in the Young’s
modulus $E\ $ below $T_{\mathrm{C}}$, accompanied by a positive step in the
elastic energy loss $Q^{-1}$ and preceded by precursor softening extending
tens of kelvin above $T_{\mathrm{C}}$. This type of anomaly is characteristic
of a phase transition whose order parameter (OP) $P$ is coupled with strain
$\epsilon$ through a term $\propto\epsilon P^{2}$ in the free energy Reh73 .
This is the case of a ferroelectric transition where $P$ is the electric
polarization, but also of a (anti)ferromagnetic or antiferrodistortive
transition (like octahedral tilting in perovskites KSS00 ).

A magnetic transition can be excluded, since MXenes are mainly Pauli
paramagnets. In some cases the magnetization of Ti3C2Tx is very small down to
60 K, where a small kink may indicate a paramagnetic-to-antiferromagnetic
transition JWM22 .

Regarding the possibility of an antiferrodistortive transition in vdW solids,
we know of only two possibilities in layered organic-inorganic halide
perovskites. One is coordinated Jahn-Teller distortions of the BX6 octahedra
SJK20 , but this is not possible with Ti${}^{4}+$. The other is tilting of the
octahedra NL21 . This type of transition is common in perovskites AMX3 Gla72 ,
with short strong M-X bonds and longer weaker A-X bonds. When the network of
rigid corner-sharing octahedra MX6, sharing only their vertices X, cannot
follow the thermal contraction of the more anharmonic A-X bonds, the rigid
octahedra tilt in order to fit into the smaller lattice, giving rise to the
antiferrodistortive transition. In Ti3C2Tx there are only Ti-C bonds, and the
network of edge sharing CTi6 octahedra HLH17  has neither the structural
flexibility nor a driving force for an antiferrodistortive instability.
Indeed, deviations from the in-plane hexagonal structure have never been
reported, neither with traditional X-ray diffraction nor with its pair
distribution function analysis SBN14 .

Therefore, the remaining possibility is that the transition is ferroelectric.
Indeed, the elastic anomaly in Ti3C2Tx is very similar to that found in
classical ferroelectrics, for example in BaTiO3-δ, observable both in the
insulating state with $\delta=0$ Cor18  and when the material is made metallic
by introducing O vacancies CTC19 . The similarity is not limited to the steps
in elastic modulus and damping, but includes a precursor softening extending
at least tens of kelvin above $T_{\mathrm{C}}$ Cor18 ; CTS23 . The major
difference between the elastic anomalies in Ti3C2Tx and BaTiO3-δ is the
magnitude of the softening, which is up to $50\%$ in BaTiO3 and $\sim 4\%$ in
Ti3C2Tx.

The ferroelectric nature of the transition, however, cannot be ascertained
with the usual electric methods, since Ti3C2Tx is a good electric conductor
and any applied electric field is shielded by the free charge carriers. There
are only few known metallic ferroelectrics, among which is the vdW layered
WTe2 SXS19 . Probing FE in vdW layered materials is achieved through
sophisticated experiments on mono-, bi- and tri-layer nanoflakes, but the task
is particularly challenging in the case of electrically conducting materials.
The present results show that the possible ferroelectric transition in Ti3C2Tx
can be probed also with a macroscopic measurement like the temperature
dependence of the complex elastic modulus of a self-standing thick film. Not
only it is not necessary to probe the polarization switching at the atomic
level, but the good electric conductivity of the sample, which hinders the
transition to the usual electrical methods, does not affect the elastic
properties, which therefore reveal the elastic anomaly from the coupling
between strain and polarization.

As far as the elastic anomaly is concerned, FE in Ti3C2Tx may be due to the
usual lowering of the symmetry of the paraelectric phase, like Ti off-
centering in BaTiO3 or vertical shifting of the intermediate Se plane in the
vdW In2Se3 XZW18 . In the present case, a possibility would be vertical
shifting of the C planes with respect to the Ti planes. This type of polar
structure, however, has never been reported for Ti3C2Tx; rather, it has been
found that monolayers are piezoelectric with null polarization in the absence
of strain TJS21 . This corresponds to intrinsic piezoelectricity rather than
switchable spontaneous polarization within a monolayer.

Ti3C2Tx presents similarities with bilayer WTe2, which is both semi-metallic
and ferroelectric with $T_{\mathrm{C}}\simeq 350$ K FZP18 ; YWL18 . The origin
of its FE has been identified with the relative sliding between layers, which
forms out-of-plane electric dipoles, due to the transfer of electronic charge
between the facing atoms. The fact that the electric dipoles are formed
between pairs of layers and approximately perpendicular to them explains why
the polarization is not completely screened by the in-plane conductivity YWL18
. Relative sliding of two facing monolayers can switch the polarization over
an extended area without requiring atomic displacements within the monolayers.
A clear indication that the transition is due to an interlayer mechanism, and
not intrinsic of each layer, is the fact that it is not observed in the as-
prepared state, abundantly intercalated, and disappears by intercalating
water.

The Curie temperature in sliding FE is well above room temperature, even
though the sliding barrier is of $\lesssim$ meV YWL18 . This is very
convenient for applications, since it combines low switching fields with
robust ferroelectricity, and has been explained in the framework of continuum
electromechanics as a consequence of the large in-plane rigidity of the layers
TB23 . For Ti3C2Tx there would be the complication that the facing species are
not the ordered planes of surface Ti atoms but the –OH, =O and –F
terminations. We are not aware of reports of sliding FE between non uniformly
terminated layers, but will discuss the elastic anomaly in this light, though
the analysis will be valid for any type of FE. In fact, we can exploit the
fact that in semiconducting bilayer WSe2, exhibiting sliding FE, the
spontaneous polarization $P\left(T\right)$ can be fitted with the usual Landau
free energy with terms up to the 6th power of $P$ LLL22b ,

$F=\frac{a}{2}\left(T-T_{\mathrm{C}}\right)P^{2}+\frac{B}{4}P^{4}+\frac{C}{6}P^{6},$
(1)

generally used for describing any first ($B<0$) or second ($B>0$) order
ferroelectric transition. Once the transition is phenomenologically described
in terms of Eq. (1), the elastic anomaly can be evaluated introducing a
suitable coupling between stress/strain and order parameter $P$. We disregard
a possible coupling of $P$ with the "intrinsic" piezoelectricity found in the
non-centrosymmetric monolayer of Ti3C2Tx TJS21 , with in-plane polarization
$p$, because $p$ and $P$ are approximately perpendicular to each other and
physically separated. In fact, $p$ is in-plane and confined by the good
electrical conductivity within the layers, while $P$ is approximately
perpendicular to and between the layers. In addition, while $p$ is due to the
atomic displacements within the layers, $P$ is considered to arise from
interlayer electronic transfer with minimal relative atomic displacements
within layers LWW23 .

In order to evaluate the effect of the transition on the elastic modulus, we
include in the free energy the elastic and the mixed terms containing both $P$
and strain $\varepsilon$ or, considering the Gibbs free energy
$G=F-\sigma\varepsilon$, the terms containing $P$ and stress $\sigma$. Without
entering in the details of the mechanism causing the polarization, we can
safely include the electrostrictive coupling $Q\sigma P^{2}$, always allowed
LG77 ; Reh73 , while the piezoelectric coupling $d\sigma p$ is with the
intrinsic $p$, so that

$\displaystyle G$ $\displaystyle=$
$\displaystyle\frac{a}{2}\left(T-T_{\mathrm{C}}\right)P^{2}+\frac{B}{4}P^{4}+\frac{C}{6}P^{6}-\frac{S^{0}}{2}\sigma^{2}-Q\sigma
P^{2}+$ (3) $\displaystyle+\frac{1}{2\chi_{p}}p^{2}-d\sigma p,$

where the susceptibility $\chi_{p}$ of $p$, nearly independent of $T$, is
included.

The elastic compliance $S=1/E$ is calculated as usual Reh73 ; SL98 ; CCT16  as
$S=$ $d\varepsilon/d\sigma$ with $\varepsilon=$ $-\partial G/\partial\sigma$
and one obtains (see SM)

$\begin{array}[]{c}S\left(T>T_{\mathrm{C}}\right)=S^{0}+\chi_{p}d^{2}\\\
S\left(T<T_{\mathrm{C}}\right)=S^{0}+\chi_{p}d^{2}+\Delta s\\\ \Delta
S=\frac{2Q^{2}}{B\sqrt{\frac{T_{\mathrm{C}}+\Delta T-T}{\Delta
T}}},~{}~{}\Delta T=\frac{B^{2}}{4aC}\end{array}$ (4)

The constant term $\chi_{p}d^{2}$ is the softening due to the intrinsic
piezoelectricity, and just reorganizes the background compliance $S^{0}$. The
coupling to $P$ does not produce any effect in the paraelectric phase
($T>T_{\mathrm{C}}$ for $B<0$ and up to $T_{\mathrm{C}}+\Delta T$ for $B<0$),
where $P$ is null apart from fluctuations, and produces a steplike softening
$\Delta S$ in the ferroelectric phase. The step is constant of magnitude
$2Q^{2}/B$ if $C=0$ and acquires a cusped shape for $C>0$. This type of
elastic anomaly is the same for any OP that is coupled quadratically to
stress/strain, including magnetization and the rotation angle in
antiferrodistortive transition, as in SrTiO3 KSS00 . The softening is due to
the relaxation of the order parameter upon application of stress, with
consequent additional strain. In the FE case, the softening is of
piezoelectric origin CCT16 .

In view of the marked dependence on frequency shown in Fig. 3, the time
dependence of the response of $P$ to stress should be taken into account. This
is usually done in the framework of the Landau-Khalatnikov theory, assuming
that $\dot{P}=-\left(P-P_{0}\right)/\left(\chi L\right)$, where $L$ is a
nearly temperature independent kinetic coefficient and $\chi$ the
susceptibility $\sim\left|T-T_{\mathrm{C}}\right|^{-1}$, yielding a relaxation
time $\tau=\tau_{0}/\left(T_{\mathrm{C}}-T\right)$. Introducing this time
dependence of $P$ into the elastic response to a periodic excitation with
angular frequency $\omega$ yields a complex compliance $\Delta
S/\left(1+i\omega\tau\right)$ KSS00 ; YVK06b  and therefore a frequency
dependence of the real part

$\displaystyle\Delta S\left(\omega\right)$ $\displaystyle=$
$\displaystyle\Delta S/\left[1+\left(\omega\tau\right)^{2}\right]$ (5)
$\displaystyle\tau$ $\displaystyle=$
$\displaystyle\tau_{0}/\left(T_{\mathrm{C}}-T\right)$

Figure 4: Fit of the elastic anomaly presented in Fig. 1, measured at 523 Hz
during the 3rd cooling, using Eq. (4) with the parameters $\Theta=$ 490 K;
$T_{\mathrm{C}}=$ 354 K, $\Delta T=$ 70 K; $\Delta s\times E_{0}=$ 0.039,
$\tau_{0}=$ $7\times 10^{-4}$ s. The red line is calculated at 1 Hz, The
dashed line is the background $E^{0}$.

In order to fit the anomaly of the Young’s modulus $E$ with an expression like
Eqs. (4–5), it is necessary to establish the background variation of
$E\left(T\right)$. This usually is a linear stiffening during cooling, due to
anharmonic effects, with quantum saturation below some temperature $\Theta$,
and can be written in terms of the compliance as SWT91

$1/E^{0}=S^{0}=S_{0}+S_{1}/\tanh\left(\Theta/T\right)~{}.$ (6)

However, additional phenomena affect $E\left(T\right)$, as apparent in Figs. 1
and 2, and therefore a fit cannot be extended above 500 K, where a smooth
frequency dependent kink is evident in Fig. 1. The low frequency experiment,
spanning two decades in $f$, actually shows dependence of the background
$E^{0}$ down to room temperature and is also noisier than in the resonance
experiments. For this reason, we present a fit of the elastic anomaly measured
during the 3rd cooling run in resonance, when the sample had not yet been
subjected to the various treatments with water. The best fit with Eqs. (4–6)
is found with the parameters indicated in the caption to Fig. 4 (blue line).
The red line is calculated at 1 Hz, in order to show the effect of the
relaxation of the order parameter: increasing frequency, the minimum shifts to
temperatures lower than $T_{\mathrm{C}}$ and its magnitude decreases. This is
of the same type as observed in Fig. 3, though its full magnitude cannot be
reproduced with Eq. (5). It is possible that a contribution to softening below
the transition is also given by the motion of domain walls, which is typically
larger at lower frequency KSS00 , and may well explain the larger magnitude of
the step at lower frequency.

As anticipated, Eq. (4) does not contain the contribution of fluctuations
above $T_{\mathrm{C}}$, and therefore cannot reproduce the evident precursor
softening. Precursor softening is found commonly and in the classical
ferroelectric BaTiO3 it extends even to hundreds of kelvin above
$T_{\mathrm{C}}$ CTS23 . It is notable that the Curie temperatures reported up
to now for sliding ferroelectrics, WSe2 LLL22b  and WTe2 FZP18 ; YWL18 , are
$T_{\mathrm{C}}\simeq 350$ K as in our case.

These measurements demonstrate that nanoscale phenomena like FE in mono and
bi-layers of vdW solids, generally studied with advanced techniques at the
nanoscale, can be probed with mechanical spectroscopy on thick films. Notice
that it is not necessary that the direction of the interlayer polarization is
coherent along different pairs of layers, since strain is coupled to $P^{2}$
and strains from opposite polarizations do not cancel with each other.

The dissipation $Q^{-1}\left(T\right)$ of Ti3C2Tx presents a positive step
below $T_{\mathrm{C}}$, which is the normal behaviour from the interaction
between stress and the walls between the domains formed in the low-temperature
phase CS98 ; KSS00 ; Cor18 . With increase of temperature, the dissipation is
dominated by intense relaxation processes with a broad spectrum of
characteristic times toward high temperature (P2 in Fig. 1). Among the
possible contributions to these losses are relative sliding of the flakes,
hopping of the residual intercalated species, diffusion of the terminating
molecules, and motion of point and extended defects within the monolayers.

The magnitude of the Young’s modulus of the film consolidated in high vacuum
at 970 K, $E=$ $40-50$ GPa, is larger than that achieved in Ti3C2Tx densified
by addition of sodium carboxymethyl cellulose and sodium borate (28 GPa)
WLC21b  or carboxymethylated cellulose nanofibrils (42 GPa) TVR19 , but the
films are fragile and $E$ is still one order of magnitude smaller than the 330
GPa found with AFM on single flakes LLA18 . The large discrepancy between the
in-plane Young’s modulus of single and aggregated flakes must be due to their
easy relative sliding.

We measured for the first time the temperature dependence of the complex
Young’s modulus $E$ of the Ti3C2Tx MXene in the form of self-standing thick
films. In the as-prepared state the films have a very low $E\sim 4-10$ GPa,
that may be increased up to $40-50$ GPa by annealing in high vacuum at 970 K,
so reducing the amount of intercalated species. This is still lower than 330
GPa of a single flake, presumably due to an easy relative sliding of the
flakes, but allows reproducible anelastic spectra to be measured up to 900 K.
The major feature is a phase transition below $T_{\mathrm{C}}\sim 350$ K,
producing a steplike softening of $\sim 4\%$ of $E$ and increase of
dissipation, which is robust against thermal treatments up to 900 K in vacuum
or inert gas. The nature of the phase transition is discussed and it is
concluded that it should be ferroelectric, based on the fact that the
available structural and magnetic measurements exclude deviations from the
hexagonal structure and magnetism at room temperature. Particular attention
has been devoted to the mechanism of sliding ferroelectricity, where sliding
of the layers with respect to each other induces charge transfer between
atoms/molecules belonging to the facing layers and hence to interlayer
polarization. Notably, $T_{\mathrm{C}}\simeq 350$ K is the same of other two
known sliding ferroelectrics, WSe2 has LLL22b  and WTe2 FZP18 ; YWL18 . This
would add a new feature and perspectives of applications to the already
numerous ones of Ti3C2Tx MXene. In this view, the simple fact that this
transition is slightly above room temperature calls for further investigations
of its nature.

FC and ADC acknowledge the precious technical assistance of M.P. Latino (CNR-
ISM).

## References

  * (1) Sung Hyuk Park, Jae Young Kim, Jae Yong Song, and Ho Won Jang.  Overcoming Size Effects in Ferroelectric Thin Films.  Adv. Phys. Res., 2:2200096, 2023.
  * (2) Ruixiang Fei, Wei Kang, and Li Yang.  Ferroelectricity and Phase Transitions in Monolayer Group-IV Monochalcogenides.  Phys. Rev. Lett., 117:097601, 2016.
  * (3) A. Belianinov, Q. He, A. Dziaugys, P. Maksymovych, E. Eliseev, A. Borisevich, A. Morozovska, J. Banys, Y. Vysochanskii, and S. V. Kalinin.  CuInP2S6 Room Temperature Layered Ferroelectric.  Nano Lett., 15:3808, 2015.
  * (4) Fucai Liu, Lu You, Kyle L. Seyler, Xiaobao Li, Peng Yu, Junhao Lin, Xuewen Wang, Jiadong Zhou, Hong Wang, Haiyong He, Sokrates T. Pantelides, Wu Zhou, Pradeep Sharma, Xiaodong Xu, Pulickel M. Ajayan, Junling Wang, and Zheng Liu.  Room-temperature ferroelectricity in CuInP2S6 ultrathin flakes.  Nat. Commun., 7:12357, 2016.
  * (5) Siyuan Wan, Yue Li, Wei Li, Xiaoyu Mao, Wenguang Zhu, and Hualing Zeng.  Room-temperature ferroelectricity and a switchable diode effect in two-dimensional $\alpha$-In2Se3 thin layers.  Nanoscale, 10:14885, 2018.
  * (6) Fengrui Sui, Min Jin, Yuanyuan Zhang, Ruijuan Qi, Yu-Ning Wu, Rong Huang, Fangyu Yue, and Junhao Chu.  Sliding ferroelectricity in van der Waals layered $\gamma$-InSe semiconductor.  Nat. Commun., page 36, 2023.
  * (7) Naoki Higashitarumizu, Hayami Kawamoto, Chien-Ju Lee, Bo-Han Lin, Fu-Hsien Chu, Itsuki Yonemori, Tomonori Nishimura, Katsunori Wakabayashi, Wen-Hao Chang, and Kosuke Nagashio.  Purely in-plane ferroelectricity in monolayer SnS at room temperature.  Nat. Commun., 11:2428, 2020.
  * (8) Kai Chang, Junwei Liu, Haicheng Lin, Na Wang, Kun Zhao, Anmin Zhang, Feng Jin, Yong Zhong, Xiaopeng Hu, Wenhui Duan, Qingming Zhang, Liang Fu, Qi-Kun Xue, Xi Chen, , and Shuai-Hua Ji.  Discovery of robust in-plane ferroelectricity in atomic-thick SnTe.  Science, 353:274, 2016.
  * (9) Shuoguo Yuan, Xin Luo, Hung Lit Chan, Chengcheng Xiao, Yawei Dai, Maohai Xie, and Jianhua Hao.  Room-temperature ferroelectricity in MoTe2 down to the atomic monolayer limit.  Nat. Commun., 10:1775, 2019.
  * (10) A. Lipatov, P. Chaudhary, Z. Guan, H. Lu, G. Li, O. Crégut, K. D. Dorkenoo, R. Proksch, S. Cherifi-Hertel, D. F. Shao, E. Y. Tsymbal, J. Íñiguez, A. Sinitskii, and A. Gruverman.  Direct observation of ferroelectricity in two-dimensional MoS2.  npj 2D Mater. Appl., 6:18, 2022.
  * (11) Z. Fei, W. Zhao, T. A. Palomaki, B. Sun, M. K. Miller, Z. Zhao, J. Yan, X. Xu, and D. H. Cobden.  Ferroelectric switching of a two-dimensional metal.  Nature, 560:336, 2018.
  * (12) Pankaj Sharma, Fei-Xiang Xiang, Ding-Fu Shao, Dawei Zhang, Evgeny Y. Tsymbal, Alex R. Hamilton, and Jan Seidel and.  A room-temperature ferroelectric semimetal.  Sci. Adv., 5:eaax5080, 2019.
  * (13) M. Vizner Stern, Y. Waschitz, W. Cao, I. Nevo, K. Watanabe, T. Taniguchi, E. Sela, M. Urbakh, O. Hod, and M. Ben Shalom.  Interfacial ferroelectricity by van der Waals sliding.  Science, 372:1462, 2021.
  * (14) Astrid Weston, Eli G. Castanon, Vladimir Enaldiev, Fábio Ferreira, Shubhadeep Bhattacharjee, Shuigang Xu, Héctor Corte-León, Zefei Wu, Nicholas Clark, Alex Summerfield, Teruo Hashimoto, Yunze Gao, Wendong Wang, Matthew Hamer, Harriet Read, Laura Fumagalli, Andrey V. Kretinin, Sarah J. Haigh, Olga Kazakova, A. K. Geim, Vladimir I. Falko, and Roman Gorbachev.  Interfacial ferroelectricity in marginally twisted 2D semiconductors.  Nat. Nanotechnol., 17:390, 2022.
  * (15) Qing Yang, Menghao Wu, and Ju Li.  Origin of Two-Dimensional Vertical Ferroelectricity in WTe2 Bilayer and Multilayer.  J. Phys. Chem. Lett., 9:7160, 2018.
  * (16) Menghao Wu and Ju Li.  Sliding ferroelectricity in 2D van der Waals materials: Related physics and future opportunities.  PNAS, 118:e2115703118, 2021.
  * (17) Shuhui Li, Feng Wang, Yanrong Wang, Jia Yang, Xinyuan Wang, Xueying Zhan, Jun He, and Zhenxing Wang.  Van der Waals Ferroelectrics: Theories, Materials and Device Applications.  Advan. Mater., page 2301472, 2023.
  * (18) Zhe Wang, Zhigang Gui, and Li Huang.  Sliding ferroelectricity in bilayer honeycomb structures: A first-principles study.  Phys. Rev. B, 107:035426, 2023.
  * (19) Liu Yang, Shiping Ding, Jinhua Gao, and Menghao Wu.  Atypical Sliding and Moiré Ferroelectricity in Pure Multilayer Graphene.  Phys. Rev. Lett., 131:096801, 2023.
  * (20) Lei Li and Menghao Wu.  Binary Compound Bilayer and Multilayer with Vertical Polarizations: Two-Dimensional Ferroelectrics, Multiferroics, and Nanogenerators.  ACS Nano, 11:6382, 2017.
  * (21) Jacob Parker and Yi Gu.  van der Waals ferroelectrics: Progress and an outlook for future research directions.  J. Appl. Phys., 132:160901, 2022.
  * (22) Anha Bhat, Shoaib Anwer, Kiesar Sideeq Bhat, M. Infas H. Mohideen, Kin Liao, and Ahsanulhaq Qurashi.  Prospects challenges and stability of 2D MXenes for clean energy conversion and storage applications.  npj 2D Mater. Appl., 5:61, 2021.
  * (23) Guohao Li, Shuhan Lian, Jie Wang, Guanshun Xie, Nan Zhang, and Xiuqiang Xie.  Surface chemistry engineering and the applications of MXenes.  J. Materiomics, 2023.
  * (24) A. Agresti, A. Pazniak, S. Pescetelli, A. Di Vito, D. Rossi, A. Pecchia, M. Auf der Maur, A. Liedl, R. Larciprete, Denis V. Kuznetsov, D. Saranin, and A. Di Carlo.  Titanium-carbide MXenes for work function and interface engineering in perovskite solar cells.  Nat. Mater., 18:1228, 2019.
  * (25) Kanit Hantanasirisakul and Yury Gogotsi.  Electronic and Optical Properties of 2D Transition Metal Carbides and Nitrides (MXenes).  Adv. Mater., 30:1804779, 2018.
  * (26) Yizhou Wang, Tianchao Guo, Zhengnan Tian, Lin Shi, Sharat C. Barman, and Husam N. Alshareef.  MXenes for soft robotics.  Matter, 6:2807, 2023.
  * (27) Anand Chandrasekaran, Avanish Mishra, and Abhishek Kumar Singh.  Ferroelectricity, Antiferroelectricity, and Ultrathin 2D Electron/Hole Gas in Multifunctional Monolayer MXene.  Nano Lett., 4:3290, 2017.
  * (28) Dongchen Tan, Chengming Jiang, Nan Sun, Jijie Huang, Zhe Zhang, Qingxiao Zhang, Jingyuan Bu, Sheng Bi, Qinglei Guo, and Jinhui Song.  Piezoelectricity in monolayer MXene for nanogenerators and piezotronics.  Nano Energy, 90:106528, 2021.
  * (29) Dongchen Tan, Nan Sun, Long Chen, Jingyuan Bu, and Chengming Jiang.  Piezoelectricity in Monolayer and Multilayer Ti3C2Tx MXenes: Implications for Piezoelectric Devices.  ACS Appl. Nano Mater., 5:1034, 2022.
  * (30) A. Lipatov, M. Alhabeb, M. R. Lukatskaya, A. Boson, Y. Gogotsi, and A. Sinitskii.  Effect of Synthesis on Quality, Electronic Properties and Environmental Stability of Individual Monolayer Ti3C2 MXene Flakes.  Adv. Electron. Mater., 2:1600255, 2016.
  * (31) F. Cordero, L. Dalla Bella, F. Corvasce, P. M. Latino, and A. Morbidini.  An insert for anelastic spectroscopy measurements from 80 K to 1100 K.  Meas. Sci. Technol., 20:015702, 2009.
  * (32) A. S. Nowick and B. S. Berry.  Anelastic Relaxation in Crystalline Solids.  Academic Press, New York, 1972.
  * (33) W. Rehwald.  The Study of Structural Phase Transitions by Means of Ultrasonic Experiments.  Adv. Phys., 22:721, 1973.
  * (34) M. A. Carpenter and E. H. K. Salje.  Elastic anomalies in minerals due to structural phase transitions.  Eur. J. Mineral., 10:693–812, 1998.
  * (35) W. Schranz, H. Kabelka, and A. Tröster.  Superelastic softening of ferroelastic multidomain crystals.  Ferroelectrics, 426:242, 2012.
  * (36) A. V. Kityk, W. Schranz, P. Sondergeld, D. Havlik, E. K. H. Salje, and J. F. Scott.  Low-frequency superelasticity and nonlinear elastic behavior of SrTiO3 crystals.  Phys. Rev. B, 61:946, 2000.
  * (37) Haolin Jin, Kai Wang, Zhongquan Mao, Lingyun Tang, Jiang Zhang, and Xi Chen.  The structural, magnetic, Raman and electrical transport properties of Mn intercalated Ti3C2Tx.  J. Phys.: Condens. Matter, 34:015701, 2022.
  * (38) Dipayan Sen, Gour Jana, Nitin Kaushal, Anamitra Mukherjee, and Tanusri Saha-Dasgupta.  Intrinsic ferromagnetism in atomically thin two-dimensional organic-inorganic van der Waals crystals.  Phys. Rev. B, 102:054411, 2020.
  * (39) J. A. McNulty and P. Lightfoot.  Structural chemistry of layered lead halide perovskites containing single octahedral layers.  IUCrJ, 8:485, 2021.
  * (40) A. M. Glazer.  The classification of tilted octahedra in perovskites.  Acta Cryst. B, 28:3384, 1972.
  * (41) Tao Hu, Zhaojin Li, Minmin Hu, Jiemin Wang, Qingmiao Hu, Qingzhong Li, and Xiaohui Wang.  Chemical Origin of Termination-Functionalized MXenes: Ti3C2T2 as a Case Study.  J. Phys. Chem. C, 121:19254, 2017.
  * (42) Chenyang Shi, Majid Beidaghi, Michael Naguib, Olha Mashtalir, Yury Gogotsi, and Simon J. L. Billinge.  Structure of Nanocrystalline Ti3C2 MXene Using Atomic Pair Distribution Function.  Phys. Rev. Lett., 112:125501, 2014.
  * (43) F. Cordero.  Quantitative evaluation of the piezoelectric response of unpoled ferroelectric ceramics from elastic and dielectric measurements: Tetragonal BaTiO3.  J. Appl. Phys., 123:094103, 2018.
  * (44) F. Cordero, F. Trequattrini, F. Craciun, H. T. Langhammer, D. A. B. Quiroga, and Jr. P. S. Silva.  Probing ferroelectricity in highly conducting materials through their elastic response: Persistence of ferroelectricity in metallic BaTiO3-d.  Phys. Rev. B, 99:064106, 2019.
  * (45) Francesco Cordero, Francesco Trequattrini, Paulo Sergio da Silva, Jr., Michel Venet, Oktay Aktas, and Ekhard K. H. Salje.  Elastic precursor effects during Ba1-xSrxTiO3 ferroelastic phase transitions.  Phys. Rev. Research, 5:013121, 2023.
  * (46) Jun Xiao, Hanyu Zhu, Ying Wang, Wei Feng, Yunxia Hu, Arvind Dasgupta, Yimo Han, Yuan Wang, David A. Muller, Lane W. Martin, PingAn Hu, and Xiang Zhang.  Intrinsic Two-Dimensional Ferroelectricity with Dipole Locking.  Phys. Rev. Lett., 120:227601, 2018.
  * (47) Ping Tang and Gerrit E.W. Bauer.  Sliding Phase Transition in Ferroelectric van der Waals Bilayers.  Phys. Rev. Lett., 130:176801, 2023.
  * (48) Yang Liu, Song Liu, Baichang Li, Won Jong Yoo, and James Hone.  Identifying the Transition Order in an Artificial Ferroelectric van der Waals Heterostructure.  Nano Lett., 22:1265, 2022.
  * (49) M. E. Lines and A. M. Glass.  Principles and Applications of Ferroelectrics and Related Materials.  Oxford University Press, Oxford, 1977.
  * (50) B. A. Strukov and A. P. Levanyuk.  Ferroelectric Phenomena in Crystals.  Springer, Heidelberg, 1998.
  * (51) F. Cordero, F. Craciun, F. Trequattrini, and C. Galassi.  Piezoelectric softening in ferroelectrics: ferroelectric versus antiferroelectric PbZr1-xTixO3.  Phys. Rev. B, 93:174111, 2016.
  * (52) R M. Yevych, Yu. M. Vysochanskii, M. M. Khoma, and S. I. Perechinskii.  Lattice instability at phase transitions near the Lifshitz point in proper monoclinic ferroelectrics.  J. Phys.: Condens. Matter, 18:4047–4064, 2006.
  * (53) E.K.H. Salje, B. Wruck, and H. Thomas.  Order-parameter saturation and low-temperature extension of Landau theory.  Z. Phys. B - Condensed Matter, 82:399, 1991.
  * (54) Sijie Wan, Xiang Li, Ying Chen, Nana Liu, Yi Du, Shixue Dou, Lei Jiang, and Qunfeng Cheng.  High-strength scalable MXene films through bridging-induced densification.  Science, 374:96, 2021.
  * (55) Weiqian Tian, Armin VahidMohammadi, Michael S. Reid, Zhen Wang, Liangqi Ouyang, Johan Erlandsson, Torbjörn Pettersson, Lars Wågberg, Majid Beidaghi, and Mahiar M. Hamedi.  Multifunctional Nanocomposites with High Strength and Capacitance Using 2D MXene and 1D Nanocellulose.  Adv. Mater., 31:1902877, 2019.
  * (56) A. Lipatov, H. Lu, M. Alhabeb, B. Anasori, A. Gruverman, Y. Gogotsi, and A. Sinitskii.  Elastic properties of 2D Ti3C2Tx MXene monolayers and bilayers.  Sci. Adv., 4:eaat0491, 2018.


# Gradient-guided Loss Masking for Neural Machine Translation

Xinyi Wang1, Ankur Bapna2, Melvin Johnson2, Orhan Firat2
1Language Technology Institute, Carnegie Mellon University
2Google Research
xinyiw1@cs.cmu.edu,{ankurbpn, melvinp<EMAIL_ADDRESS>
###### Abstract

To mitigate the negative effect of low quality training data on the
performance of neural machine translation models, most existing strategies
focus on filtering out harmful data before training starts. In this paper, we
explore strategies that dynamically optimize data usage during the training
process using the model’s gradients on a small set of clean data. At each
training step, our algorithm calculates the gradient alignment between the
training data and the clean data to mask out data with negative alignment. Our
method has a natural intuition: good training data should update the model
parameters in a similar direction as the clean data. Experiments on three WMT
language pairs show that our method brings significant improvement over strong
baselines, and the improvements are generalizable across test data from
different domains.

##  1 Introduction

The quality of training data has a large effect on the performance of neural
machine translation (NMT) models (Khayrallah and Koehn, 2018). Parallel
sentences crawled from the web and aligned from multilingual documents Esplà
et al. (2019) provide a large supply of training data to boost the performance
of NMT, but automatically extracted training examples often have noise that
could hurt the model performance. Designing a good strategy to utilize the
training data with varying quality levels is essential to improving the
performance of NMT models.

Prior methods mainly focus on filtering data before training starts (Moore and
Lewis, 2010; van der Wees et al., 2017; Wang et al., 2018). Junczys-Dowmunt
(2018) proposed selecting clean training data by using a heuristic threshold
on the perplexity difference between the forward and backward translation
models. Wang et al. (2018) propose a curriculum-based data selection strategy
using a small set of clean trusted data.

In this paper, we examine the problem of optimizing the training data usage
from a different angle. Instead of filtering data before the training, we
utilize the gradient information of the data during the training process to
guide the data usage. The intuition behind our method is natural: given a very
small set of clean or trusted data (which is generally easy to obtain in
practice), the preferred training data should have similar gradient direction
as the gradient of the clean data, so that updating the model parameters on
this training data would improve the model performance on the clean data.
Based on this intuition, we propose gradient-guided loss masking (GLMask),
which calculates the gradient alignment between each training example and the
clean data, and simply masks out the loss of the training examples that have
negative gradient alignments. GLMask is inspired by methods that optimize
training data usage using meta-learning. Ren et al. (2018) calculates a
weighting of noisy training data that minimizes the model loss on clean data
for image classification tasks. Wang et al. (2020b, a) optimizes the training
data distribution of multilingual data such that the model loss on the
development set is minimized. Our method, on the other hand, uses a simple
gradient alignment signal to determine the parallel sentence pairs or target
words to mask out during NMT training.

We test GLMask on the standard WMT English-German, English-Chinese and
English-French translation tasks, using WMT test sets from prior years as the
small clean data. GLMask brings significant improvements over strong baselines
on all three language pairs. We further evaluate the trained models on the
IWSLT test sets which are sampled from a different domain, and demonstrate
that GLMask also delivers improvements on these out of domain sets. This shows
that improvements from GLMask generalize to data from different domains,
beyond the domain of the clean data.

##  2 Method

To explain our method, we first provide a mathematical framework for the
problem it aims to address.

###  2.1 Problem with training data

To train a NMT model with parameters $\theta$ that translates from a source
language $S$ to a target language $T$, we want to find the optimal model
parameters $\theta^{*}$, which minimizes the loss function $\ell(x,y,\theta)$
over the true distribution of the parallel data from $S$-$T$, denoted by
$P(S,T)$. However, in practice, one usually only has access to a limited
number of parallel training sentences sampled from the training distribution
$P_{\text{train}}(x,y)$. The standard training approach finds the $\theta^{*}$
that minimizes the loss function $\ell(\cdot;\theta)$ over this training
distribution:

$\displaystyle J_{\text{train}}(\theta)=\mathbf{E}_{x,y\sim
P_{\text{train}}(x,y)}[\ell(x,y;\theta)]$ (1)

Problems arise when the training distribution $P_{\text{train}}(x,y)$, from
which we draw training sentences, has discrepancies with $P(S,T)$, the true
distribution of the parallel data from $S$-$T$. For example, some samples from
the training data might be noisy and might be detrimental to final model
performance on the clean data (Khayrallah and Koehn, 2018).

To remedy this training data discrepancy, one strategy is to collect a small
set of high quality data drawn from distribution $P_{\text{clean}}(x,y)$,
which is closer to the true data distribution $P(S,T)$, to guide training on
the large noisy training data. With the help of this clean data, we want to
train the model using data sampled from $P_{\text{train}}(x,y)$, while the
loss over the clean data is minimized:

$\displaystyle J_{\text{clean}}(\theta)=\mathbf{E}_{x,y\sim
P_{\text{clean}}(x,y)}[\ell(x,y;\theta)]$ (2)

###  2.2 Gradient-guided loss masking

We propose gradient-guided loss masking (GLMask), which uses gradient
information to mask out the training examples that can be harmful for
minimizing the model’s loss on the clean data. GLMask modifies the training
objective $J_{\text{train}}(\theta)$ such that optimizing the model parameter
over this objective also optimizes the objective over the small clean data
$J_{\text{clean}}(\theta)$. The intuition behind our approach is that a
training example $x,y\sim P_{\text{train}}(x,y)$ is more likely to minimize
the loss over the clean data, if the gradient of this training data is in the
same direction as the gradient of the clean data. Formally, we use the dot
product between these two gradients as their alignment

$\displaystyle
g(x,y)=\nabla_{\theta}\ell(x,y;\theta)^{\top}\cdot\nabla_{\theta}J_{\text{clean}}(\theta)$
(3)

A negative $g(x,y)$ indicates that the training example has a conflicting
gradient with the clean data, which might be detrimental for the model’s
performance on the clean data. To optimize the model objective on the clean
data, we can mask out the examples that have negative $g(x,y)$ with the clean
data. That is, we modify the training objective in Eq. 1 to

$\displaystyle J_{\text{train}}(\theta)=\mathbf{E}_{x,y\sim
P_{\text{train}}(x,y)}\left[m(x,y)\cdot\ell(x,y)\right]$ (4)

where $m(x,y)=\mathbbm{1}[g(x,y)>0]$. The masking term assigns a weight of 0
to examples that have negative gradient alignment with the clean data.

#### Implementation

The pseudo code for GLMask is in Alg. 1. Note that calculating the loss mask
at Alg. 1 requires calculating the gradient alignment of each training example
and the gradient of the clean data, which is potentially very expensive. We
use a technique that allows us to efficiently compute this value with only two
additional backward passes, which is supported by modern deep learning
libraries such as Tensorflow Abadi et al. (2016)111We use the technique
introduced here: https://j-towns.github.io/2017/06/12/A-new-trick.html. The
function in Tensorflow is provided in § A.1. In our preliminary experiments,
using GLMask at the later stage of training works as well as or even better
than using it from the beginning. Therefore, we use GLMask for the last $20\%$
of training steps, which further decreases the overall training overhead of
our method.

Input : Training corpus $\mathcal{D}_{\text{train}}$; clean data
$\mathcal{D}_{\text{clean}}$

Output : The converged model $\theta^{*}$

1

2while  _not converged_ do

3

$\triangleright$ Sample a batch of training data

4 $(x_{1},y_{1})...(x_{B},y_{B})\sim\mathcal{D}_{\text{train}}$

$\triangleright$ Sample a batch of clean data

5
$(x^{\prime}_{1},y^{\prime}_{1})...(x^{\prime}_{B},y^{\prime}_{B})\sim\mathcal{D}_{\text{clean}}$

$\triangleright$ Calculate data mask

6
$g^{\prime}\leftarrow\nabla_{\theta_{t}}\frac{1}{B}\sum_{i=1}^{B^{\prime}}\ell(x^{\prime}_{i},y^{\prime}_{i};\theta_{t})$

7 $m(x_{i},y_{i})$

8
$\leftarrow\mathbbm{1}(g^{\prime\top}\nabla_{\theta}\ell(x_{i},y_{i};\theta_{t})>0)\text{
for i in 1...B}$

$\triangleright$ Calculate masked objective

9
$g_{\text{train}}\leftarrow\nabla_{\theta_{t}}\frac{1}{B}\sum_{i=1}^{B}m(x_{i},y_{i})\ell(x_{i},y_{i};\theta_{t})$

10 $\theta_{t+1}\leftarrow\text{Update}(\theta_{t},g_{\text{train}})$

11 end while

Algorithm 1 Training with GLMask

##  3 Experiment

###  3.1 Dataset and setup

#### Data

We use parallel data from the WMT evaluation campaign. To verify the
effectiveness of our approach, we test it on three language pairs with varying
amounts of resources: English to German (en-de), English to Chinese (en-zh),
and English to French (en-fr).

The en-de experiments are conducted using the WMT’14 training data with about
4 million parallel sentences. We use newstest2013 as the validation set, and
newstest2014 as the test set. The en-zh experiments use the WMT’17 training
data with around 22 million parallel sentences. We use newsdev2017 as the
validation set and newstest2017 as the test set. The en-fr experiments use the
WMT’14 training data with about 40 million parallel sentences. We use
newstest2013 as validation set and newstest2014 as the test set. We use
sacreBLEU Post (2018) to evaluate all our models.

#### Model and preprocessing

We use the standard Transformer base model Vaswani et al. (2017), with 6
layers and 8 attention heads. The dropout rate is set to 0.1 and we use label
smoothing of 0.1. For all datasets, we process the data using sentencepice
Kudo and Richardson (2018) with a vocabulary size of 40k. Parallel sentences
with length longer than 200 word pieces are filtered out during training.

#### Construction of the clean data

GLMask requires a small set of clean data to guide model training. Here we
simply use previous years’ WMT evaluation sets as our clean data. For en-de
and en-fr, we concatenate the news test sets from 2010 to 2012 as the clean
data, which contains approximately 8k sentences. For en-zh, we re-use our
validation set, newsdev2017, as the clean data, which has about 2k sentences.
In practice, it is generally reasonable to obtain a small amount of high
quality annotated data.

###  3.2 Baselines

We compare with two baselines: 1) vanilla: standard transformer model trained
on all parallel data; 2) finetune: we finetune the vanilla model on the small
cleaned dev set used for calculating loss masking in our method.

For GLMask, we use two variations of loss masking: 1) sentence level (GLMask-
sent): we mask out the loss of the parallel sentences in a batch; 2) word
level (GLMask-word): we mask out the loss of each individual subwords on the
target side.

###  3.3 Results

We evaluate the baselines and our methods on the WMT test sets, and document
the results in Tab. 1. First, we notice that finetuning is a strong baseline,
especially when the training data is relatively diverse. It does not lead to
improvement for en-de, which has the least amount of training data, but it
improves over the vanilla model by 1 BLEU for the higher resourced en-zh and
en-fr language pairs. GLMask improves over the strong finetune baseline for
all three language pairs. Specifically, using GLMask on the word level
consistently outperforms masking on the sentence level.

| en-de | en-zh | en-fr
---|---|---|---
Vanilla | 27.29 | 32.99 | 39.30
Finetune | 27.26 | 33.99 | 40.33
GLMask-sent | 27.49 | 34.19 | 40.34
GLMask-word | 27.94 | 34.96 | 40.66
Table 1: Results on the WMT test set. | en-de | en-zh | en-fr
---|---|---|---
Vanilla | 28.79 | 25.64 | 42.24
Finetune | 28.54 | 25.17 | 42.02
GLMask-sent | 28.74 | 25.60 | 42.74
GLMask-word | 29.22 | 26.10 | 42.48
Table 2: Results on the IWSLT test set.

#### Out-of-domain generalization results

GLMask utilizes a small clean dataset to guide the training of NMT models. One
potential criticism of this design choice is that the model might overfit to
this small data. To examine how well the model trained using GLMask
generalizes to new domains, we construct an additional test set using the
IWSLT (Cettolo et al., 2012) data from TED talks. Since the small clean data
we use is drawn from the news domain, the model performance on the IWSLT test
sets is a good indicator how well GLMask generalizes to out-of-domain data. We
aggregate the test sets from IWSLT222https://wit3.fbk.eu/ 2011 to 2015 to
construct a large test set with about 5k sentence pairs.

We evaluate the performance of all models on the IWSLT test set and record the
results in Tab. 2. Although finetuning outperforms the vanilla model on the
WMT test sets from the news domain, it does not improve on the IWSLT test
sets. This indicates that finetuning on the small clean data might overfit to
the specific domain of the clean data. On the other hand, GLMask improves over
the vanilla model on the out-of-domain test set for all three language pairs
without any supervision for the IWSLT domain.

###  3.4 What data gets masked out?

Figure 1: Ave. percentage of words that are not masked out for all examples,
and copied examples. Examples where source and target sentences are identical
are masked out more.

Figure 2: Percentage of alphabetical words for the lowest and highest masked
out words. Words that are masked out more tend to have less semantically rich
alphabetical words.

In this section, we analyze the training sentences and words that tend to be
masked out by GLMask. We use a converged vanilla model to calculate the
gradient alignments of a subset of training data, with about 250k sentence
pairs.

First, we examine the sentence pairs using the percentage of words that are
not masked for each sentence. In Fig. 1, we plot the average percentage of
unmasked words in all the sampled training examples (All), and examples that
have identical source and target sentences (Copied). En-de has higher
percentage of unmasked, probably because it has a relatively cleaner training
set with the least amount of data. For all three language pairs, especially
the more noisy en-zh and en-fr, copied examples have much less unmasked words
than average. This indicates that GLMask is able to filter out copied
sentences, which is known to be one of the most harmful categories of noise
for NMT Khayrallah and Koehn (2018).

Next, we analyze the type of target words that tend to be masked out by
GLMask. We sort the target words by the percentage of being masked, and
compare the 1k words with the highest masking rate and the 1k words with the
lowest masking rate. Alphabetical words, or words with only alphabetical
characters, usually have richer semantic meaning than words that contain
numbers or symbols. In Fig. 2, we compare the percentage of alphabetical words
for the two group of words with the highest and lowest masking rates. For all
three languages, the words that have a higher masking rate tend to have less
alphabetical words, which indicates that the words having clearer and richer
meanings are more favored by GLMask.

We show some training examples and the target words that are masked out by
GLMask in Tab. 3. Our method masks out the punctuation words in the target
that do not align with the source sentences.

| Src | Trg
---|---|---
en-de | Neither have you , I hope . | Sie doch hoffentlich auch nicht ?
en-zh | What about you r | 你认为 呢? r

Table 3: Examples training data. Red words are masked out.

##  4 Conclusion

In this paper, we evaluate a strategy to dynamically mask out unhelpful data
during NMT training . We propose GLMask, a simple method that uses the
gradient alignments between the training data and a small clean dataset to
improve data usage. Experiments show that our method not only brings
significant improvements on three WMT datasets, but also improves out-of-
domain performance.

## References

  * Abadi et al. (2016) Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016\.  Tensorflow: A system for large-scale machine learning.  In _$\\{$ USENIX$\\}$_.
  * Cettolo et al. (2012) M. Cettolo, Girardi C., and Federico M. 2012.  Wit3: Web inventory of transcribed and translated talks.  In _EAMT_.
  * Esplà et al. (2019) Miquel Esplà, Mikel Forcada, Gema Ramírez-Sánchez, and Hieu Hoang. 2019\.  ParaCrawl: Web-scale parallel corpora for the languages of the EU.  In _EACL: Machine Translation Summit_.
  * Junczys-Dowmunt (2018) Marcin Junczys-Dowmunt. 2018.  Dual conditional cross-entropy filtering of noisy parallel corpora.  In _WMT_.
  * Khayrallah and Koehn (2018) Huda Khayrallah and Philipp Koehn. 2018.  On the impact of various types of noise on neural machine translation.  In _WMT_.
  * Kudo and Richardson (2018) Taku Kudo and John Richardson. 2018.  Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing.  In _EMNLP_.
  * Moore and Lewis (2010) Robert C. Moore and William Lewis. 2010.  Intelligent selection of language model training data.  In _ACL_.
  * Post (2018) Matt Post. 2018.  A call for clarity in reporting BLEU scores.  In _WMT_.
  * Ren et al. (2018) Mengye Ren, Wenyuan Zeng, Bin Yang, and Raquel Urtasun. 2018.  Learning to reweight examples for robust deep learning.  In _ICML_.
  * Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017.  Attention is all you need.  In _NIPS_.
  * Wang et al. (2018) Wei Wang, Taro Watanabe, Macduff Hughes, Tetsuji Nakagawa, and Ciprian Chelba. 2018\.  Denoising neural machine translation training with trusted data and online data selection.  In _WMT_.
  * Wang et al. (2020a) Xinyi Wang, Hieu Pham, Paul Mitchel, Antonis Anastasopoulos, Jaime Carbonell, and Graham Neubig. 2020a.  Optimizing data usage via differentiable rewards.  In _ICML_.
  * Wang et al. (2020b) Xinyi Wang, Yulia Tsvetkov, and Graham Neubig. 2020b.  Balancing training for multilingual neural machine translation.  In _ACL_.
  * van der Wees et al. (2017) Marlies van der Wees, Arianna Bisazza, and Monz Christof. 2017.  Dynamic data selection for neural machine transaltion.  In _EMNLP_.

##  Appendix A Appendix

###  A.1 Source Code for Training with GLMask in TensorFlow

Get masked training gradient in TensorFlow

⬇

1def get_train_gradient(train_loss, valid_loss, model):

2 """Sample a batch of corrupted examples from sents.

3

4 Args:

5 train_loss: Tensor [batch_size, n_steps]. The loss for

6 each training example in a batch.

7 valid_loss: Tensor [1]. The aggregated loss for a batch of valid data.

8 model: the parameters of the NMT model.

9

10 Returns:

11 train_grad: Tensor [batch_size, n_steps]. The training gradient after
masking.

12 """

13 # A dummy variable to assist calculating gradient dot product

14 z = tf.ones(train_loss.shape)

15

16 train_grad = tf.gradient(train_loss, model, grad_ys=z)

17 valid_grad = tf.gradient(valid_loss, model)

18 # dot product between train_grad and vali_grad

19 dot_prod = tf.gradient(train_grad, z, grad_ys=valid_grad)

20

21 # mask out the examples with negative gradient alignment

22 gradient_mask = tf.greater(dot_prod, tf.zeros(dot_prod.shape))

23

24 train_grad = train_grad * gradient_mask

25 return train_grad