EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH (CERN)

 CERN-EP-2018-244 LHCb-PAPER-2018-033 5 April 2019

Measurement of the branching fractions of the decays
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$,
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ and
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$

LHCb collaboration†††Authors are listed at the end of this paper.

The branching fractions of the doubly Cabibbo-suppressed decays
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$,
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ and
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$ are measured using
the decays ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ and
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ as normalisation
channels. The measurements are performed using proton-proton collision data
collected with the LHCb detector at a centre-of-mass energy of
8$\mathrm{\,Te\kern-1.00006ptV}$, corresponding to an integrated luminosity of
2.0$\mbox{\,fb}^{-1}$. The results are

$\displaystyle\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
$\displaystyle=(6.541\pm 0.025\pm 0.042)\times 10^{-4},$
$\displaystyle\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
$\displaystyle=(5.231\pm 0.009\pm 0.023)\times 10^{-3},$
$\displaystyle\frac{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}$
$\displaystyle=(2.372\pm 0.024\pm 0.025)\times 10^{-3},$

where the uncertainties are statistical and systematic, respectively. These
are the most precise measurements up to date.

Published in JHEP 03 (2019) 176

© 2024 CERN for the benefit of the LHCb collaboration. CC-BY-4.0 licence.

##  1 Introduction

Precise measurements of the branching fractions of doubly Cabibbo-suppressed
(DCS) decays of charmed mesons provide important information for the
understanding of the decay dynamics of these particles. The theoretical
description of charm-meson decays is challenging. The charm quark is not heavy
enough for a reliable application of the factorisation approach and heavy-
quark expansion tools, successfully used in $B$-meson decays. It is also not
light enough for the application of chiral perturbation theory, as in the case
of kaon decays. Phenomenological models and approximate symmetries, such as
those based on the diagrammatic approach [1, 2], rely on the knowledge of
branching fractions and, in the case of multi-body final states, resonant
structures, as key inputs. Whilst the branching fractions of some decay modes
of charmed mesons are well measured, the uncertainties on branching fractions
of doubly Cabibbo-suppressed decays are still large.

In this paper, three ratios of branching fractions of DCS decays of ${D}^{+}$
and ${D}^{+}_{s}$ mesons111Throughout this paper, charge conjugated decays are
implied. are measured with unprecedented precision,

$\frac{{\cal B}({{D}^{+}}\\!\rightarrow K^{-}K^{+}K^{+})}{{\cal
B}({{D}^{+}}\\!\rightarrow K^{-}\pi^{+}\pi^{+})},\hskip 5.69046pt\frac{{\cal
B}({{D}^{+}}\\!\rightarrow\pi^{-}\pi^{+}K^{+})}{{\cal
B}({{D}^{+}}\\!\rightarrow K^{-}\pi^{+}\pi^{+})},\hskip 5.69046pt\frac{{\cal
B}({{D}^{+}_{s}}\\!\rightarrow\pi^{-}K^{+}K^{+})}{{\cal
B}({{D}^{+}_{s}}\\!\rightarrow K^{-}K^{+}\pi^{+})}.$ (1)

In addition, the branching fraction of the Cabibbo-suppressed (CS)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ decay is measured
relative to that of the Cabibbo-favoured (CF)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ decay.

(a) (b) (c)

Figure 1: Feynman diagrams for the DCS decays (a)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$, (b)
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ and (c)
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$.

Tree-level diagrams for the three DCS decays are exemplified in Fig. 1, where
the final state particles can be produced through resonances not explicitly
shown. The decay ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ is
expected to occur through an annihilation process as in Fig. 1(a) but it is
also possible to produce the ${K}^{-}$ ${K}^{+}$ ${K}^{+}$ final state through
a diagram similar to that in Fig. 1(b), where a ${K}^{+}$ ${K}^{-}$ pair could
be formed through the ${{K}^{0}}{{\kern
1.99997pt\overline{\kern-1.99997ptK}{}}{}^{0}}\\!\rightarrow{{K}^{+}}{{K}^{-}}$
rescattering or through a resonance that couples to both $d\bar{d}$ and
$s\bar{s}$.

The world averages [3] of these ratios of branching fractions are listed in
Table 1. In the case of the
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ decay, there is only one
previous measurement by the FOCUS collaboration [4], based on a sample of 65
$\pm$ 15 decays and with a precision of 23 %.

Table 1: World averages for the branching-fractions ratios under consideration [3]. Ratio | Value [$\times 10^{-3}$]
---|---
${\cal B}$(${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$)$/{\cal B}$(${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$) | 0.95 $\pm$ | 0.22
${\cal B}$(${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$)$/{\cal B}$(${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$) | 5.77 $\pm$ | 0.22
${\cal B}$(${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$)$/{\cal B}$(${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$) | 2.33 $\pm$ | 0.23
${\cal B}$(${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$)$/{\cal B}$(${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$) | 105.9 $\pm$ | 1.8

The results presented in this paper are obtained with a sample of
$pp$-collision data corresponding to an integrated luminosity of 2.0 fb-1,
collected at a centre-of-mass energy of 8$\mathrm{\,Te\kern-1.00006ptV}$ with
the LHCb detector. In Section 2 a description of the detector and simulation
is presented. The method used to measure the ratio of branching fractions is
described in Section 3. The selection is discussed in Section 4. The
determination of the efficiencies in bins of the phase space is explained in
Section 5. The fit model and the evaluation of the signal yields are presented
in Section 6. Systematic uncertainties associated with the measurements are
discussed in Section 7. Finally, the results and conclusions are presented in
Section 8.

##  2 Detector and simulation

The LHCb detector [5, 6] is a single-arm forward spectrometer covering the
pseudorapidity range $2<\eta<5$, designed for the study of particles
containing $b$ or $c$ quarks. The detector includes a high-precision tracking
system consisting of a silicon-strip vertex detector surrounding the $pp$
interaction region, a large-area silicon-strip detector located upstream of a
dipole magnet with a bending power of about $4{\mathrm{\,Tm}}$, and three
stations of silicon-strip detectors and straw drift tubes placed downstream of
the magnet. The tracking system provides a measurement of the momentum, $p$,
of charged particles with a relative uncertainty that varies from 0.5% at low
momentum to 1.0% at 200${\mathrm{\,Ge\kern-1.00006ptV\\!/}c}$. The minimum
distance of a track to a primary vertex (PV), the impact parameter (IP), is
measured with a resolution of
$(15+29/\mbox{$p_{\mathrm{T}}$}){\,\upmu\mathrm{m}}$, where $p_{\mathrm{T}}$
is the component of the momentum transverse to the beam, in
${\mathrm{\,Ge\kern-1.00006ptV\\!/}c}$. Different types of charged hadrons are
distinguished using information from two ring-imaging Cherenkov detectors [7].
Photons, electrons and hadrons are identified by a calorimeter system
consisting of scintillating-pad and preshower detectors, an electromagnetic
calorimeter and a hadronic calorimeter. Muons are identified by a system
composed of alternating layers of iron and multiwire proportional chambers.

The polarity of the dipole magnet is reversed periodically throughout data
taking. The configurations with the magnetic field upwards, MagUp, and
downwards, MagDown, bend respectively positively and negatively charged
particles in the horizontal plane towards the centre of the LHC.

The online event selection is performed by a trigger system[8], which consists
of a hardware stage, based on information from the calorimeter and muon
systems, followed by a software stage, which applies a full event
reconstruction. At the hardware-trigger stage, events are required to have a
muon with high $p_{\mathrm{T}}$ or a hadron, photon or electron with high
transverse energy in the calorimeters. In the offline selection, the hardware
trigger signals are associated with reconstructed particles. Selection
requirements can therefore be made on whether the decision is due to the
signal candidate, other particles produced in the $pp$ collision, or a
combination of both. The latter is used in this analysis. The software trigger
is divided into two parts. The first part employs a partial reconstruction of
the tracks, and a requirement on $p_{\mathrm{T}}$ and IP is applied to, at
least, one final-state track forming the ${D}^{+}_{({s})}$ candidate. In the
second part a full event reconstruction is performed and dedicated algorithms
are used to select $D^{+}_{(s)}$ candidates decaying into three charged
hadrons.

In the simulation, $pp$ collisions are generated using Pythia [9,
*Sjostrand:2007gs] with a specific LHCb configuration [11]. Decays of hadronic
particles are described by EvtGen [12], in which final-state radiation is
generated using Photos [13]. The interaction of the generated particles with
the detector, and its response, are implemented using the Geant4 toolkit [14,
*Agostinelli:2002hh] as described in Ref. [15].

##  3 Method

The ratios of branching fractions are measured as

$\frac{{\cal B}({D}^{+}_{({s})}\\!\rightarrow f_{\mathrm{signal}})}{{\cal
B}({D}^{+}_{({s})}\\!\rightarrow
f_{\mathrm{norm}})}=\frac{N_{\mathrm{signal}}^{\mathrm{prod}}}{N_{\mathrm{norm}}^{\mathrm{prod}}},$
(2)

where $f_{\mathrm{signal}}$ and $f_{\mathrm{norm}}$ correspond to the final
states of the signal and normalisation ${D}^{+}_{({s})}$ decays, and
$N_{\mathrm{signal}}^{\mathrm{prod}}$ and $N_{\mathrm{norm}}^{\mathrm{prod}}$
are the total number of produced signal and normalisation decays. These
numbers are determined by correcting the observed yields of signal
($N_{\mathrm{signal}}^{\mathrm{obs}}$) and normalisation
($N_{\mathrm{norm}}^{\mathrm{obs}}$) decays after full selection criteria by
the total respective efficiencies, which are obtained from simulation and from
calibration data samples. Since there are no reliable decay amplitude models
available for all the ${D}^{+}_{({s})}\\!\rightarrow
h^{-}_{\mathrm{1}}h^{+}_{\mathrm{2}}h^{+}_{\mathrm{3}}$ decays,222Here $h$
denotes a pion or a kaon and the particle ordering is such that
$h_{\mathrm{1}}$ has opposite charge with respect to the ${D}^{+}_{({s})}$
candidate. the simulated samples are generated according to phase space
distribution. As the efficiency, $\varepsilon$, is not uniform across the
phase space, both the efficiency and the number of observed decays are
obtained in bins of the Dalitz plot (DP) [16], built with two independent
invariant masses squared, denoted as $s(h_{\mathrm{1}}^{-}h_{\mathrm{2}}^{+})$
and $s(h_{\mathrm{1}}^{-}h_{\mathrm{3}}^{+})$. The total number of produced
decays is then evaluated as

$N^{\mathrm{prod}}=\sum_{i}^{N_{\mathrm{bins}}}\frac{N_{i}^{\mathrm{obs}}}{\varepsilon_{i}},$
(3)

where the index $i$ runs over the bins within the kinematically allowed region
of the decay DP. When the decay has two identical particles in the final
state, the DP is folded, with axes corresponding to the highest and lowest
values of the two invariants, $s_{\mathrm{hi}}(h^{-}h^{\prime+})$ and
$s_{\mathrm{lo}}(h^{-}h^{\prime+})$.

The distributions of both the efficiencies and observed yields over the phase
space are obtained separately for statistically independent datasets split by
magnet polarity. For each pair of signal and normalisation decays, the final
experimental result is the combination of the MagDown and MagUp measurements
of the ratio of branching fractions.

Systematic uncertainties are estimated using the ratios of observed yields
$N_{\mathrm{signal}}^{\mathrm{obs}}/N_{\mathrm{norm}}^{\mathrm{obs}}$ and the
ratios of average efficiencies, where the average is over the DP bins with
weights given by the corresponding yields of observed candidates. They are
also obtained separately for the different magnet polarities. The
contributions from the relative uncertainties on the ratios of yields and on
effective efficiencies are then added in quadrature to provide the relative
uncertainty on each ratio of branching fractions.

##  4 Offline selection

The offline candidate selection reduces the combinatorial background and
suppresses specific peaking structures in the various mass spectra. These
structures are due to crossfeeds from decays of other charm particles, which
occur when one or more final-state particles are misidentified or not
reconstructed.

A first set of requirements exploits the decay topology by selecting
combinations of three charged hadrons forming a good quality decay vertex,
well detached from the PV. The PV is that with the smallest value of
$\chi^{2}_{\text{IP}}$, where $\chi^{2}_{\text{IP}}$ is defined as the
difference in the vertex-fit $\chi^{2}$ of the PV reconstructed with and
without the particle under consideration, in this case the ${D}^{+}_{({s})}$
candidate. The requirements at this level are made on the following
quantities: the distance between the PV and the ${D}^{+}_{({s})}$ decay
vertex; the IP of the ${D}^{+}_{({s})}$ candidate; the angle between the
reconstructed ${D}^{+}_{({s})}$ momentum and the flight direction; the
$\chi^{2}$ of the ${D}^{+}_{({s})}$ decay vertex fit; the distance of closest
approach between any two final-state tracks; and the momentum, the transverse
momentum and the $\chi^{2}_{\text{IP}}$ of the ${D}^{+}_{({s})}$ candidate and
of its decay products. For each branching-fraction ratio measurement, signal
and normalisation-channel candidates are selected with the same topology
requirements, allowing a partial cancellation of the systematic uncertainties.
Besides being effective to reduce combinatorial background, these topology
criteria suppress the background from the decays $D^{*+}\rightarrow
D^{0}\pi^{+}$, where the $D^{0}$ decays to two charged hadrons, such as
$D^{0}\rightarrow K^{-}\pi^{+}$ or $D^{0}\rightarrow K^{-}\pi^{+}\pi^{0}$.

Particle identification (PID) criteria are used to distinguish between kaons
and pions and to veto muons from semileptonic decays with two charged hadrons
and a muon in the final state, such as the ${D}^{+}\rightarrow
K^{-}\pi^{+}\mu^{+}\nu_{\mu}$ decay. Further selection criteria based on more
stringent PID requirements or invariant-mass vetoes are used to suppress
crossfeeds contributing to each decay mode, except for the
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$ channel, which does
not present this kind of contamination.

The two main crossfeeds in the
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ channel are those from
${\mathchar 28931\relax}^{+}_{c}$ decays into $K^{-}K^{+}p$ and
$K^{-}p\pi^{+}$ final states. The former is the dominant contribution, in
spite of being Cabibbo suppressed, since this background is caused by a single
$p-K$ misidentification. These backgrounds are removed using invariant-mass
vetoes. Candidates are reconstructed under the $K^{-}K^{+}p$ and
$K^{-}p\pi^{+}$ mass hypotheses and rejected if the resulting invariant masses
are within [2280, 2296]${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$. This veto
is slightly different for other decay modes as the reconstructed width of the
${\mathchar 28931\relax}^{+}_{c}$ mass peak is affected by the decay channel-
dependent selection criteria.

The main exclusive backgrounds for the
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ decay are the fully
reconstructed decays
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$,
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$, ${{\mathchar
28931\relax}^{+}_{c}}\rightarrow\pi^{-}\pi^{+}p$ and
$D^{+}\rightarrow{{K}^{0}_{\mathrm{\scriptscriptstyle S}}}K^{+}$, where
${K}^{0}_{\mathrm{\scriptscriptstyle S}}$ decays to ${\pi}^{-}$ ${\pi}^{+}$.
The ${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ decay is the
most abundant contamination, occurring when the $K^{+}$ meson is misidentified
as a pion. The contamination from the decay
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ is due to a double
$K-\pi$ misidentification. These two backgrounds are suppressed by stringent
PID requirements on the kaon and opposite-charge pion candidates. The
crossfeed from the ${{\mathchar
28931\relax}^{+}_{c}}\rightarrow\pi^{-}\pi^{+}p$ decays, on the other hand, is
eliminated by an invariant-mass veto. The $p-K$ misidentification occurs
mostly at high momenta, where the discrimination between these two particles
is limited. Candidates are reconstructed under the $\pi^{-}\pi^{+}p$
hypothesis and rejected if their invariant mass is within the interval [2275,
2300]${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$. The decay
$D^{+}\rightarrow{{K}^{0}_{\mathrm{\scriptscriptstyle
S}}}(\pi^{-}\pi^{+})K^{+}$ has the same final state as the
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ decay, hence this
contamination cannot be suppressed using PID. In this case, candidates with
$\pi^{-}\pi^{+}$ invariant mass within the interval [488,
508]${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ are discarded.

The main backgrounds in the
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ sample are the
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ and the ${{\mathchar
28931\relax}^{+}_{c}}\rightarrow K^{-}p\pi^{+}$ decays. A stringent PID
requirement on the $K^{+}$ candidate is used to suppress the contamination
from ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ decays, whereas
an invariant-mass veto eliminates the ${{\mathchar
28931\relax}^{+}_{c}}\rightarrow K^{-}p\pi^{+}$ background. The
$K^{-}K^{+}\pi^{+}$ candidate is reconstructed as $K^{-}p\pi^{+}$ and the
candidate is discarded if the resulting invariant mass is within [2275,
2305]${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$.

The ${{\mathchar 28931\relax}^{+}_{c}}\rightarrow K^{-}p\pi^{+}$ decay is the
main specific background contribution in the
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ sample. The
$K^{-}K^{+}\pi^{+}$ candidates are reconstructed as $K^{-}p\pi^{+}$ and those
with invariant mass within [2275,
2305]${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ are vetoed.

There are two backgrounds in the
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ sample, the decays
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ and ${{\mathchar
28931\relax}^{+}_{c}}\rightarrow K^{-}p\pi^{+}$. To reject the ${\mathchar
28931\relax}^{+}_{c}$ background the $K^{-}\pi^{+}\pi^{+}$ candidates are
reconstructed as $K^{-}p\pi^{+}$ and those with invariant mass within [2280,
2300]${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ are vetoed. The crossfeed from
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ is suppressed using
a stringent PID requirement on the pion candidate with the highest momentum.

##  5 Efficiencies

In order to take into account the variation of the efficiencies across the
phase space, the measurement of the ratios of branching fractions in this
analysis is based upon the correction of the observed yields in bins of the
corresponding DP.

In each bin $i$ of the DP, the overall selection efficiencies for signal and
normalisation modes, $\varepsilon_{i}$ in Eq. 3, are factorised into
components that are independently measured. The acceptance due to the detector
geometry and the efficiencies due to trigger, final state particles
reconstruction, offline selection and invariant-mass vetoes are obtained from
simulation.

The PID efficiency of each candidate is estimated by multiplying the
efficiencies for each final-state particle, which are evaluated from
calibration samples of ${{D}^{0}}\rightarrow{{K}^{-}}{{\pi}^{+}}$ decays [17]
and depend on the particle momentum, pseudorapidity and event charged-particle
multiplicity. Average PID efficiencies are in the range of 60 to 70%.

There are some small differences in the hardware trigger and tracking
efficiencies between data and simulation. These differences are accounted for
by weighting the simulation using data. The tracking-correction weight is
obtained by multiplying the weights for each final-state particle, determined
as a function of the particle momentum, transverse momentum, dipole magnet
polarity and event charged-particle multiplicity [18]. The impact of this
correction on the individual efficiencies is at the level of 3%.

The trigger efficiency correction follows the method described in Ref. [19].
The total data sample for each decay mode is separated into two mutually
exclusive subsamples. The first is composed of candidates that are triggered
at the hardware level by one or more of the final state particles interacting
in the hadronic calorimeter. The second is composed of candidates triggered
only by particles in the rest of the event. The correction makes use of
calibration data samples of ${{D}^{0}}\rightarrow{{K}^{-}}{{\pi}^{+}}$ decays
and affects differently these two subsamples. The correction factors are
evaluated as a function of the DP position for each subsample and combined
into a single efficiency correction map according to their proportions in
data.

The final efficiency maps, obtained after the full selection and corrections
described above, are shown in Fig. 2, for all decays, for MagDown polarity
(the plots for MagUp are similar). The binning schemes used for each mode are
introduced in these plots. The corresponding average efficiencies vary among
the different decay modes from $2.7\times 10^{-4}$ (for
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$) to $7.0\times
10^{-4}$ (for ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$). The
different lifetimes of the parent mesons and different PID criteria are the
predominant contributions to this variation. The ratios between signal and
normalisation channels are given in Table 2. The impact of the different
corrections (tracking, trigger and charged-particle multiplicity) applied to
the efficiencies is below 1.5% for all ratios.

Table 2: Ratios of average efficiencies for the full selection. The quoted uncertainty is due to the limited size of the simulated sample only (see Section 7). Ratio of efficiencies | MagDown | MagUp
---|---|---
${\varepsilon_{{{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}}}/{\varepsilon_{{{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}}}$ | 1.0024 $\pm$ | 0.0034 | 1.0077 $\pm$ | 0.0033
${\varepsilon_{{{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}}}/{\varepsilon_{{{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}}}$ | 0.958 $\pm$ | 0.005 | 0.956 $\pm$ | 0.005
${\varepsilon_{{{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}}}/{\varepsilon_{{{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}}}$ | 1.242 $\pm$ | 0.013 | 1.215 $\pm$ | 0.014
${\varepsilon_{{{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}}}/{\varepsilon_{{{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}}}$ | 1.096 $\pm$ | 0.008 | 1.108 $\pm$ | 0.009

Figure 2: Efficiency maps for (top left)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$, (top right)
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$, (middle left)
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$, (middle right)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$, (bottom left)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ and (bottom right)
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ decays with PID
efficiency, tracking, multiplicity and hardware trigger efficiency
corrections, for MagDown polarity.

##  6 Determination of the yields

The yields of the signal and normalisation channels are determined by extended
binned maximum-likelihood fits to the invariant-mass distribution of each
sample independently. For each channel, the signal probability distribution
function (PDF) is represented by a sum of a Gaussian function and two Crystal
Ball (CB) [20] functions, while the background is modelled by an exponential
function. The signal PDF is

${\cal P}_{\rm sig}(M)=f_{\rm G}\times{\rm G}(\mu,\sigma_{\rm G})+(1-f_{\rm
G})\times\left[f_{\rm CB}\times{\rm CB}_{1}(\mu,R_{1}\sigma_{\rm
G},\alpha_{1},N_{1})+\right.\\\ \left.(1-f_{\rm CB})\times{\rm
CB}_{2}(\mu,R_{2}\sigma_{\rm G},\alpha_{2},N_{2})\right],$ (4)

where $\mu$ and $\sigma_{\rm G}$ are the mean value and the width of the
Gaussian function G. The two Crystal Ball functions, CB1 and CB2, have widths
$R_{1}\sigma_{\rm G}$ and $R_{2}\sigma_{\rm G}$, and tail parameters
$\alpha_{1}$, $N_{1}$ and $\alpha_{2}$, $N_{2}$. A common parameter, $\mu$,
describes the most probable mass value of the two Crystal Balls and the mean
of the Gaussian function.

The fractions of each PDF component are $f_{\rm G}$ for the Gaussian function,
$(1-f_{\rm G})\times f_{\rm CB}$ for CB1 and $(1-f_{\rm G})\times(1-f_{\rm
CB})$ for CB2. The parameters $\alpha_{i}$, $N_{i}$, $R_{i}$, $f_{\rm CB}$ and
$f_{\rm G}$ defining the signal PDF are fixed to the values obtained from a
fit to the simulation sample. The position of the signal mass peak presents a
small dependence on the charge of the ${D}^{+}_{({s})}$ meson and on the
magnet polarity. Therefore, the samples are divided into four subsamples to
ensure a precise determination of the yields.

Due to the large size of the samples of CF channels
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$,
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ and for the CS
channel ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$, the
convergence and goodness of the fit are sensitive to the momentum-dependent
resolution of the ${D}^{+}_{({s})}$ candidate, making it difficult to fit
these samples with a single set of parameters. For this reason, the
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ and
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$
(${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$) samples are
further divided into 50 (20) bins of ${D}^{+}_{({s})}$ momentum. The variation
of $\sigma_{\rm G}$ over the momentum bins is of the order of 50%. For each
magnet polarity, the total signal yield, shown in Table 3, is the sum of the
yields in the different subsets. For illustration purposes, the invariant-mass
distribution for each of the DCS decay modes and for the CS channel are shown
in Fig. 3 for the whole sample, summing also over the two magnet polarities,
with the associated fit results superimposed. The mass distributions for the
CF normalisation modes are shown in Fig. 4.

Table 3: Observed yields for signal and normalisation modes with statistical uncertainties. The entry for the decay ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ (†) corresponds to the yields obtained from the fit to ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ sample with cuts optimised for the ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ and ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ selections. The entry for the decay ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ (††) is for fits to ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ samples with cuts optimised for the ${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ selection. | Yields [$\times 10^{3}$]
---|---
Channel | MagDown | MagUp | Total
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ | 67.61 $\pm$ | 0.33 | 66.69 $\pm$ | 0.33 | 134.30 $\pm$ | 0.47
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ | 401.2 $\pm$ | 1.0 | 393.7 $\pm$ | 1.0 | 794.9 $\pm$ | 1.4
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$ | 33.7 $\pm$ | 0.4 | 33.6 $\pm$ | 0.4 | 67.2 $\pm$ | 0.5
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ | 11 657 $\pm$ | 4 | 11 482 $\pm$ | 4 | 23 139 $\pm$ | 5
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ (†) | 103 282 $\pm$ | 10 | 101 008 $\pm$ | 10 | 204 290 $\pm$ | 14
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ (††) | 80 197 $\pm$ | 10 | 78 530 $\pm$ | 10 | 158 727 $\pm$ | 13
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ | 11 629 $\pm$ | 4 | 11 414 $\pm$ | 4 | 23 044 $\pm$ | 5

Figure 3: Invariant-mass distributions of (top left)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$, (top right)
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$, (bottom left)
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$ and (bottom right)
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ with the corresponding
fit result superimposed (red solid line). The blue dotted line corresponds to
the signal PDF and the dashed grey line shows the background PDF.

Figure 4: Invariant-mass distributions of candidates of the normalisation
modes (left) ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ and
(right) ${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ with the
corresponding fit result superimposed (red solid line). The blue dotted line
corresponds to the signal PDF and the dashed grey line shows the background
PDF.

The observed signal yields in bins of the DP, $N_{i}^{\rm obs}$ in Eq. 3, are
determined using the sPlot technique [21]. For each data subset, the signal
and background sWeights are obtained from the maximum-likelihood fit, and the
former are used to compute the number of signal candidates in each bin of the
phase space for each data subset. No significant correlation between the
$D^{+}_{(s)}$ candidate mass and position in the DP is observed. The DP with
the total signal yields for all decays (merging ${D}^{+}_{({s})}$ and
${D}^{-}_{({s})}$, MagDown and MagUp subsets) are shown in Fig. 5.

With the yields of observed candidates and the efficiencies obtained in bins
of the DP for signal and normalisation modes, the total yields produced in the
$pp$ collisions are evaluated using Eq. 3. These numbers are listed in Table
4, separately for the MagDown and MagUp samples and can then be used for the
determination of the different ratios of branching fractions.

Figure 5: Dalitz plots of the (top left) ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$, (top right) ${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$, where the ${K}^{0}_{\mathrm{\scriptscriptstyle S}}$ veto can be seen, (middle left) ${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$, (middle right) ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$, (bottom left) ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ and (bottom right) ${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ decays, with signal weights from sPlot. A logarithmic scale is used for the ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ and ${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ channels. Table 4: Produced yields for each decay mode with statistical uncertainties, shown separately for MagDown and MagUp samples. The numbers given for the decay ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ (†) correspond to the sample with cuts optimised for the ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ and ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ and those for the decay ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$(††) to the sample with cuts optimised for the ${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ selection. | Produced yields $[\times 10^{7}]$
---|---
Decay | MagDown | MagUp
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}}$ | 10.52 $\pm$ | 0.06 | 10.54 $\pm$ | 0.06
${{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}}$ | 84.2 $\pm$ | 0.4 | 84.4 $\pm$ | 0.4
${{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}}$ | 9.85 $\pm$ | 0.15 | 10.04 $\pm$ | 0.17
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ | 1659 $\pm$ | 12 | 1651 $\pm$ | 13
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$(†) | 16103 $\pm$ | 40 | 16092 $\pm$ | 40
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$($ˆ{\dagger\dagger}$) | 16130 $\pm$ | 50 | 16101 $\pm$ | 50
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ | 4221 $\pm$ | 34 | 4150 $\pm$ | 33

##  7 Systematic uncertainties

Due to the similar decay topologies and selections applied to signal and
normalisation channels within independent data subsets, many systematic
uncertainties related to the final state factorise and cancel in the ratio of
signal to normalisation yields and efficiencies.

The systematic uncertainties due to the limited size of the simulation samples
are determined using pseudoexperiments. The number of generated events in each
pseudoexperiment is obtained randomly in bins of the DP according to a Poisson
distribution and corrected by the nominal efficiency and correction maps due
to PID, tracking and trigger. The uncertainties of the signal and
normalisation efficiencies are taken as the Gaussian width of the resulting
distributions of average efficiencies. The mean value of these distributions
are compatible with the nominal values. The resulting uncertainties of the
ratios of efficiencies are given in Table 2 and the corresponding relative
systematic uncertainties are given in Table 5.

In order to estimate the uncertainties on the ratios of effective efficiencies
arising from the limited size of the calibration samples used to determine the
PID efficiency and the tracking and trigger corrections, 100 tables are
generated, with efficiencies or corrections fluctuating according to Gaussian
functions centred at the nominal value and with width equal to their nominal
uncertainties. For each generated table, the DP map of efficiencies are re-
evaluated for signal and normalisation channels using the same procedure as
the one used for the determination of the nominal efficiency maps. The
distribution of the efficiency ratio is fitted with a Gaussian function, whose
width is taken as systematic uncertainty.

Table 5: Relative systematic uncertainties for the MagDown and MagUp results (in %). The statistical uncertainties are also given for comparison. Source | $\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | $\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | $\frac{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}$ | $\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
---|---|---|---|---
| MagDown
Size of simulation | 0.34 | 0.47 | 1.0 | 0.75
PID | 0.022 | 0.019 | 0.022 | 0.013
Tracking | 0.22 | 0.069 | 0.079 | 0.11
Trigger corr. | 0.011 | 0.0025 | 0.0050 | 0.0057
Mat. description | 0.53 | $-$ | $-$ | 0.27
Fit Model | 0.14 | 0.03 | 0.64 | 0.06
Sec. decays | 0.18 | 0.25 | 0.31 | 0.11
DP Binning | 0.09 | 0.05 | 0.30 | 0.13
Total syst. | 0.72 | 0.54 | 1.3 | 0.82
Statistical | 0.54 | 0.25 | 1.4 | 0.03
| MagUp
Size of simulation | 0.32 | 0.52 | 1.2 | 0.81
PID | 0.030 | 0.020 | 0.023 | 0.021
Tracking | 0.22 | 0.070 | 0.080 | 0.10
Trigger corr. | 0.011 | 0.0024 | 0.0057 | 0.0060
Mat. description | 0.53 | $-$ | $-$ | 0.27
Fit Model | 0.13 | 0.07 | 0.54 | 0.06
Sec. decays | 0.18 | 0.24 | 0.38 | 0.09
DP Binning | 0.11 | 0.03 | 0.07 | 0.28
Total syst. | 0.71 | 0.58 | 1.3 | 0.91
Statistical | 0.54 | 0.25 | 1.4 | 0.03

An additional systematic uncertainty is assigned to the ratios of tracking
efficiencies when signal and normalisation decay modes have a different number
of kaons and pions in the final state. The fractions of kaons and pions which
cannot be reconstructed due to hadronic interactions that occur before the
last tracking station are estimated using a simulated sample of
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ decays. Assuming a
10% uncertainty on the description of the detector material [18], per-track
uncertainties on the efficiency of kaons and pions of (1.432$\pm$0.015)% and
(1.702$\pm$0.011)%, respectively, are obtained. The residual uncertainties due
to the different interactions of particles with opposite charge with the
detector material are estimated to be negligible when compared to the
uncertainty due to the limited size of the calibration samples, since the
final branching-fraction ratios are averaged over particle charge and magnet
polarity. This is the most important source of systematic uncertainty for the
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
measurement.

The systematic uncertainty due to the fit model is estimated using an
alternative parametrisation for the signal based on the sum of two CB
functions with a common mean. The observed yields obtained with this model are
used to measure the branching-fraction ratios with the same procedure as for
the nominal evaluation and the difference between the two results is assigned
as systematic uncertainty.

The effect of residual charm contamination is studied. The stringent PID
requirements are chosen to suppress the charm backgrounds to minimal levels,
so that any remaining contribution does not affect the signal yields, either
because the number of candidates is very low or because its shape is broad
enough to be absorbed in the yield of the combinatorial background. This
assumption is tested by explicitly estimating the residual contaminations and
their shapes from data and simulation, and including them in the mass fits. No
significant effects are found for any of the signal modes. The impact of the
mass vetoes used to reject the ${\mathchar 28931\relax}^{+}_{c}$ contamination
is studied by further enlarging the mass-veto window by
5${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ for all channels. No significant
deviation is observed in any of the final results and therefore no systematic
uncertainty is assigned.

The effect of a potential contamination from decays of ${D}^{+}_{({s})}$ from
$b$-hadron decays, which could be different for signal and normalisation
samples, is investigated by tightening the requirement on
$\chi^{2}_{\text{IP}}$ to two alternative values and measuring the ratios of
branching fractions. The largest deviation from the nominal value is assigned
as systematic uncertainty.

The systematic uncertainty due to the choice of DP binning scheme is evaluated
as the deviation of the ratio of produced yields obtained with alternative
binning schemes from that obtained with the nominal binning schemes. These
binning schemes are defined by changing the nominal number of bins by $\pm$1,
$\pm$2 and $\pm$4 units in each DP axis.

The systematic uncertainties due to the different sources considered in this
analysis are summarised in Table 5, separately for the MagDown and MagUp
results. Except for the
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
measurement, the most important source of systematic uncertainty is the
limited size of the simulation samples. However, the only result with total
uncertainty dominated by this contribution is the branching-fraction ratio of
the CS decay ${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$.

The ratios of branching fractions obtained with data taken with the two magnet
polarities are shown in Table 6, with statistical and systematic
uncertainties. For each decay mode the two results are compatible and no
additional systematic uncertainty is assigned to the effect of detector
asymmetry.

Table 6: Ratios of branching fractions, shown separately for MagDown and MagUp samples. The first uncertainty is statistical and the second is systematic. Channel |  MagDown ($\times 10^{-3}$)
---|---
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | 0.653 $\pm$ | 0.004 $\pm$ | 0.005
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | 5.220 $\pm$ | 0.013 $\pm$ | 0.028
${{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}$ | 2.333 $\pm$ | 0.033 $\pm$ | 0.030
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | 103.00 $\pm$ | 0.03 $\pm$ | 0.85
Channel |  MagUp ($\times 10^{-3}$)
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | 0.655 $\pm$ | 0.004 $\pm$ | 0.005
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | 5.244 $\pm$ | 0.013 $\pm$ | 0.030
${{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}})}/{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}$ | 2.419 $\pm$ | 0.035 $\pm$ | 0.032
${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}/{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$ | 102.59 $\pm$ | 0.03 $\pm$ | 0.93

##  8 Results

Final ratios of branching fractions are obtained by combining the two
measurements shown in Table 6, accounting for 100% correlation [22] between
the systematic uncertainties due to the material description in the
simulation, fit model, contamination from secondary decays and DP binning. For
the doubly Cabibbo-suppressed channels, the results are

$\displaystyle\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
$\displaystyle=(6.541\pm 0.025\pm 0.042)\times 10^{-4},$
$\displaystyle\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}$
$\displaystyle=(5.231\pm 0.009\pm 0.023)\times 10^{-3},$
$\displaystyle\frac{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}})}{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}$
$\displaystyle=(2.372\pm 0.024\pm 0.025)\times 10^{-3},$

where the first uncertainty is statistical and the second systematic. These
values are consistent with the current world averages, being compatible at the
1.4$\sigma$, 2.4$\sigma$ and 0.2$\sigma$ levels, respectively.

In addition, the result for the Cabibbo-suppressed mode
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ is

$\frac{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}})}=(10.282\pm
0.002\pm 0.068)\times 10^{-2},$

where again the first uncertainties are statistical and the second systematic.
It is in agreement with the world average [3] at the 1.6$\sigma$ level,
improving the precision by a factor 2.6.

The ratios of branching fractions are combined with the world-average values
[3] of the branching fractions of the CF decays
${{D}^{+}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}{{\pi}^{+}}$ ($8.98\pm 0.28$)% and
${{D}^{+}_{s}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}}$ ($5.45\pm 0.17$)%
to compute the branching fractions of the DCS modes

$\displaystyle{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{K}^{+}})}$
$\displaystyle=(5.87\pm 0.02\pm 0.04\pm 0.18)\times 10^{-5},$
$\displaystyle{{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{\pi}^{-}}{{\pi}^{+}}{{K}^{+}})}$
$\displaystyle=(4.70\pm 0.01\pm 0.02\pm 0.15)\times 10^{-4},$
$\displaystyle{{\mathcal{B}}({{D}^{+}_{s}}\\!\rightarrow{{\pi}^{-}}{{K}^{+}}{{K}^{+}})}$
$\displaystyle=(1.293\pm 0.013\pm 0.014\pm 0.040)\times 10^{-4},$

and of the CS mode

${{\mathcal{B}}({{D}^{+}}\\!\rightarrow{{K}^{-}}{{K}^{+}}{{\pi}^{+}})}=(9.233\pm
0.002\pm 0.061\pm 0.288)\times 10^{-3},$

where the uncertainties are statistical, systematic and due to the uncertainty
of the normalisation channel, respectively. Altogether, these represent the
best measurements up to date.

## Acknowledgements

We express our gratitude to our colleagues in the CERN accelerator departments
for the excellent performance of the LHC. We thank the technical and
administrative staff at the LHCb institutes. We acknowledge support from CERN
and from the national agencies: CAPES, CNPq, FAPERJ and FINEP (Brazil); MOST
and NSFC (China); CNRS/IN2P3 (France); BMBF, DFG and MPG (Germany); INFN
(Italy); NWO (Netherlands); MNiSW and NCN (Poland); MEN/IFA (Romania); MSHE
(Russia); MinECo (Spain); SNSF and SER (Switzerland); NASU (Ukraine); STFC
(United Kingdom); NSF (USA). We acknowledge the computing resources that are
provided by CERN, IN2P3 (France), KIT and DESY (Germany), INFN (Italy), SURF
(Netherlands), PIC (Spain), GridPP (United Kingdom), RRCKI and Yandex LLC
(Russia), CSCS (Switzerland), IFIN-HH (Romania), CBPF (Brazil), PL-GRID
(Poland) and OSC (USA). We are indebted to the communities behind the multiple
open-source software packages on which we depend. Individual groups or members
have received support from AvH Foundation (Germany); EPLANET, Marie
Skłodowska-Curie Actions and ERC (European Union); ANR, Labex P2IO and OCEVU,
and Région Auvergne-Rhône-Alpes (France); Key Research Program of Frontier
Sciences of CAS, CAS PIFI, and the Thousand Talents Program (China); RFBR, RSF
and Yandex LLC (Russia); GVA, XuntaGal and GENCAT (Spain); the Royal Society
and the Leverhulme Trust (United Kingdom); Laboratory Directed Research and
Development program of LANL (USA).

## References

  * [1] B. Bhattacharya and J. L. Rosner, _Flavor symmetry and decays of charmed mesons to pairs of light pseudoscalars_ , Phys. Rev. D77 (2008) 114020
  * [2] L. L. Chau, _Quark mixing in weak interactions_ , Phys. Rep. 95 (1983) 1
  * [3] Particle Data Group, M. Tanabashi et al., _Review of particle physics_ , Phys. Rev. D98 (2018) 030001
  * [4] FOCUS collaboration, J. M. Link et al., _Measurement of the $D^{+}$ and $D^{+}_{s}$ decays into $K^{+}K^{-}K^{+}$_, Phys. Lett. B 541 (2002) 227
  * [5] LHCb collaboration, A. A. Alves Jr. et al., _The LHCb detector at the LHC_, JINST 3 (2008) S08005
  * [6] LHCb collaboration, R. Aaij et al., _LHCb detector performance_ , Int. J. Mod. Phys. A30 (2015) 1530022, arXiv:1412.6352
  * [7] M. Adinolfi et al., _Performance of the LHCb RICH detector at the LHC_, Eur. Phys. J. C73 (2013) 2431, arXiv:1211.6759
  * [8] R. Aaij et al., _The LHCb trigger and its performance in 2011_, JINST 8 (2013) P04022, arXiv:1211.3055
  * [9] T. Sjöstrand, S. Mrenna, and P. Skands, _PYTHIA 6.4 physics and manual_ , JHEP 05 (2006) 026, arXiv:hep-ph/0603175
  * [10] T. Sjöstrand, S. Mrenna, and P. Skands, _A brief introduction to PYTHIA 8.1_ , Comput. Phys. Commun. 178 (2008) 852, arXiv:0710.3820
  * [11] I. Belyaev et al., _Handling of the generation of primary events in Gauss, the LHCb simulation framework_ , J. Phys. Conf. Ser. 331 (2011) 032047
  * [12] D. J. Lange, _The EvtGen particle decay simulation package_ , Nucl. Instrum. Meth. A462 (2001) 152
  * [13] P. Golonka and Z. Was, _PHOTOS Monte Carlo: A precision tool for QED corrections in $Z$ and $W$ decays_, Eur. Phys. J. C45 (2006) 97, arXiv:hep-ph/0506026
  * [14] J. Allison et al., _Geant4 developments and applications_ , IEEE Trans. Nucl. Sci. 53 (2006) 270
  * [15] M. Clemencic et al., _The LHCb simulation application, Gauss: Design, evolution and experience_, J. Phys. Conf. Ser. 331 (2011) 032023
  * [16] R. H. Dalitz, _On the analysis of $\tau$-meson data and the nature of the $\tau$-meson_, Phil. Mag. 44 (1953) 1068
  * [17] L. Anderlini et al., _The PIDCalib package_ , LHCb-PUB-2016-021
  * [18] LHCb collaboration, R. Aaij et al., _Measurement of the track reconstruction efficiency at LHCb_ , JINST 10 (2015) P02007, arXiv:1408.1251
  * [19] LHCb collaboration, R. Aaij et al., _Dalitz plot analysis of ${{B}^{0}_{s}}\rightarrow{{\kern 1.99997pt\overline{\kern-1.99997ptD}{}}{}^{0}}{{K}^{-}}{{\pi}^{+}}$ decays_, Phys. Rev. D90 (2014) 072003, arXiv:1407.7712
  * [20] T. Skwarnicki, A study of the radiative cascade transitions between the Upsilon-prime and Upsilon resonances, PhD thesis, Institute of Nuclear Physics, Krakow, 1986, DESY-F31-86-02
  * [21] M. Pivk and F. R. Le Diberder, _sPlot: A statistical tool to unfold data distributions_ , Nucl. Instrum. Meth. A555 (2005) 356, arXiv:physics/0402083
  * [22] R. Nisius, _On the combination of correlated estimates of a physics observable_ , Eur. Phys. J. C74 (2014) 3004, arXiv:1402.4016

LHCb collaboration

R. Aaij27, C. Abellán Beteta44, B. Adeva41, M. Adinolfi48, C.A. Aidala76, Z.
Ajaltouni5, S. Akar59, P. Albicocco18, J. Albrecht10, F. Alessio42, M.
Alexander53, A. Alfonso Albero40, G. Alkhazov33, P. Alvarez Cartelle55, A.A.
Alves Jr41, S. Amato2, S. Amerio23, Y. Amhis7, L. An3, L. Anderlini17, G.
Andreassi43, M. Andreotti16, J.E. Andrews60, R.B. Appleby56, F. Archilli27, P.
d’Argent12, J. Arnau Romeu6, A. Artamonov39, M. Artuso61, K. Arzymatov37, E.
Aslanides6, M. Atzeni44, B. Audurier22, S. Bachmann12, J.J. Back50, S.
Baker55, V. Balagura7,b, W. Baldini16, A. Baranov37, R.J. Barlow56, S.
Barsuk7, W. Barter56, F. Baryshnikov72, V. Batozskaya31, B. Batsukh61, A.
Battig10, V. Battista43, A. Bay43, J. Beddow53, F. Bedeschi24, I. Bediaga1, A.
Beiter61, L.J. Bel27, S. Belin22, N. Beliy64, V. Bellee43, N. Belloli20,i, K.
Belous39, I. Belyaev34, E. Ben-Haim8, G. Bencivenni18, S. Benson27, S.
Beranek9, A. Berezhnoy35, R. Bernet44, D. Berninghoff12, E. Bertholet8, A.
Bertolin23, C. Betancourt44, F. Betti15,42, M.O. Bettler49, M. van Beuzekom27,
Ia. Bezshyiko44, S. Bhasin48, J. Bhom29, S. Bifani47, P. Billoir8, A.
Birnkraut10, A. Bizzeti17,u, M. Bjørn57, M.P. Blago42, T. Blake50, F. Blanc43,
S. Blusk61, D. Bobulska53, V. Bocci26, O. Boente Garcia41, T. Boettcher58, A.
Bondar38,w, N. Bondar33, S. Borghi56,42, M. Borisyak37, M. Borsato41, F.
Bossu7, M. Boubdir9, T.J.V. Bowcock54, C. Bozzi16,42, S. Braun12, M.
Brodski42, J. Brodzicka29, A. Brossa Gonzalo50, D. Brundu22,42, E. Buchanan48,
A. Buonaura44, C. Burr56, A. Bursche22, J. Buytaert42, W. Byczynski42, S.
Cadeddu22, H. Cai66, R. Calabrese16,g, R. Calladine47, M. Calvi20,i, M. Calvo
Gomez40,m, A. Camboni40,m, P. Campana18, D.H. Campora Perez42, L. Capriotti15,
A. Carbone15,e, G. Carboni25, R. Cardinale19,h, A. Cardini22, P. Carniti20,i,
L. Carson52, K. Carvalho Akiba2, G. Casse54, L. Cassina20, M. Cattaneo42, G.
Cavallero19, R. Cenci24,p, D. Chamont7, M.G. Chapman48, M. Charles8, Ph.
Charpentier42, G. Chatzikonstantinidis47, M. Chefdeville4, V. Chekalina37, C.
Chen3, S. Chen22, S.-G. Chitic42, V. Chobanova41, M. Chrzaszcz42, A.
Chubykin33, P. Ciambrone18, X. Cid Vidal41, G. Ciezarek42, P.E.L. Clarke52, M.
Clemencic42, H.V. Cliff49, J. Closier42, V. Coco42, J.A.B. Coelho7, J. Cogan6,
E. Cogneras5, L. Cojocariu32, P. Collins42, T. Colombo42, A. Comerma-
Montells12, A. Contu22, G. Coombs42, S. Coquereau40, G. Corti42, M. Corvo16,g,
C.M. Costa Sobral50, B. Couturier42, G.A. Cowan52, D.C. Craik58, A.
Crocombe50, M. Cruz Torres1, R. Currie52, C. D’Ambrosio42, F. Da Cunha
Marinho2, C.L. Da Silva77, E. Dall’Occo27, J. Dalseno48, A. Danilina34, A.
Davis3, O. De Aguiar Francisco42, K. De Bruyn42, S. De Capua56, M. De Cian43,
J.M. De Miranda1, L. De Paula2, M. De Serio14,d, P. De Simone18, C.T. Dean53,
D. Decamp4, L. Del Buono8, B. Delaney49, H.-P. Dembinski11, M. Demmer10, A.
Dendek30, D. Derkach37, O. Deschamps5, F. Desse7, F. Dettori54, B. Dey67, A.
Di Canto42, P. Di Nezza18, S. Didenko72, H. Dijkstra42, F. Dordei42, M.
Dorigo42,x, A. Dosil Suárez41, L. Douglas53, A. Dovbnya45, K. Dreimanis54, L.
Dufour27, G. Dujany8, P. Durante42, J.M. Durham77, D. Dutta56, R.
Dzhelyadin39, M. Dziewiecki12, A. Dziurda29, A. Dzyuba33, S. Easo51, U.
Egede55, V. Egorychev34, S. Eidelman38,w, S. Eisenhardt52, U. Eitschberger10,
R. Ekelhof10, L. Eklund53, S. Ely61, A. Ene32, S. Escher9, S. Esen27, T.
Evans59, A. Falabella15, N. Farley47, S. Farry54, D. Fazzini20,42,i, L.
Federici25, P. Fernandez Declara42, A. Fernandez Prieto41, F. Ferrari15, L.
Ferreira Lopes43, F. Ferreira Rodrigues2, M. Ferro-Luzzi42, S. Filippov36,
R.A. Fini14, M. Fiorini16,g, M. Firlej30, C. Fitzpatrick43, T. Fiutowski30, F.
Fleuret7,b, M. Fontana42, F. Fontanelli19,h, R. Forty42, V. Franco Lima54, M.
Frank42, C. Frei42, J. Fu21,q, W. Funk42, C. Färber42, M. Féo Pereira Rivello
Carvalho27, E. Gabriel52, A. Gallas Torreira41, D. Galli15,e, S. Gallorini23,
S. Gambetta52, Y. Gan3, M. Gandelman2, P. Gandini21, Y. Gao3, L.M. Garcia
Martin75, B. Garcia Plana41, J. García Pardiñas44, J. Garra Tico49, L.
Garrido40, D. Gascon40, C. Gaspar42, L. Gavardi10, G. Gazzoni5, D. Gerick12,
E. Gersabeck56, M. Gersabeck56, T. Gershon50, D. Gerstel6, Ph. Ghez4, S.
Gianì43, V. Gibson49, O.G. Girard43, L. Giubega32, K. Gizdov52, V.V.
Gligorov8, D. Golubkov34, A. Golutvin55,72, A. Gomes1,a, I.V. Gorelov35, C.
Gotti20,i, E. Govorkova27, J.P. Grabowski12, R. Graciani Diaz40, L.A. Granado
Cardoso42, E. Graugés40, E. Graverini44, G. Graziani17, A. Grecu32, R.
Greim27, P. Griffith22, L. Grillo56, L. Gruber42, B.R. Gruberg Cazon57, O.
Grünberg69, C. Gu3, E. Gushchin36, A. Guth9, Yu. Guz39,42, T. Gys42, C.
Göbel63, T. Hadavizadeh57, C. Hadjivasiliou5, G. Haefeli43, C. Haen42, S.C.
Haines49, B. Hamilton60, X. Han12, T.H. Hancock57, S. Hansmann-Menzemer12, N.
Harnew57, S.T. Harnew48, T. Harrison54, C. Hasse42, M. Hatch42, J. He64, M.
Hecker55, K. Heinicke10, A. Heister10, K. Hennessy54, L. Henry75, E. van
Herwijnen42, J. Heuel9, M. Heß69, A. Hicheur62, R. Hidalgo Charman56, D.
Hill57, M. Hilton56, P.H. Hopchev43, W. Hu67, W. Huang64, Z.C. Huard59, W.
Hulsbergen27, T. Humair55, M. Hushchyn37, D. Hutchcroft54, D. Hynds27, P.
Ibis10, M. Idzik30, P. Ilten47, K. Ivshin33, R. Jacobsson42, J. Jalocha57, E.
Jans27, A. Jawahery60, F. Jiang3, M. John57, D. Johnson42, C.R. Jones49, C.
Joram42, B. Jost42, N. Jurik57, S. Kandybei45, M. Karacson42, J.M. Kariuki48,
S. Karodia53, N. Kazeev37, M. Kecke12, F. Keizer49, M. Kelsey61, M. Kenzie49,
T. Ketel28, E. Khairullin37, B. Khanji42, C. Khurewathanakul43, K.E. Kim61, T.
Kirn9, S. Klaver18, K. Klimaszewski31, T. Klimkovich11, S. Koliiev46, M.
Kolpin12, R. Kopecna12, P. Koppenburg27, I. Kostiuk27, S. Kotriakhova33, M.
Kozeiha5, L. Kravchuk36, M. Kreps50, F. Kress55, P. Krokovny38,w, W. Krupa30,
W. Krzemien31, W. Kucewicz29,l, M. Kucharczyk29, V. Kudryavtsev38,w, A.K.
Kuonen43, T. Kvaratskheliya34,42, D. Lacarrere42, G. Lafferty56, A. Lai22, D.
Lancierini44, G. Lanfranchi18, C. Langenbruch9, T. Latham50, C. Lazzeroni47,
R. Le Gac6, A. Leflat35, J. Lefrançois7, R. Lefèvre5, F. Lemaitre42, O.
Leroy6, T. Lesiak29, B. Leverington12, P.-R. Li64, T. Li3, Z. Li61, X.
Liang61, T. Likhomanenko71, R. Lindner42, F. Lionetto44, V. Lisovskyi7, G.
Liu65, X. Liu3, D. Loh50, A. Loi22, I. Longstaff53, J.H. Lopes2, G.H.
Lovell49, D. Lucchesi23,o, M. Lucio Martinez41, A. Lupato23, E. Luppi16,g, O.
Lupton42, A. Lusiani24, X. Lyu64, F. Machefert7, F. Maciuc32, V. Macko43, P.
Mackowiak10, S. Maddrell-Mander48, O. Maev33,42, K. Maguire56, D.
Maisuzenko33, M.W. Majewski30, S. Malde57, B. Malecki29, A. Malinin71, T.
Maltsev38,w, G. Manca22,f, G. Mancinelli6, D. Marangotto21,q, J. Maratas5,v,
J.F. Marchand4, U. Marconi15, C. Marin Benito7, M. Marinangeli43, P. Marino43,
J. Marks12, P.J. Marshall54, G. Martellotti26, M. Martin6, M. Martinelli42, D.
Martinez Santos41, F. Martinez Vidal75, A. Massafferri1, M. Materok9, R.
Matev42, A. Mathad50, Z. Mathe42, C. Matteuzzi20, A. Mauri44, E. Maurice7,b,
B. Maurin43, A. Mazurov47, M. McCann55,42, A. McNab56, R. McNulty13, J.V.
Mead54, B. Meadows59, C. Meaux6, N. Meinert69, D. Melnychuk31, M. Merk27, A.
Merli21,q, E. Michielin23, D.A. Milanes68, E. Millard50, M.-N. Minard4, L.
Minzoni16,g, D.S. Mitzel12, A. Mogini8, R.D. Moise55, J. Molina Rodriguez1,y,
T. Mombächer10, I.A. Monroy68, S. Monteil5, M. Morandin23, G. Morello18, M.J.
Morello24,t, O. Morgunova71, J. Moron30, A.B. Morris6, R. Mountain61, F.
Muheim52, M. Mulder27, C.H. Murphy57, D. Murray56, A. Mödden 10, D. Müller42,
J. Müller10, K. Müller44, V. Müller10, P. Naik48, T. Nakada43, R.
Nandakumar51, A. Nandi57, T. Nanut43, I. Nasteva2, M. Needham52, N. Neri21, S.
Neubert12, N. Neufeld42, M. Neuner12, R. Newcombe55, T.D. Nguyen43, C. Nguyen-
Mau43,n, S. Nieswand9, R. Niet10, N. Nikitin35, A. Nogay71, N.S. Nolte42, D.P.
O’Hanlon15, A. Oblakowska-Mucha30, V. Obraztsov39, S. Ogilvy18, R.
Oldeman22,f, C.J.G. Onderwater70, A. Ossowska29, J.M. Otalora Goicochea2, P.
Owen44, A. Oyanguren75, P.R. Pais43, T. Pajero24,t, A. Palano14, M. Palutan18,
G. Panshin74, A. Papanestis51, M. Pappagallo52, L.L. Pappalardo16,g, W.
Parker60, C. Parkes56, G. Passaleva17,42, A. Pastore14, M. Patel55, C.
Patrignani15,e, A. Pearce42, A. Pellegrino27, G. Penso26, M. Pepe Altarelli42,
S. Perazzini42, D. Pereima34, P. Perret5, L. Pescatore43, K. Petridis48, A.
Petrolini19,h, A. Petrov71, S. Petrucci52, M. Petruzzo21,q, B. Pietrzyk4, G.
Pietrzyk43, M. Pikies29, M. Pili57, D. Pinci26, J. Pinzino42, F. Pisani42, A.
Piucci12, V. Placinta32, S. Playfer52, J. Plews47, M. Plo Casasus41, F.
Polci8, M. Poli Lener18, A. Poluektov50, N. Polukhina72,c, I. Polyakov61, E.
Polycarpo2, G.J. Pomery48, S. Ponce42, A. Popov39, D. Popov47,11, S.
Poslavskii39, C. Potterat2, E. Price48, J. Prisciandaro41, C. Prouve48, V.
Pugatch46, A. Puig Navarro44, H. Pullen57, G. Punzi24,p, W. Qian64, J. Qin64,
R. Quagliani8, B. Quintana5, B. Rachwal30, J.H. Rademacker48, M. Rama24, M.
Ramos Pernas41, M.S. Rangel2, F. Ratnikov37,ab, G. Raven28, M. Ravonel
Salzgeber42, M. Reboud4, F. Redi43, S. Reichert10, A.C. dos Reis1, F. Reiss8,
C. Remon Alepuz75, Z. Ren3, V. Renaudin7, S. Ricciardi51, S. Richards48, K.
Rinnert54, P. Robbe7, A. Robert8, A.B. Rodrigues43, E. Rodrigues59, J.A.
Rodriguez Lopez68, M. Roehrken42, S. Roiser42, A. Rollings57, V.
Romanovskiy39, A. Romero Vidal41, M. Rotondo18, M.S. Rudolph61, T. Ruf42, J.
Ruiz Vidal75, J.J. Saborido Silva41, N. Sagidova33, B. Saitta22,f, V.
Salustino Guimaraes63, C. Sanchez Gras27, C. Sanchez Mayordomo75, B. Sanmartin
Sedes41, R. Santacesaria26, C. Santamarina Rios41, M. Santimaria18,42, E.
Santovetti25,j, G. Sarpis56, A. Sarti18,k, C. Satriano26,s, A. Satta25, M.
Saur64, D. Savrina34,35, S. Schael9, M. Schellenberg10, M. Schiller53, H.
Schindler42, M. Schmelling11, T. Schmelzer10, B. Schmidt42, O. Schneider43, A.
Schopper42, H.F. Schreiner59, M. Schubiger43, M.H. Schune7, R. Schwemmer42, B.
Sciascia18, A. Sciubba26,k, A. Semennikov34, E.S. Sepulveda8, A. Sergi47,42,
N. Serra44, J. Serrano6, L. Sestini23, A. Seuthe10, P. Seyfert42, M.
Shapkin39, Y. Shcheglov33,†, T. Shears54, L. Shekhtman38,w, V. Shevchenko71,
E. Shmanin72, B.G. Siddi16, R. Silva Coutinho44, L. Silva de Oliveira2, G.
Simi23,o, S. Simone14,d, I. Skiba16, N. Skidmore12, T. Skwarnicki61, M.W.
Slater47, J.G. Smeaton49, E. Smith9, I.T. Smith52, M. Smith55, M. Soares15, l.
Soares Lavra1, M.D. Sokoloff59, F.J.P. Soler53, F.L. Souza De Almeida2, B.
Souza De Paula2, B. Spaan10, E. Spadaro Norella21,q, P. Spradlin53, F.
Stagni42, M. Stahl12, S. Stahl42, P. Stefko43, S. Stefkova55, O. Steinkamp44,
S. Stemmle12, O. Stenyakin39, M. Stepanova33, H. Stevens10, A. Stocchi7, S.
Stone61, B. Storaci44, S. Stracka24, M.E. Stramaglia43, M. Straticiuc32, U.
Straumann44, S. Strokov74, J. Sun3, L. Sun66, K. Swientek30, A. Szabelski31,
T. Szumlak30, M. Szymanski64, S. T’Jampens4, Z. Tang3, A. Tayduganov6, T.
Tekampe10, G. Tellarini16, F. Teubert42, E. Thomas42, J. van Tilburg27, M.J.
Tilley55, V. Tisserand5, M. Tobin30, S. Tolk42, L. Tomassetti16,g, D.
Tonelli24, D.Y. Tou8, R. Tourinho Jadallah Aoude1, E. Tournefier4, M.
Traill53, M.T. Tran43, A. Trisovic49, A. Tsaregorodtsev6, G. Tuci24,p, A.
Tully49, N. Tuning27,42, A. Ukleja31, A. Usachov7, A. Ustyuzhanin37, U.
Uwer12, A. Vagner74, V. Vagnoni15, A. Valassi42, S. Valat42, G. Valenti15, R.
Vazquez Gomez42, P. Vazquez Regueiro41, S. Vecchi16, M. van Veghel27, J.J.
Velthuis48, M. Veltri17,r, G. Veneziano57, A. Venkateswaran61, M. Vernet5, M.
Veronesi27, N.V. Veronika13, M. Vesterinen57, J.V. Viana Barbosa42, D.
Vieira64, M. Vieites Diaz41, H. Viemann69, X. Vilasis-Cardona40,m, A.
Vitkovskiy27, M. Vitti49, V. Volkov35, A. Vollhardt44, D. Vom Bruch8, B.
Voneki42, A. Vorobyev33, V. Vorobyev38,w, J.A. de Vries27, C. Vázquez
Sierra27, R. Waldi69, J. Walsh24, J. Wang61, M. Wang3, Y. Wang67, Z. Wang44,
D.R. Ward49, H.M. Wark54, N.K. Watson47, D. Websdale55, A. Weiden44, C.
Weisser58, M. Whitehead9, J. Wicht50, G. Wilkinson57, M. Wilkinson61, I.
Williams49, M.R.J. Williams56, M. Williams58, T. Williams47, F.F. Wilson51, M.
Winn7, J. Wishahi10, W. Wislicki31, M. Witek29, G. Wormser7, S.A. Wotton49, K.
Wyllie42, D. Xiao67, Y. Xie67, A. Xu3, M. Xu67, Q. Xu64, Z. Xu3, Z. Xu4, Z.
Yang3, Z. Yang60, Y. Yao61, L.E. Yeomans54, H. Yin67, J. Yu67,aa, X. Yuan61,
O. Yushchenko39, K.A. Zarebski47, M. Zavertyaev11,c, D. Zhang67, L. Zhang3,
W.C. Zhang3,z, Y. Zhang7, A. Zhelezov12, Y. Zheng64, X. Zhu3, V. Zhukov9,35,
J.B. Zonneveld52, S. Zucchelli15.

1Centro Brasileiro de Pesquisas Físicas (CBPF), Rio de Janeiro, Brazil
2Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil
3Center for High Energy Physics, Tsinghua University, Beijing, China
4Univ. Grenoble Alpes, Univ. Savoie Mont Blanc, CNRS, IN2P3-LAPP, Annecy,
France
5Clermont Université, Université Blaise Pascal, CNRS/IN2P3, LPC, Clermont-
Ferrand, France
6Aix Marseille Univ, CNRS/IN2P3, CPPM, Marseille, France
7LAL, Univ. Paris-Sud, CNRS/IN2P3, Université Paris-Saclay, Orsay, France
8LPNHE, Sorbonne Université, Paris Diderot Sorbonne Paris Cité, CNRS/IN2P3,
Paris, France
9I. Physikalisches Institut, RWTH Aachen University, Aachen, Germany
10Fakultät Physik, Technische Universität Dortmund, Dortmund, Germany
11Max-Planck-Institut für Kernphysik (MPIK), Heidelberg, Germany
12Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg, Heidelberg,
Germany
13School of Physics, University College Dublin, Dublin, Ireland
14INFN Sezione di Bari, Bari, Italy
15INFN Sezione di Bologna, Bologna, Italy
16INFN Sezione di Ferrara, Ferrara, Italy
17INFN Sezione di Firenze, Firenze, Italy
18INFN Laboratori Nazionali di Frascati, Frascati, Italy
19INFN Sezione di Genova, Genova, Italy
20INFN Sezione di Milano-Bicocca, Milano, Italy
21INFN Sezione di Milano, Milano, Italy
22INFN Sezione di Cagliari, Monserrato, Italy
23INFN Sezione di Padova, Padova, Italy
24INFN Sezione di Pisa, Pisa, Italy
25INFN Sezione di Roma Tor Vergata, Roma, Italy
26INFN Sezione di Roma La Sapienza, Roma, Italy
27Nikhef National Institute for Subatomic Physics, Amsterdam, Netherlands
28Nikhef National Institute for Subatomic Physics and VU University Amsterdam,
Amsterdam, Netherlands
29Henryk Niewodniczanski Institute of Nuclear Physics Polish Academy of
Sciences, Kraków, Poland
30AGH - University of Science and Technology, Faculty of Physics and Applied
Computer Science, Kraków, Poland
31National Center for Nuclear Research (NCBJ), Warsaw, Poland
32Horia Hulubei National Institute of Physics and Nuclear Engineering,
Bucharest-Magurele, Romania
33Petersburg Nuclear Physics Institute (PNPI), Gatchina, Russia
34Institute of Theoretical and Experimental Physics (ITEP), Moscow, Russia
35Institute of Nuclear Physics, Moscow State University (SINP MSU), Moscow,
Russia
36Institute for Nuclear Research of the Russian Academy of Sciences (INR RAS),
Moscow, Russia
37Yandex School of Data Analysis, Moscow, Russia
38Budker Institute of Nuclear Physics (SB RAS), Novosibirsk, Russia
39Institute for High Energy Physics (IHEP), Protvino, Russia
40ICCUB, Universitat de Barcelona, Barcelona, Spain
41Instituto Galego de Física de Altas Enerxías (IGFAE), Universidade de
Santiago de Compostela, Santiago de Compostela, Spain
42European Organization for Nuclear Research (CERN), Geneva, Switzerland
43Institute of Physics, Ecole Polytechnique Fédérale de Lausanne (EPFL),
Lausanne, Switzerland
44Physik-Institut, Universität Zürich, Zürich, Switzerland
45NSC Kharkiv Institute of Physics and Technology (NSC KIPT), Kharkiv, Ukraine
46Institute for Nuclear Research of the National Academy of Sciences (KINR),
Kyiv, Ukraine
47University of Birmingham, Birmingham, United Kingdom
48H.H. Wills Physics Laboratory, University of Bristol, Bristol, United
Kingdom
49Cavendish Laboratory, University of Cambridge, Cambridge, United Kingdom
50Department of Physics, University of Warwick, Coventry, United Kingdom
51STFC Rutherford Appleton Laboratory, Didcot, United Kingdom
52School of Physics and Astronomy, University of Edinburgh, Edinburgh, United
Kingdom
53School of Physics and Astronomy, University of Glasgow, Glasgow, United
Kingdom
54Oliver Lodge Laboratory, University of Liverpool, Liverpool, United Kingdom
55Imperial College London, London, United Kingdom
56School of Physics and Astronomy, University of Manchester, Manchester,
United Kingdom
57Department of Physics, University of Oxford, Oxford, United Kingdom
58Massachusetts Institute of Technology, Cambridge, MA, United States
59University of Cincinnati, Cincinnati, OH, United States
60University of Maryland, College Park, MD, United States
61Syracuse University, Syracuse, NY, United States
62Laboratory of Mathematical and Subatomic Physics , Constantine, Algeria,
associated to 2
63Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio), Rio de
Janeiro, Brazil, associated to 2
64University of Chinese Academy of Sciences, Beijing, China, associated to 3
65South China Normal University, Guangzhou, China, associated to 3
66School of Physics and Technology, Wuhan University, Wuhan, China, associated
to 3
67Institute of Particle Physics, Central China Normal University, Wuhan,
Hubei, China, associated to 3
68Departamento de Fisica , Universidad Nacional de Colombia, Bogota, Colombia,
associated to 8
69Institut für Physik, Universität Rostock, Rostock, Germany, associated to 12
70Van Swinderen Institute, University of Groningen, Groningen, Netherlands,
associated to 27
71National Research Centre Kurchatov Institute, Moscow, Russia, associated to
34
72National University of Science and Technology ”MISIS”, Moscow, Russia,
associated to 34
73National Research University Higher School of Economics, Moscow, Russia,
Moscow, Russia
74National Research Tomsk Polytechnic University, Tomsk, Russia, associated to
34
75Instituto de Fisica Corpuscular, Centro Mixto Universidad de Valencia -
CSIC, Valencia, Spain, associated to 40
76University of Michigan, Ann Arbor, United States, associated to 61
77Los Alamos National Laboratory (LANL), Los Alamos, United States, associated
to 61

aUniversidade Federal do Triângulo Mineiro (UFTM), Uberaba-MG, Brazil
bLaboratoire Leprince-Ringuet, Palaiseau, France
cP.N. Lebedev Physical Institute, Russian Academy of Science (LPI RAS),
Moscow, Russia
dUniversità di Bari, Bari, Italy
eUniversità di Bologna, Bologna, Italy
fUniversità di Cagliari, Cagliari, Italy
gUniversità di Ferrara, Ferrara, Italy
hUniversità di Genova, Genova, Italy
iUniversità di Milano Bicocca, Milano, Italy
jUniversità di Roma Tor Vergata, Roma, Italy
kUniversità di Roma La Sapienza, Roma, Italy
lAGH - University of Science and Technology, Faculty of Computer Science,
Electronics and Telecommunications, Kraków, Poland
mLIFAELS, La Salle, Universitat Ramon Llull, Barcelona, Spain
nHanoi University of Science, Hanoi, Vietnam
oUniversità di Padova, Padova, Italy
pUniversità di Pisa, Pisa, Italy
qUniversità degli Studi di Milano, Milano, Italy
rUniversità di Urbino, Urbino, Italy
sUniversità della Basilicata, Potenza, Italy
tScuola Normale Superiore, Pisa, Italy
uUniversità di Modena e Reggio Emilia, Modena, Italy
vMSU - Iligan Institute of Technology (MSU-IIT), Iligan, Philippines
wNovosibirsk State University, Novosibirsk, Russia
xSezione INFN di Trieste, Trieste, Italy
yEscuela Agrícola Panamericana, San Antonio de Oriente, Honduras
zSchool of Physics and Information Technology, Shaanxi Normal University
(SNNU), Xi’an, China
aaPhysics and Micro Electronic College, Hunan University, Changsha City, China
abNational Research University Higher School of Economics, Moscow, Russia
†Deceased


# How Will Your Tweet Be Received? Predicting the Sentiment Polarity of Tweet
Replies††thanks: ∗ Corresponding author. ††thanks: A major part of the
research was carried out by the first two authors.

Soroosh Tayebi Arasteh∗23, Mehrpad Monajem2, Vincent Christlein2, Philipp
Heinrich2,
Anguelos Nicolaou2, Hamidreza Naderi Boldaji2, Mahshad Lotfinia4 and Stefan
Evert2
<EMAIL_ADDRESS>2Friedrich-Alexander-Universität Erlangen-Nürnberg,
Germany  3Harvard Medical School, United States  4Sharif University of
Technology, Iran

###### Abstract

Twitter sentiment analysis, which often focuses on predicting the polarity of
tweets, has attracted increasing attention over the last years, in particular
with the rise of deep learning (DL). In this paper, we propose a new task:
predicting the predominant sentiment among (first-order) replies to a given
tweet. Therefore, we created ReTweet, a large dataset of tweets and replies
manually annotated with sentiment labels. As a strong baseline, we propose a
two-stage DL-based method: first, we create automatically labeled training
data by applying a standard sentiment classifier to tweet replies and
aggregating its predictions for each original tweet; our rationale is that
individual errors made by the classifier are likely to cancel out in the
aggregation step. Second, we use the automatically labeled data for supervised
training of a neural network to predict reply sentiment from the original
tweets. The resulting classifier is evaluated on the new ReTweet dataset,
showing promising results, especially considering that it has been trained
without any manually labeled data. Both the dataset and the baseline
implementation are publicly available.

##  I Introduction

Twitter is one of the dominant social media platforms and has become a major
source for local and global news as well as political argumentation. Sentiment
analysis is a research area in Natural Language Processing that aims to
identify the opinions, attitudes or emotions expressed in a text document or
sentence, often with respect to a particular topic [1]. Twitter sentiment
analysis usually involves detecting whether a given tweet expresses a
_positive_ , _negative_ , or _neutral_ sentiment polarity, but has also looked
at subparts of the tweet (words, phrases) or at aggregated sentiment in tweet
collections, as well as sentiment towards a specific topic, e. g., a person,
product, or event [2].

Nakov et al. [3] first introduced the task of message- and expression-level
sentiment analysis on tweets; since then many further challenges have emerged.
Rosenthal et al. [4] introduced the task of sentiment towards a topic.
Variations of the task include more fine-grained classification (e. g., a
five-point sentiment classification with labels _highly positive, positive,
neutral, negative, highly negative_) and quantification of the overall
distribution towards a given topic [5]. There is also work on sentiment
analysis of figurative language [6] and detecting stance in tweets [7]. Other
tasks such as out-of-context sentiment intensity of words and phrases [8] and
implicit event polarity [9] are closely related, but do not target tweets.

In this paper, we introduce a new challenge in Twitter sentiment analysis:
given only an original tweet, predict the predominant polarity among all
first-order replies this source tweet will receive, i. e., whether the
audience will overall react positively, negatively, or neutrally to the tweet.
It is challenging to collect large amounts of training data for this task
because, for each training item, annotators need to read all first-order
replies in order to label the source tweet. Therefore, we propose to generate
automatically labeled training data by applying a standard Twitter sentiment
classifier to sets of replies and aggregating the results. These data can then
be used for standard supervised training of the final classifier. For
evaluation purposes, we have created ReTweet, a publicly
available111Accessible under this link:
https://www.kaggle.com/soroosharasteh/retweet/. test set with manual
annotation, and use it for validating our proposed method222The source code is
accessible below: https://github.com/starasteh/retweet/..

##  II Standard Tweet Sentiment Classification

For our method presented in Sec. III, we need a state-of-the-art sentiment
classifier for tweets. As a baseline, we evaluate models for task 10.B of
SemEval 2015 [4], which is the task of three-way classification according to
_positive_ , _negative_ , or _neutral_ sentiment for English tweets. We use
$51\,875$ manually labeled tweets from the SemEval datasets [3, 10, 4, 5, 2]
as training data and $9155$ tweets as validation data. We combined individual
training and test sets to ensure there is no overlap.

Our own classifier is based on a recurrent neural network. The tweets are
tokenized using SpaCy [11]. The embedding layer uses a Vocabulary of the 50K
the most frequent words in the training data; out-of-vocabulary words are
replaced with a special “unknown” token (<unk>). The embeddings are
initialised with a 200-dimensional GloVe [12] model trained on 27 billion
tweets.

Figure 1: The network architecture.

The main building-blocks of our model are two Bi-directional Long-Short Term
Memory Units (BiLSTM) [13] with the hidden and cell states of length 256.
First of all, a dropout layer is applied to the input embedding vectors. Then
the result is fed into a BiLSTM followed by another dropout layer and then the
second BiLSTM followed by another dropout layer. Finally, a fully connected
layer (followed by a SoftMax) is used for the classification. Fig. 1 shows our
selected architecture. All dropout layers use a drop probability of 0.5. As
loss, we chose the weighted cross-entropy with inverted class frequencies of
the training data as loss weights to counteract the imbalanced dataset. The
model is optimized using the Adam optimizer, learning rate of $10^{-4}$ and
weight decay of $10^{-5}$.

To compare our tweet classifier with other state-of-the-art models, we test
our trained model on the 2014 and 2015 test sets of SemEval, which consist of
$1852$ and $2389$ manually labeled tweets, respectively. In order to be
consistent with historical editions of this competition, we use the average
$F_{1}$ score of the positive and negative classes as the metric of interest,
see Equ. 1. Note that, except accuracy, which is calculated on the whole set,
the rest of the evaluation scores are all calculated according to Equ. 1
throughout this paper, i. e., discarding the neutral class results:

$\text{score}=\frac{\text{score}_{\text{pos}}+\text{score}_{\text{neg}}}{2}$
(1)

Tab. I shows that our tweet classifier is in the range of state-of-the-art and
conventional machine learning models and we can proceed to utilize it for
automatic labeling.

TABLE I: Evaluation of sentiment classifiers on the SemEval 2014 and 2015 official test sets. The scores present the average F1 scores of the positive and negative classes according to Equ. 1. Numbers taken from [14, 10, 4]. System | 2014 | 2015
---|---|---
Logistic regression on 1-3 grams baseline | 0.629 | 0.586
The 9th place of the original task | 0.674 | 0.620
The winner of the original task | 0.709 | 0.648
The state-of-the-art | 0.748 | 0.688
_Our sentiment polarity classifier_ (cf. Sec. II) | 0.652 | 0.624

##  III Methodology

Having a trained sentiment classifier, we can start solving the main problem
of predicting overall sentiment polarity of the replies. In the automatic
labeling process, given a source tweet and a set of corresponding replies, the
trained sentiment classifier is first applied to all the replies as inputs,
while ignoring the source tweet. Consequently, the network predicts a
sentiment category for each reply, which are collected in a vector. A
heuristic algorithm (cf. algorithm 1) is used to derive a final label for the
overall sentiment of replies to the source tweet based on this label vector.

Result: One label for every tweet based on its replies.

for  _all tweets_ do

if  _#total neutral replies $>$ $85\text{\,}\mathrm{\char 37\relax}$ of the
total replies_ then

Label = neutral ;

else

if  _#total positives replies $>$ # $1.5\times$ of total negative replies_
then

Label = positive ;

else if  _#total negative replies $>$ # $1.6\times$ of total positives
replies_ then

Label = negative ;

else

Label = neutral ;

Algorithm 1 Automatic label assignment strategy.

Some justifications are needed to support our method, as there is no
theoretical motivation or empirical evidence behind the chosen ratios.

Firstly, we generally expect most of the polarity predictions in the vector to
be neutral as not all replies can be expected to carry positive/negative
sentiment. Therefore, the overall reply sentiment is only considered to be
neutral if the proportion of neutral replies exceeds a fairly high threshold,
here set to $85\text{\,}\mathrm{\char 37\relax}$. Secondly, we observe that
usually every tweet triggers both positive and negative replies as there are
different people with different mindsets commenting on the tweet. In other
words: there is usually no exclusively positive or negative reply vector for a
tweet, even for seemingly clear-cut cases such as, e. g., birthday wishes. We
therefore assign positive overall sentiment if there are at least $1.5$ times
as many positive replies as negative replies, and vice versa. Note that the
exact values were fine-tuned based on various observations on the behavior of
the network in different situations. Finally, we assign _neutral_ to a tweet
when the total numbers of positive and negative replies were close to each
other, as none of them is dominant and there are two sets of contradicting
replies, making it difficult to opt for one side.333Note that despite the name
the _neutral_ class thus also contains tweets that trigger both positive and
negative replies.

##  IV ReTweet: Dataset of Tweets & Overall Predominant Sentiment of their
Replies

As there is no publicly available dataset for our purposes, we downloaded data
using the Twitter API. We collected $35\,072$ training tweets together with a
total of $1\,519\,504$ replies, and used $10\text{\,}\mathrm{\char 37\relax}$
of them for validation. To download all of the replies to a tweet, the Search
API could be used. However, it is limited to $75\,000$ requests per hour,
which causes the mining and downloading process to be slow. Furthermore, using
the Twitter API, there is no possibility of obtaining completely random data.
Therefore, we tried to make the procedure as random as possible by utilizing
two different strategies for data selection, using the collected data in an
intermixed manner.

Our first strategy is based on a sample of English tweets obtained by
filtering the Twitter stream via a list of cultural keywords [15]. This list
consists of 147 words that are deemed to play a “pivotal role in discussions
of culture and society” [15, xvii], covering diverse words such as
_aesthetics_ , _environment_ , _feminism_ , _power_ , or _tourism_. We
extracted all tweets in 2019 having a minimum of 20 first-order replies in the
dataset. The data come with an obvious caveat: both the source tweet and
replies must contain at least one word from the keyword list.444This makes it
highly unlikely that the list of replies for any given source is exhaustive,
i. e., there will usually be many other first-order replies to the source
tweet that are not included in the dataset.

As our second approach, we use the GetOldTweets3 [16] library to download all
the replies corresponding to every tweet. To increase randomness, instead of
using the same list of keywords as in the first strategy, we manually selected
keywords that are likely to trigger long discussions, such as _Coronavirus_
and _football_ , or strong opinions, such as _birthday_ , _war_ , or _racism_.
In addition to the constraint that every tweet must have at least 20 first-
order replies, we only accepted tweets (both source tweets and replies)
consisting of at least 20 tokens. This is because our tweet classifier (cf.
Sec. II) is optimized based on the message-level classification paradigm and
thus relies on a sufficient number of words in the message.

A total of $5015$ tweets with all of their corresponding first-order replies,
collected by a combination of the two collection strategies, were given to
three trained annotators. Annotators were asked to judge intuitively whether
the replies taken together indicate an overall positive, negative or neutral
reaction to the tweet and decide on _one_ final sentiment label for the
replies. Annotators were not shown the respective source tweet in order to
avoid a bias based on their own reaction to the tweet. _Positive_ and
_negative_ polarity is defined in the same way as for the SemEval tasks. In
addition to tweets which actually have a neutral overall polarity among their
first-order replies, annotators were asked to also assign neutral label to a
tweet when neither positive nor negative replies have predominance on the
other one. For the final dataset, we only chose tweets for which all
annotators judged unanimously, leading to a data set of $1519$ source tweets
manually labeled with the overall sentiment of their replies (excluding
sentiment of the source tweet itself).

During the manual annotation process, we observed that most disagreements
between the annotators were related to neutral labels, i. e., either one
annotator classified the overall label of tweet as neutral and the other two
not, or two classified as neutral and the other one not. Although the
annotators were trained in a same way, the concept of neutral label was still
not as clear as positive/negative to them. This can be because human brain,
when trying to annotate such phrases, expects more to face phrases with
positive or negative polarities than neutral and is more sensitive towards
these polarities. Thus, chance of having different opinions between negative
and positive is very lower than between neutral and positive/negative. Label
distribution in our final test set is as follows: $32.5\text{\,}\mathrm{\char
37\relax}$ positive, $37.5\text{\,}\mathrm{\char 37\relax}$ negative, and
$30.0\text{\,}\mathrm{\char 37\relax}$ neutral; the training set contains
$23.5\text{\,}\mathrm{\char 37\relax}$ positive, $32.5\text{\,}\mathrm{\char
37\relax}$ negative, and $44.0\text{\,}\mathrm{\char 37\relax}$ neutral
labels.

##  V Experiments and Results on ReTweet

We utilize the same data pre-processing, training process, and architecture as
for the standard Twitter sentiment classifier (Sec. II); the only differences
being that the lengths of hidden and cell states are 300, vocabulary is built
choosing 750K of the most common words of the training data, and a learning
rate of $9\times 10^{-5}$ with a decay of $10^{-4}$ is chosen.

Tab. II shows the confusion matrix and evaluation results of the final
classifier on the ReTweet test set. Even though the standard sentiment
polarity classifier (Sec. II) used for automatic labelling of the training
data (Sec. III), our classifier for response sentiment prediction already
achieves highly promising results, especially considering the general
difficulty of message-level sentiment analysis and the indirect nature of the
ReTweet task.

In order to explore the correlation between tweet sentiments and their reply
sentiments, we additionally create a “direct” baseline classifier by
predicting the sentiment of the original tweet and assuming that its replies
will have the same predominant sentiment. Tab. III shows the evaluation
metrics of our proposed system versus the “direct” baseline on the ReTweet
test set. We observe a highly significant increase in $F_{1}$ score by $15.4$
percent points.

Accuracy | F1 score | Recall | Precision
---|---|---|---
61.7 | 71.9 | 79.1 | 66.1
(a) (b)

TABLE II: II(a) evaluation results of the final classifier on the ReTweet test
set. F1 score, recall, and precision are calculated based on Equ. 1. All
results given in percent. II(b) Detailed confusion matrix.

Furthermore, we follow the state-of-the-art model of standard tweet sentiment
polarity classification proposed by Cliche [14], implementing a Convolutional
Neural Network-based model and averaging its predictions with those of our
BiLSTM model.

Firstly, each input embedding vector is fed to three 1D convolutional layers
with filter sizes of 3, 4, and 5, respectively, and the Rectified Linear Unit
(ReLU) is used as activation function. All filters have the same output
feature map dimension of 200. Moreover, in order to remove the dependency of
the feature maps on the length, we additionally feed them to 1D max pooling
layers with kernel sizes of equal to sentence lengths of each activation.
Finally, the results are concatenated, leading to a 600-dimensional feature
map for every input, followed by a dropout, and a fully connected layer
(followed by a SoftMax) is used for the classification.

We train the CNN independently with similar optimization and data pre-
processing parameters as for our BiLSTM model. The evaluation results on the
ReTweet test set (see Tab. IV), unexpectedly, show only a very slight
improvement over the BiLSTM model, which indicates the fact that having
already a good enough classifier architecture, the choice of the label
assignment algorithm plays a more vital role than maximizing the capacity of
the classifier architecture in this task.

Furthermore, Tab. II(b) shows that most of the wrong classification results
have actually neutral reply polarity. It is because, using algorithm 1, we
make our system sensitive to positive/negative sentiments in order to boost
our performance of interest. And it does not hurt us to lose to a certain
extent the neutral prediction performance as long as we are improving in our
primary goal.

TABLE III: Comparison between our proposed method (Sec. III) and directly classifying the tweet and assuming its replies will have the same dominant polarity as itself. F1 score, recall, and precision are calculated based on Equ. 1. All results given in percent. Metric | Direct | Proposed
---|---|---
Accuracy | 49.7 | 61.7
F1 score | 56.5 | 71.9
Recall | 54.0 | 79.1
Precision | 60.0 | 66.1
TABLE IV: Evaluation results of the ensemble model (cf. Sec. V) on the ReTweet test set. F1 score, recall, and precision are calculated based on Equ. 1. All results given in percent. Accuracy | F1 score | Recall | Precision
---|---|---|---
62.0 | 73.2 | 81.0 | 66.8

##  VI Conclusion And Discussion

In this paper, we introduced a new challenge: prediction of the overall
polarity of first-order replies to an English source tweet. As a baseline for
solving this task, we proposed a Deep Learning approach. We first predicted
the overall message-level polarity of the tweets, i. e., whether it is
received positive, negative, or neutral, which led to creation of automatic
labels for replies, and subsequently trained a network that predicts the
predominant reaction of the tweet audience. We used a heuristic algorithm for
the final label selection in the automatic labeling process.

Additionally, we created ReTweet, the first public dataset for sentiment
prediction of first-order Twitter replies. The evaluation results of our
approach on ReTweet show its effectiveness. Although utilizing more training
data may still somewhat increase the performance, whether a reply is positive
or negative, except for extreme cases, may not purely be determined only by
the content of the tweet itself but in a rather dynamic environment.
Therefore, our proposed method serves as an upper-bound baseline in prediction
of the polarity of predominant first-reaction to a given tweet.

As mentioned in Sec. IV, the ReTweet is extracted in a way to be as random as
possible, since we do not consider the dynamic environment and rather only do
content-based prediction. For future work, it would be beneficial to extend
the method by taking some prior knowledge into consideration. Time of posting,
status quo when the tweet is posted, etc., can be influential factors.
Consequently, more task-specific data collection strategies will be explored
to extend ReTweet.

## References

  * [1] B. Liu, “Sentiment analysis and opinion mining,” _Synthesis Lectures on Human Language Technologies_ , vol. 5, no. 1, pp. 1–167, 2012.
  * [2] S. Rosenthal, N. Farra, and P. Nakov, “SemEval-2017 task 4: Sentiment analysis in twitter,” in _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)_. Vancouver, Canada: Association for Computational Linguistics, Aug. 2017, pp. 502–518.
  * [3] P. Nakov, S. Rosenthal, Z. Kozareva, V. Stoyanov, A. Ritter, and T. Wilson, “SemEval-2013 task 2: Sentiment analysis in twitter,” in _Second Joint Conference on Lexical and Computational Semantics (*SEM), Volume 2: Proceedings of the Seventh International Workshop on Semantic Evaluation (SemEval 2013)_. Atlanta, Georgia, USA: Association for Computational Linguistics, Jun. 2013, pp. 312–320.
  * [4] S. Rosenthal, S. M. Mohammad, P. Nakov, A. Ritter, S. Kiritchenko, and V. Stoyanov, “SemEval-2015 task 10: Sentiment analysis in twitter,” in _Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval-2015)_. Denver, Colorado: Association for Computational Linguistics, 2015, pp. 451–463.
  * [5] P. Nakov, A. Ritter, S. Rosenthal, F. Sebastiani, and V. Stoyanov, “SemEval-2016 task 4: Sentiment analysis in twitter,” in _Proceedings of the 10th International Workshop on Semantic Evaluation_. San Diego, California: Association for Computational Linguistics, 2016, pp. 1–18.
  * [6] A. Ghosh, G. Li, T. Veale, P. Rosso, E. Shutova, J. Barnden, and A. Reyes, “SemEval-2015 task 11: Sentiment analysis of figurative language in twitter,” in _Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)_. Denver, Colorado: Association for Computational Linguistics, 2015, pp. 470–478.
  * [7] S. Mohammad, S. Kiritchenko, P. Sobhani, X. Zhu, and C. Cherry, “SemEval-2016 task 6: Detecting stance in tweets,” in _Proceedings of the 10th International Workshop on Semantic Evaluation_. San Diego, California: Association for Computational Linguistics, 2016, pp. 31–41.
  * [8] S. Kiritchenko, S. Mohammad, and M. Salameh, “SemEval-2016 task 7: Determining sentiment intensity of English and Arabic phrases,” in _Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)_. San Diego, California: Association for Computational Linguistics, 2016, pp. 42–51.
  * [9] I. Russo, T. Caselli, and C. Strapparava, “SemEval-2015 task 9: CLIPEval implicit polarity of events,” in _Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)_. Denver, Colorado: Association for Computational Linguistics, 2015, pp. 443–450.
  * [10] S. Rosenthal, P. Nakov, A. Ritter, and V. Stoyanov, “SemEval-2014 task 9: Sentiment analysis in twitter,” in _Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval-2014)_. Dublin, Ireland: Association for Computational Linguistics, 2014, pp. 73–80.
  * [11] M. Honnibal and M. Johnson, “An improved non-monotonic transition system for dependency parsing,” in _Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing_. Lisbon, Portugal: Association for Computational Linguistics, Sep. 2015, pp. 1373–1378.
  * [12] J. Pennington, R. Socher, and C. D. Manning, “Glove: Global vectors for word representation,” in _EMNLP_ , 2014.
  * [13] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” _Neural computation_ , vol. 9, no. 8, pp. 1735–1780, 1997.
  * [14] M. Cliche, “BB_twtr at SemEval-2017 task 4: Twitter sentiment analysis with CNNs and LSTMs,” in _Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)_. Vancouver, Canada: Association for Computational Linguistics, Aug. 2017, pp. 573–580.
  * [15] T. Bennett, L. Grossberg, and M. Morris, _New Keywords: A Revised Vocabulary of Culture and Society_. Blackwell, 2005.
  * [16] D. Mottl and J. Henriquel, “Getoldtweets3,” 2019. [Online]. Available: https://pypi.org/project/GetOldTweets3/