EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH (CERN)

 CERN-EP-2019-050 LHCb-PAPER-2019-007 June 12, 2019

Observation of an excited $B_{c}^{+}$ state

LHCb collaboration†††Authors are listed at the end of this Letter.

Using $pp$ collision data corresponding to an integrated luminosity of
$8.5\,\mathrm{fb}^{-1}$ recorded by the LHCb experiment at centre-of-mass
energies of $\sqrt{s}=7$, $8$ and $13\mathrm{\,Te\kern-1.00006ptV}$, the
observation of an excited $B_{c}^{+}$ state in the $B_{c}^{+}\pi^{+}\pi^{-}$
invariant-mass spectrum is reported. The observed peak has a mass of
$6841.2\pm 0.6{\,\rm(stat)\,}\pm 0.1{\,\rm(syst)\,}\pm
0.8\,(B_{c}^{+})\mathrm{\,MeV}/c^{2}$, where the last uncertainty is due to
the limited knowledge of the $B_{c}^{+}$ mass. It is consistent with
expectations of the $B_{c}^{*}(2^{3}S_{1})^{+}$ state reconstructed without
the low-energy photon from the $B_{c}^{*}(1^{3}S_{1})^{+}\rightarrow
B_{c}^{+}\gamma$ decay following $B_{c}^{*}(2^{3}S_{1})^{+}\rightarrow
B_{c}^{*}(1^{3}S_{1})^{+}\pi^{+}\pi^{-}$. A second state is seen with a global
(local) statistical significance of $2.2\,\sigma$ ($3.2\,\sigma$) and a mass
of $6872.1\pm 1.3{\,\rm(stat)\,}\pm 0.1{\,\rm(syst)\,}\pm
0.8\,(B_{c}^{+})\mathrm{\,MeV}/c^{2}$, and is consistent with the
$B_{c}(2^{1}S_{0})^{+}$ state. These mass measurements are the most precise to
date.

Published in Phys. Rev. Lett. 122 (2019) 232001

© 2024 CERN for the benefit of the LHCb collaboration. CC-BY-4.0 licence.

The $B_{c}$ meson family is unique in the Standard Model as its states are
formed from two heavy quarks of different flavours. The spectrum of masses of
$B_{c}$ mesons can reveal information on heavy-quark dynamics and improve the
understanding of the strong interaction. Specifically, it provides tests of
nonrelativistic quark-potential models [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12,
13], which have been successfully applied to quarkonium, since the $B_{c}$
family shares properties with both the charmonium and bottomonium systems. The
$B_{c}$ family is predicted to have a rich spectroscopy by various potential
models [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13] and lattice quantum
chromodynamics [12]. However, the $B_{c}$ mesons are much less explored
compared to quarkonia due to the small production rate, since their
predominant production mechanism requires the production of both
${c}{\overline{c}}$ and ${b}{\overline{b}}$ pairs. The ground state meson,
${{B}_{c}^{+}}$, was first observed by the CDF experiment [14, *Abe:1998fb] at
the Tevatron collider. Knowledge of the properties of the ${{B}_{c}^{+}}$
meson has been greatly advanced by the LHCb experiment with the measurement of
the ${{B}_{c}^{+}}$ mass, lifetime and production rate [16, 17, 18, 19, 20,
21], and the discovery and precise measurement of the branching fractions of
several new decay channels [22, 23, 17, 24, 25, 26, 27, 28, 29, 30, 31].
Charge conjugation is implied throughout this Letter.

Excited ${{B}_{c}^{+}}$ states that lie below the threshold for decay into a
beauty and charm meson pair are expected to have decay widths smaller than a
few hundred  keV [3, 4]. Depending on its mass, an excited ${{B}_{c}^{+}}$
resonance may undergo either cascade radiative or pionic decays to the
${{B}_{c}^{+}}$ state, which decays weakly. The second $S$-wave $B_{c}$ state
occurs as either a pseudoscalar $(0^{-})$ or a vector $(1^{-})$ spin state,
i.e., the singlet ${{B}_{c}(2^{1}S_{0})^{+}}$ or the triplet
${{B}_{c}(2^{3}S_{1})^{+}}$. The ${{B}_{c}(2^{1}S_{0})^{+}}$ and
${{B}_{c}(2^{3}S_{1})^{+}}$ states are denoted as ${{B}_{c}(2S)^{+}}$ and
${{B}_{c}^{*}(2S)^{+}}$, respectively. The ${{B}_{c}(2S)^{+}}$ state decays
directly to ${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$, while the
${{B}_{c}^{*}(2S)^{+}}$ state decays to
${{B}_{c}(1^{3}S_{1})^{+}}{{\pi}^{+}}{{\pi}^{-}}$, followed by the
${{B}_{c}(1^{3}S_{1})^{+}}\rightarrow{{B}_{c}^{+}}\gamma$ electromagnetic
transition. The low-energy photon produced in this decay is not considered in
this analysis, since the reconstruction efficiency for such photons is too low
to be useful with the current data sample. The ${{B}_{c}(1^{3}S_{1})^{+}}$
state is denoted as ${{B}_{c}^{*+}}$ hereafter. The transitions among the
${{B}_{c}^{(*)}(2S)^{+}}$ and $B_{c}^{(*)+}$ states are illustrated in Fig. 1.
Decays of both ${{B}_{c}^{(*)}(2S)^{+}}$ states produce a narrow peak in the
${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$ invariant-mass spectrum [32, 33],
however, the ${{B}_{c}^{*}(2S)^{+}}$ state peaks at
${M({{B}_{c}^{*}(2S)^{+}})_{\rm rec}=M({{B}_{c}^{*}(2S)^{+}})-\Delta
M({{B}_{c}^{*+}})}$ due to the missing photon, where $\Delta
M({{B}_{c}^{*+}})$ is the mass difference between the intermediate state
${{B}_{c}^{*+}}$ and the ${{B}_{c}^{+}}$ meson. Since the ${{B}_{c}^{*+}}$
state has not been observed yet, the quantity $\Delta M({{B}_{c}^{*+}})$ is
unknown and the value of $M({{B}_{c}^{*}(2S)^{+}})$ can not be determined with
this technique at the moment. Taking into account the unreconstructed photon,
the mass difference between the two peaks in the
${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$ mass distribution originating from the
two ${{B}_{c}^{(*)}(2S)^{+}}$ states,
$M({{B}_{c}(2S)^{+}})-M({{B}_{c}^{*}(2S)^{+}})_{\rm rec}$, is predicted to be
in the range 11 to 53$\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ [1, 2, 3, 4, 5, 6,
7, 8, 9, 10, 11, 12, 13]. The production cross-section of the
${{B}_{c}^{*}(2S)^{+}}$ state is predicted to be twice as large as that of the
${{B}_{c}(2S)^{+}}$ state [34, 35, 8, 32], while the branching fractions of
the decays
${{B}_{c}(2S)^{+}}\\!\rightarrow{{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$ and
${{B}_{c}^{*}(2S)^{+}}\\!\rightarrow{{B}_{c}^{*+}}{{\pi}^{+}}{{\pi}^{-}}$ are
expected to be similar [35, 8].

Figure 1: Transitions among the ${{B}_{c}^{(*)}(2S)^{+}}$ and $B_{c}^{(*)+}$
states.

With the large samples of ${{B}_{c}^{+}}$ mesons produced at the Large Hadron
Collider, the ATLAS collaboration first reported the observation of a signal
in the ${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$ mass distribution peaking at a
value of $6842\pm 4\text{\,(stat)}\pm
5\text{\,(syst)}\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ using $pp$ collision
data at $\sqrt{s}=7$ and $8\text{\,Te\kern-1.00006ptV}$ corresponding to a
luminosity of $24\text{\,fb}^{-1}$ [36]. Due to large mass resolution and low
signal yield, no determination could be made as to whether the observed peak
was either the ${{B}_{c}(2S)^{+}}$, the ${{B}_{c}^{*}(2S)^{+}}$ state, or a
combination of the two states. The LHCb experiment also performed a search for
excited ${{B}_{c}^{+}}$ states in the ${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$
mass distribution using $pp$ collision data at centre-of-mass energy of
$\sqrt{s}=8\text{\,Te\kern-1.00006ptV}$, corresponding to an integrated
luminosity of $2\text{\,fb}^{-1}$. No evidence of any signal was found [37].
Recently, the CMS collaboration reported the observation of the
${{B}_{c}(2S)^{+}}$ and ${{B}_{c}^{*}(2S)^{+}}$ states [38], in which the mass
of the ${{B}_{c}(2S)^{+}}$ state and the mass difference between the two peaks
were measured to be $6871.0\pm 1.2\text{\,(stat)}\pm 0.8\text{\,(syst)}\pm
0.8({{B}_{c}^{+}})\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ and $29.0\pm
1.5\text{\,(stat)}\pm 0.7\text{\,(syst)}\text{\,Me\kern-1.00006ptV\\!/}c^{2}$,
respectively. The third uncertainty is due to the limited knowledge of the
${{B}_{c}^{+}}$ mass.

This Letter presents an updated search for excited $B_{c}$ mesons in the
${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$ mass distribution. The analysis makes
use of Run 1 and Run 2 data collected by the LHCb experiment from 2011 to 2018
at centre-of-mass energies of $\sqrt{s}=7$, $8$ and
$13\text{\,Te\kern-1.00006ptV}$, corresponding to integrated luminosities of
about $1.0$, $2.0$ and $5.5\text{\,fb}^{-1}$, respectively.

The LHCb detector [39, 40] is a single-arm forward spectrometer covering the
pseudorapidity range $2<\eta<5$, designed for the study of particles
containing $b$ and/or $c$ quarks. The detector elements that are particularly
relevant to this analysis are: a silicon-strip vertex detector surrounding the
$pp$ interaction region that allows $c$ and $b$ hadrons to be identified from
their characteristically long flight distance; a tracking system that provides
a measurement of the momentum, $p$, of charged particles with a relative
uncertainty that varies from 0.5% at low momentum to 1.0% at
200$\text{\,Ge\kern-1.00006ptV\\!/}c$; and two ring-imaging Cherenkov
detectors that are able to discriminate between different species of charged
hadrons. The minimum distance of a track to a primary vertex (PV), the impact
parameter (IP), is measured with a resolution of
$(15+29/p_{\mathrm{T}})\,\upmu\text{m}$, where $p_{\mathrm{T}}$ is the
component of the momentum transverse to the beam, in
$\text{\,Ge\kern-1.00006ptV\\!/}c$. The online event selection is performed by
a trigger, which consists of a hardware stage, based on information from the
calorimeter and muon systems, followed by a software stage, which applies a
full event reconstruction. At the hardware stage, events are required to have
at least one muon with high transverse momentum, $p_{\mathrm{T}}$, or a hadron
with high transverse energy. At the software stage, two muon tracks or three
charged tracks are required to have high $p_{\mathrm{T}}$ and to form a
secondary vertex with a significant displacement from the interaction point.
The momentum scale in data is calibrated using the
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ and $B^{+}$ mesons [41] with
well-known masses.

Simulated samples are used to model the signal behaviour. In the simulation,
$pp$ collisions are generated using Pythia 6 [42] with a specific LHCb
configuration [43]. The generator BcVegPy [34] is used to simulate the
production of ${{B}_{c}^{+}}$ mesons. Decays of unstable particles are
described by EvtGen [44], in which final-state radiation is generated using
Photos [45]. The interaction of the generated particles with the detector, and
its response, are implemented using the Geant4 toolkit [46] as described in
Ref. [47].

To form the ${{B}_{c}^{(*)}(2S)^{+}}$ candidates, first the intermediate
${{B}_{c}^{+}}$ state is reconstructed from the
${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}$ decay. The ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}$ candidates are reconstructed with a pair of oppositely charged
particles identified as muons. The muons are required to have
$p_{\mathrm{T}}>550\text{\,Me\kern-1.00006ptV\\!/}c$ and good track-fit
quality. They are required to form a common decay vertex with an invariant
mass in the range $[3040,3140]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$,
corresponding to approximately six times the
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ mass resolution. The
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ candidate is combined with a
charged pion to form the ${{B}_{c}^{+}}$ candidate. Each particle is
associated to the PV that has the smallest value of $\chi^{2}_{\text{IP}}$,
where $\chi^{2}_{\text{IP}}$ is defined as the difference in the vertex-fit
$\chi^{2}$ of a given PV reconstructed with and without the particle under
consideration. The pion must have
$p_{\mathrm{T}}>1000\text{\,Me\kern-1.00006ptV\\!/}c$, good track-fit quality,
and be inconsistent with originating from any PV. The ${{B}_{c}^{+}}$
candidate is required to have a good-quality vertex, a trajectory consistent
with coming from its associated PV, and a decay time larger than
$0.2\text{\,ps}$.

To further suppress background, a boosted decision tree (BDT) [48, 49]
classifier is used, as done in the ${{B}_{c}^{+}}$ production measurement
[21]. The input variables of the BDT classifier are taken to be the
$p_{\mathrm{T}}$ of each muon, the ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}$ meson and the charged pion; the decay length, decay time and vertex-
fit $\chi^{2}$ of the ${{B}_{c}^{+}}$ meson; and the $\chi^{2}_{\text{IP}}$ of
the muons, the pion, the ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$
meson and the ${{B}_{c}^{+}}$ meson with respect to the associated PV. The BDT
classifier is trained using signal candidates from simulation and background
candidates from the upper sideband of the
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}}$ mass distribution
in data, corresponding to the range
$[6370,6600]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$. The BDT threshold is chosen
to maximise $S/\sqrt{S+B}$, where $S$ and $B$ are the expected yields of
signal and background in the range $M({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}})\in[6251,6301]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$,
respectively. This mass window corresponds to around four times the resolution
of $M({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}})$. To improve
the signal-to-background ratio in the ${{B}_{c}^{(*)}(2S)^{+}}$ search, the
transverse momentum of the ${{B}_{c}^{+}}$ meson is required to be larger than
$10\text{\,Ge\kern-1.00006ptV\\!/}c$.

An unbinned maximum-likelihood fit is performed to the
$M({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}})$ distribution.
To improve the mass resolution, the mass
$M({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}})$ is calculated
by constraining the ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ mass to
its known value [50] and the ${{B}_{c}^{+}}$ meson to originate from the
associated PV [51]. The signal component is described by a Gaussian function
with asymmetric power-law tails [52]. The parameters of the tails are
determined from the simulation, while the mean and width of the Gaussian
function are left free in the fit. The combinatorial background is modelled
with an exponential function. The contamination from the Cabibbo-suppressed
decay ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}K^{+}$, with the kaon misidentified as a pion, is modelled by a
Gaussian function with asymmetric power-law tails. The parameters of this
Gaussian function are fixed according to the simulation, except that the mean
is constrained relative to that of the
${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}}$ signal. The invariant-mass distribution of the
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}}$ candidates is
shown in Fig. 2. The ${{B}_{c}^{+}}$ signal yield is $3785\pm 73$. The fitted
${{B}_{c}^{+}}$ mass and mass resolution are $6273.7\pm
0.3\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ and $15.1\pm
0.3\text{\,Me\kern-1.00006ptV\\!/}c^{2}$, respectively.

Figure 2:  Invariant-mass distribution of the selected ${{B}_{c}^{+}}$
candidates. The fit results are overlaid.

To reconstruct the ${{B}_{c}^{(*)}(2S)^{+}}$ candidates, ${{B}_{c}^{+}}$
candidates with $M({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}})\in[6200,6320]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ are
combined with a pair of oppositely charged particles identified as pions.
These pion candidates are required to originate from the PV, and each have
$p_{\mathrm{T}}>300\text{\,Me\kern-1.00006ptV\\!/}c$,
$p>1500\text{\,Me\kern-1.00006ptV\\!/}c$, and a good track-fit quality. The
${{B}_{c}^{(*)}(2S)^{+}}$ candidate is required to have a good vertex-fit
quality. To improve the mass resolution, a fit [51] is performed in which the
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ and ${{B}_{c}^{+}}$ masses
are constrained to their known values [50] and the daughters of the
${{B}_{c}^{(*)}(2S)^{+}}$ meson are required to point to the associated PV.
The $\chi^{2}$ per number of degrees of freedom of this fit must be smaller
than nine. The value of
$M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})-M({{B}_{c}^{+}})-M({{\pi}^{+}}{{\pi}^{-}})$
is required to be smaller than $200\text{\,Me\kern-1.00006ptV\\!/}c^{2}$. To
ensure that the selection does not produce any artificial peaks in the
$M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})$ spectrum, the same requirements are
applied to a same-sign sample, constructed from
${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{+}}$ or ${{B}_{c}^{+}}{{\pi}^{-}}{{\pi}^{-}}$
combinations. The efficiency of the selections is found to change smoothly
with the invariant mass $M({{B}_{c}^{+}}\pi\pi)$ and no peaks are seen in the
same-sign sample.

The $M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})$ distribution in the data sample
after all the selections are applied is shown in Fig. 3, with those of the
same-sign sample and a sample drawn from the ${{B}_{c}^{+}}$ sidebands
($M({{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip
2.0mu}}{{\pi}^{+}})\in[6150,6200]\cup[6320,6550]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$)
superimposed for comparison. The same-sign and ${{B}_{c}^{+}}$ mass sideband
distributions are scaled to the opposite-sign distribution in the sideband
region,
$M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})\in[6735,6825]\cup[6895,6975]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$.
The $M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})$ distribution presents an obvious
peak at approximately $6840\text{\,Me\kern-1.00006ptV\\!/}c^{2}$, and a less
significant structure at about $6870\text{\,Me\kern-1.00006ptV\\!/}c^{2}$.

Figure 3: Invariant-mass $M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})$
distributions for the data and same-sign samples with the distribution of the
${{B}_{c}^{+}}$ mass sidebands overlaid.

The masses and yields of the ${{B}_{c}^{(*)}(2S)^{+}}$ peaks are determined
using an unbinned maximum-likelihood fit to the distribution of the mass
difference, $\Delta M\equiv
M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})-M({{B}_{c}^{+}})$, to eliminate the
dependence on the reconstructed ${{B}_{c}^{+}}$ mass. Here the mass
$M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})$ is calculated with no constraint on
the ${{B}_{c}^{+}}$ mass, but only constraining the
${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ mass to its known value [50]
and requiring the ${{B}_{c}^{(*)}(2S)^{+}}$ meson to come from the associated
PV [51]. Each ${{B}_{c}^{(*)}(2S)^{+}}$ peak is modelled by a Gaussian
function with asymmetric power-law tails [52]. The tail parameters are fixed
to the values determined from simulation, while the Gaussian mean and width
are treated as free parameters. The combinatorial background is described by a
second-order polynomial function.

The fit to the $\Delta M$ distribution is shown in Fig. 4, and the results are
summarised in Table 1. The ${{B}_{c}^{*}(2S)^{+}}$ signal yield is determined
to be $51\pm 10\text{\,(stat)}$, corresponding to a local statistical
significance of $6.8\,\sigma$. The significance is evaluated with a
likelihood-based test, in which the likelihood distribution of the background-
only hypothesis is obtained using pseudoexperiments [53]. The yield of the
${{B}_{c}(2S)^{+}}$ state is $24\pm 9\text{\,(stat)}$ with a local statistical
significance of $3.2\,\sigma$. The Gaussian widths of the two peaks are
consistent with the expectation of negligible resonance widths. The mass
difference between the two peaks is measured to be $31.1\pm
1.4\text{\,(stat)}\text{\,Me\kern-1.00006ptV\\!/}c^{2}$. Taking the known
${{B}_{c}^{+}}$ mass, $M({{B}_{c}^{+}})=6274.9\pm
0.8\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ [54], the quantities
$M({{B}_{c}^{*}(2S)^{+}})_{\rm rec}$ and $M({{B}_{c}(2S)^{+}})$ are determined
to be $6841.1\pm 0.6\text{\,(stat)}\pm
0.8\,({{B}_{c}^{+}})\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ and $6872.1\pm
1.3\text{\,(stat)}\pm
0.8\,({{B}_{c}^{+}})\text{\,Me\kern-1.00006ptV\\!/}c^{2}$, respectively. The
second uncertainty is due to the limited knowledge of the ${{B}_{c}^{+}}$
mass. After considering the look-elsewhere effect in the predicted mass
regions [55],
$M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})\in[6790,6895]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$
for the ${{B}_{c}^{*}(2S)^{+}}$ state, and
$M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})\in[6845,6895]\text{\,Me\kern-1.00006ptV\\!/}c^{2}$
for the ${{B}_{c}(2S)^{+}}$ state [1, 2, 3, 4, 5, 6, 7, 8, 9, 56, 10], the
global statistical significances of the two states are determined to be
$6.3\,\sigma$ and $2.2\,\sigma$, respectively.

Figure 4: Distribution of $\Delta M=M({{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}})-M({{B}_{c}^{+}})$ with the fit results overlaid. The same-sign distribution has been normalized to the data in the ${{B}_{c}^{(*)}(2S)^{+}}$ sideband region.  Table 1:  Results of the fit to the $\Delta M$ distribution. Uncertainties are statistical only. | ${{B}_{c}^{*}(2S)^{+}}$ | ${{B}_{c}(2S)^{+}}$
---|---|---
Signal yield | $~{}~{}~{}51\pm 10$ | $~{}\,24\pm 9$
Peak $\Delta M$ value (${\rm Me\kern-1.00006ptV\\!/}c^{2}$) | $566.2\pm 0.6$ | $597.2\pm 1.3$
Resolution (${\rm Me\kern-1.00006ptV\\!/}c^{2}$) | $~{}~{}~{}2.6\pm 0.5$ | $~{}~{}~{}2.5\pm 1.0$
Local significance | $6.8\,\sigma$ | $3.2\,\sigma$
Global significance | $6.3\,\sigma$ | $2.2\,\sigma$

Several sources of systematic uncertainty on the determination of the mass
difference $\Delta M$ are studied. The dominant contribution is from the
uncertainty on the momentum scale, which is due to imperfections in the
description of the magnetic field and the imperfect alignment of the
subdetectors. The uncertainty of the momentum calibration is estimated using
other particles, such as ${{K}^{0}_{\mathrm{S}}}$ and $\mathchar 28935\relax$
mesons, and leads to an uncertainty of
$0.12\text{\,Me\kern-1.00006ptV\\!/}c^{2}$ on the $\Delta M$ measurements. The
unreconstructed photon emitted in the ${{B}_{c}^{*}(2S)^{+}}$ decay chain
could be an additional source of systematic uncertainty. Studies on simulated
events show that the missing photon introduces a small bias, and a correction
of $+0.08\text{\,Me\kern-1.00006ptV\\!/}c^{2}$, with negligible uncertainty,
is applied to the fitted value of the ${{B}_{c}^{*}(2S)^{+}}$ mass peak. All
other systematic uncertainties are negligible and are briefly described as
follows. The effects of the imperfect modelling of the signal and background
components are estimated by using alternative models. The alternative model
for the signal peaks uses Hypatia functions [57], while for the background the
alternative model consists of a sum of two threshold functions, each of the
form $(\Delta M-m_{t})^{p}\times e^{-C\cdot(\Delta M-m_{t})}$, where $p$ and
$C$ are free parameters, and $m_{t}$ represents the threshold, which is taken
to be $2m_{\pi^{\pm}}$. The changes in $\Delta M$ obtained with the
alternative models are found to be negligible. The effect of final-state
radiation is also studied with simulated events and the associated uncertainty
on the fitted mass values is found to be negligible. The total systematic
uncertainty on $\Delta M$ for both the ${{B}_{c}(2S)^{+}}$ and
${{B}_{c}^{*}(2S)^{+}}$ states of $0.12\text{\,Me\kern-1.00006ptV\\!/}c^{2}$
is fully correlated, and therefore cancels in the mass difference of the two
peaks.

In conclusion, using $pp$ collision data collected by the LHCb experiment at
centre-of-mass energies of $\sqrt{s}=7$, $8$ and
$13\text{\,Te\kern-1.00006ptV}$, corresponding to an integrated luminosity of
$8.5\text{\,fb}^{-1}$, a peaking structure consistent with the
${{B}_{c}^{*}(2S)^{+}}$ state is observed in the
${{B}_{c}^{+}}{{\pi}^{+}}{{\pi}^{-}}$ mass spectrum with a global (local)
statistical significance of $6.3\,\sigma$ ($6.8\,\sigma$). The mass associated
with the ${{B}_{c}^{*}(2S)^{+}}$ state, for which the low-energy photon in the
intermediate decay ${{B}_{c}^{*+}}\rightarrow{{B}_{c}^{+}}\gamma$ is not
reconstructed, is measured to be

$6841.2\pm 0.6\text{\,(stat)}\pm 0.1\text{\,(syst)}\pm
0.8\,({{B}_{c}^{+}})\text{\,Me\kern-1.00006ptV\\!/}c^{2},$

where the last uncertainty is due to the limited knowledge of the
${{B}_{c}^{+}}$ mass. It is equal to $M({{B}_{c}^{*}(2S)^{+}})_{\rm
rec}=M({{B}_{c}^{*}(2S)^{+}})-\left(M({{B}_{c}^{*+}})-M({{B}_{c}^{+}})\right)$.
The mass difference between the ${{B}_{c}^{*}(2S)^{+}}$ and ${{B}_{c}^{*+}}$
state is determined to be $566.3\pm 0.6\text{\,(stat)}\pm
0.1\text{\,(syst)}\text{\,Me\kern-1.00006ptV\\!/}c^{2}$. The data also show a
hint for a second structure consistent with the ${{B}_{c}(2S)^{+}}$ state with
a global (local) statistical significance of $2.2\,\sigma$ ($3.2\,\sigma$).
Assuming this peak is due to the ${{B}_{c}(2S)^{+}}$ state, its mass is
measured to be

$6872.1\pm 1.3\text{\,(stat)}\pm 0.1\text{\,(syst)}\pm
0.8\,({{B}_{c}^{+}})\text{\,Me\kern-1.00006ptV\\!/}c^{2}.$

The mass difference between the ${{B}_{c}(2S)^{+}}$ and ${{B}_{c}^{+}}$ state
is $597.2\pm 1.3\text{\,(stat)}\pm
0.1\text{\,(syst)}\text{\,Me\kern-1.00006ptV\\!/}c^{2}$. The mass difference
of the two ${{B}_{c}^{(*)}(2S)^{+}}$ peaks is determined to be

$31.0\pm 1.4\text{\,(stat)}\pm
0.0\text{\,(syst)}\text{\,Me\kern-1.00006ptV\\!/}c^{2},$

in which both the uncertainty from the ${{B}_{c}^{+}}$ mass and the systematic
uncertainty cancel. The mass measurements are the most precise to date, and
are consistent with the results from the CMS collaboration [38]. They are also
within the range of the theoretical predictions [1, 2, 3, 4, 5, 6, 7, 8, 9,
10, 12, 11, 13].

## Acknowledgements

We thank Chao-Hsi Chang and Xing-Gang Wu for frequent and interesting
discussions on the production of the $B_{c}$ mesons. We express our gratitude
to our colleagues in the CERN accelerator departments for the excellent
performance of the LHC. We thank the technical and administrative staff at the
LHCb institutes. We acknowledge support from CERN and from the national
agencies: CAPES, CNPq, FAPERJ and FINEP (Brazil); MOST and NSFC (China);
CNRS/IN2P3 (France); BMBF, DFG and MPG (Germany); INFN (Italy); NWO
(Netherlands); MNiSW and NCN (Poland); MEN/IFA (Romania); MSHE (Russia);
MinECo (Spain); SNSF and SER (Switzerland); NASU (Ukraine); STFC (United
Kingdom); NSF (USA). We acknowledge the computing resources that are provided
by CERN, IN2P3 (France), KIT and DESY (Germany), INFN (Italy), SURF
(Netherlands), PIC (Spain), GridPP (United Kingdom), RRCKI and Yandex LLC
(Russia), CSCS (Switzerland), IFIN-HH (Romania), CBPF (Brazil), PL-GRID
(Poland) and OSC (USA). We are indebted to the communities behind the multiple
open-source software packages on which we depend. Individual groups or members
have received support from AvH Foundation (Germany); EPLANET, Marie
Skłodowska-Curie Actions and ERC (European Union); ANR, Labex P2IO and OCEVU,
and Région Auvergne-Rhône-Alpes (France); Key Research Program of Frontier
Sciences of CAS, CAS PIFI, and the Thousand Talents Program (China); RFBR, RSF
and Yandex LLC (Russia); GVA, XuntaGal and GENCAT (Spain); the Royal Society
and the Leverhulme Trust (United Kingdom); Laboratory Directed Research and
Development program of LANL (USA).

## References

  * [1] S. S. Gershtein et al., _Production cross-section and spectroscopy of $B_{c}$ mesons_, Sov. J. Nucl. Phys. 48 (1988) 327, Yad. Fiz. 48 (1988) 515
  * [2] Y.-Q. Chen and Y.-P. Kuang, _Improved QCD-motivated heavy quark potentials with explicit $\Lambda_{\overline{\rm MS}}$ dependence_, Phys. Rev. D46 (1992) 1165, Erratum ibid. D47 (1993) 350
  * [3] E. J. Eichten and C. Quigg, _Mesons with beauty and charm: Spectroscopy_ , Phys. Rev. D49 (1994) 5845, arXiv:hep-ph/9402210
  * [4] S. S. Gershtein, V. V. Kiselev, A. K. Likhoded, and A. V. Tkabladze, _$B_{c}$ spectroscopy_, Phys. Rev. D51 (1995) 3613, arXiv:hep-ph/9406339
  * [5] S. N. Gupta and J. M. Johnson, _$B_{c}$ spectroscopy in a quantum-chromodynamic potential model_, Phys. Rev. D53 (1996) 312, arXiv:hep-ph/9511267
  * [6] L. P. Fulcher, _Phenomenological predictions of the properties of the $B_{c}$ system_, Phys. Rev. D60 (1999) 074006, arXiv:hep-ph/9806444
  * [7] D. Ebert, R. N. Faustov, and V. O. Galkin, _Properties of heavy quarkonia and $B_{c}$ mesons in the relativistic quark model_, Phys. Rev. D67 (2003) 014027, arXiv:hep-ph/0210381
  * [8] S. Godfrey, _Spectroscopy of $B_{c}$ mesons in the relativized quark model_, Phys. Rev. D70 (2004) 054017, arXiv:hep-ph/0406228
  * [9] K.-W. Wei and X.-H. Guo, _Mass spectra of doubly heavy mesons in Regge phenomenology_ , Phys. Rev. D81 (2010) 076005
  * [10] A. Abd El-Hady, J. R. Spence, and J. P. Vary, _Radiative decays of ${B}_{c}$ mesons in a Bethe-Salpeter model_, Phys. Rev. D71 (2005) 034006, arXiv:hep-ph/0603139
  * [11] J. Zeng, J. W. Van Orden, and W. Roberts, _Heavy mesons in a relativistic model_ , Phys. Rev. D52 (1995) 5229, arXiv:hep-ph/9412269
  * [12] C. T. H. Davies et al., _$B_{c}$ spectroscopy from lattice QCD_, Phys. Lett. B382 (1996) 131, arXiv:hep-lat/9602020
  * [13] R. J. Dowdall, C. T. H. Davies, T. C. Hammant, and R. R. Horgan, _Precise heavy-light meson masses and hyperfine splittings from lattice QCD including charm quarks in the sea_ , Phys. Rev. D86 (2012) 094510, arXiv:1207.5149
  * [14] CDF collaboration, F. Abe et al., _Observation of the $B_{c}$ meson in $p\bar{p}$ collisions at $\sqrt{s}=1.8$ TeV_, Phys. Rev. Lett. 81 (1998) 2432, arXiv:hep-ex/9805034
  * [15] CDF collaboration, F. Abe et al., _Observation of $B_{c}$ mesons in $p\bar{p}$ collisions at $\sqrt{s}=1.8$ TeV_, Phys. Rev. D58 (1998) 112004, arXiv:hep-ex/9804014
  * [16] LHCb collaboration, R. Aaij et al., _Measurements of ${{B}_{c}^{+}}$ production and mass with the ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}}$ decay_, Phys. Rev. Lett. 109 (2012) 232001, arXiv:1209.5634
  * [17] LHCb collaboration, R. Aaij et al., _Observation of ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{D}^{+}_{s}}$ and ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{D}^{*+}_{s}}$ decays_, Phys. Rev. D87 (2013) 112012, arXiv:1304.4530
  * [18] LHCb collaboration, R. Aaij et al., _First observation of a baryonic ${{B}_{c}^{+}}$ decay_, Phys. Rev. Lett. 113 (2014) 152003, arXiv:1408.0971
  * [19] LHCb collaboration, R. Aaij et al., _Measurement of the ${{B}_{c}^{+}}$ meson lifetime using ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{\mu^{+}}{{\nu}_{\mu}}X$ decays_, Eur. Phys. J. C74 (2014) 2839, arXiv:1401.6932
  * [20] LHCb collaboration, R. Aaij et al., _Measurement of the lifetime of the ${{B}_{c}^{+}}$ meson using the ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}}$ decay mode_, Phys. Lett. B742 (2015) 29, arXiv:1411.6899
  * [21] LHCb collaboration, R. Aaij et al., _Measurement of ${{B}_{c}^{+}}$ production in proton-proton collisions at $\sqrt{s}=8$ TeV_, Phys. Rev. Lett. 114 (2015) 132001, arXiv:1411.2943
  * [22] LHCb collaboration, R. Aaij et al., _First observation of the decay ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}}{{\pi}^{-}}{{\pi}^{+}}$_, Phys. Rev. Lett. 108 (2012) 251802, arXiv:1204.0079
  * [23] LHCb collaboration, R. Aaij et al., _Observation of the decay ${{B}_{c}^{+}}\rightarrow{\psi{(2S)}}{{\pi}^{+}}$_, Phys. Rev. D87 (2013) 071103(R), arXiv:1303.1737
  * [24] LHCb collaboration, R. Aaij et al., _First observation of the decay ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{+}}$_, JHEP 09 (2013) 075, arXiv:1306.6723
  * [25] LHCb collaboration, R. Aaij et al., _Observation of the decay ${{B}_{c}^{+}}\rightarrow{{B}^{0}_{s}}{{\pi}^{+}}$_, Phys. Rev. Lett. 111 (2013) 181801, arXiv:1308.4544
  * [26] LHCb collaboration, R. Aaij et al., _Observation of the decay ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{+}}{{K}^{-}}{{\pi}^{+}}$_, JHEP 11 (2013) 094, arXiv:1309.0587
  * [27] LHCb collaboration, R. Aaij et al., _Evidence for the decay ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}3{{\pi}^{+}}2{{\pi}^{-}}$_, JHEP 05 (2014) 148, arXiv:1404.0287
  * [28] LHCb collaboration, R. Aaij et al., _Measurement of the branching fraction ratio ${\mathcal{B}}({{B}_{c}^{+}}\rightarrow{\psi{(2S)}}{{\pi}^{+}})/{\mathcal{B}}({{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}})$_, Phys. Rev. D92 (2015) 057007, arXiv:1507.03516
  * [29] LHCb collaboration, R. Aaij et al., _Measurement of the ratio of branching fractions ${\mathcal{B}}({{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{+}})/{\mathcal{B}}({{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\pi}^{+}})$_, JHEP 09 (2016) 153, arXiv:1607.06823
  * [30] LHCb collaboration, R. Aaij et al., _Observation of ${{B}_{c}^{+}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}D^{(\ast)}K^{(\ast)}$ decays_, Phys. Rev. D95 (2017) 032005, arXiv:1612.07421
  * [31] LHCb collaboration, R. Aaij et al., _Observation of ${{B}_{c}^{+}}\rightarrow{{D}^{0}}{{K}^{+}}$ decays_, Phys. Rev. Lett. 118 (2017) 111803, arXiv:1701.01856
  * [32] Y.-N. Gao et al., _Experimental prospects of the $B_{c}$ studies of the LHCb experiment_, Chin. Phys. Lett. 27 (2010) 061302
  * [33] A. Berezhnoy and A. Likhoded, _The observation possibility of $B_{c}$ excitations at LHC_, PoS QFTHEP2013 (2013) 051, arXiv:1307.5993
  * [34] C.-H. Chang, X.-Y. Wang, and X.-G. Wu, _BCVEGPY2.2: A newly upgraded version for hadronic production of the meson Bc and its excited states_ , Comput. Phys. Commun. 197 (2015) 335, arXiv:1507.05176
  * [35] I. P. Gouz et al., _Prospects for the $B_{c}$ studies at LHCb_, Phys. Atom. Nucl. 67 (2004) 1559, arXiv:hep-ph/0211432
  * [36] ATLAS collaboration, G. Aad et al., _Observation of an excited $B_{c}^{\pm}$ meson state with the ATLAS detector_, Phys. Rev. Lett. 113 (2014) 212004, arXiv:1407.1032
  * [37] LHCb collaboration, R. Aaij et al., _Search for excited ${B}_{c}^{+}$ states_, JHEP 01 (2018) 138, arXiv:1712.04094
  * [38] CMS collaboration, A. M. Sirunyan et al., _Observation of two excited B ${}^{+}_{\mathrm{c}}$ states and measurement of the B${}^{+}_{\mathrm{c}}$(2S) mass in pp collisions at $\sqrt{s}=$ 13 TeV_, arXiv:1902.00571
  * [39] LHCb collaboration, A. A. Alves Jr. et al., _The LHCb detector at the LHC_, JINST 3 (2008) S08005
  * [40] LHCb collaboration, R. Aaij et al., _LHCb detector performance_ , Int. J. Mod. Phys. A30 (2015) 1530022, arXiv:1412.6352
  * [41] LHCb collaboration, R. Aaij et al., _Measurement of ${b}$-hadron masses_, Phys. Lett. B708 (2012) 241, arXiv:1112.4896
  * [42] T. Sjöstrand, S. Mrenna, and P. Skands, _PYTHIA 6.4 physics and manual_ , JHEP 05 (2006) 026, arXiv:hep-ph/0603175
  * [43] I. Belyaev et al., _Handling of the generation of primary events in Gauss, the LHCb simulation framework_ , J. Phys. Conf. Ser. 331 (2011) 032047
  * [44] D. J. Lange, _The EvtGen particle decay simulation package_ , Nucl. Instrum. Meth. A462 (2001) 152
  * [45] P. Golonka and Z. Was, _PHOTOS Monte Carlo: A precision tool for QED corrections in $Z$ and $W$ decays_, Eur. Phys. J. C45 (2006) 97, arXiv:hep-ph/0506026
  * [46] Geant4 collaboration, J. Allison et al., _Geant4 developments and applications_ , IEEE Trans. Nucl. Sci. 53 (2006) 270
  * [47] M. Clemencic et al., _The LHCb simulation application, Gauss: Design, evolution and experience_, J. Phys. Conf. Ser. 331 (2011) 032023
  * [48] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and regression trees, Wadsworth international group, Belmont, California, USA, 1984
  * [49] Y. Freund and R. E. Schapire, _A decision-theoretic generalization of on-line learning and an application to boosting_ , J. Comput. Syst. Sci. 55 (1997) 119
  * [50] Particle Data Group, C. Patrignani et al., _Review of particle physics_ , Chin. Phys. C40 (2016) 100001
  * [51] W. D. Hulsbergen, _Decay chain fitting with a Kalman filter_ , Nucl. Instrum. Meth. A552 (2005) 566, arXiv:physics/0503191
  * [52] T. Skwarnicki, A study of the radiative cascade transitions between the Upsilon-prime and Upsilon resonances, PhD thesis, Institute of Nuclear Physics, Krakow, 1986, DESY-F31-86-02
  * [53] G. Cowan, K. Cranmer, E. Gross, and O. Vitells, _Asymptotic formulae for likelihood-based tests of new physics_ , Eur. Phys. J. C71 (2011) 1554, Erratum ibid. C73 (2013) 2501, arXiv:1007.1727
  * [54] Particle Data Group, M. Tanabashi et al., _Review of particle physics_ , Phys. Rev. D98 (2018) 030001
  * [55] E. Gross and O. Vitells, _Trial factors for the look elsewhere effect in high energy physics_ , Eur. Phys. J. C70 (2010) 525, arXiv:1005.1891
  * [56] A. K. Rai and P. C. Vinodkumar, _Properties of $B_{c}$ meson_, Pramana 66 (2006) 953, arXiv:hep-ph/0606194
  * [57] D. Martínez Santos and F. Dupertuis, _Mass distributions marginalized over per-event errors_ , Nucl. Instrum. Meth. A764 (2014) 150, arXiv:1312.5000

LHCb collaboration

R. Aaij29, C. Abellán Beteta46, B. Adeva43, M. Adinolfi50, C.A. Aidala77, Z.
Ajaltouni7, S. Akar61, P. Albicocco20, J. Albrecht12, F. Alessio44, M.
Alexander55, A. Alfonso Albero42, G. Alkhazov35, P. Alvarez Cartelle57, A.A.
Alves Jr43, S. Amato2, Y. Amhis9, L. An19, L. Anderlini19, G. Andreassi45, M.
Andreotti18, J.E. Andrews62, F. Archilli29, J. Arnau Romeu8, A. Artamonov41,
M. Artuso63, K. Arzymatov39, E. Aslanides8, M. Atzeni46, B. Audurier24, S.
Bachmann14, J.J. Back52, S. Baker57, V. Balagura9,b, W. Baldini18,44, A.
Baranov39, R.J. Barlow58, S. Barsuk9, W. Barter57, M. Bartolini21, F.
Baryshnikov73, V. Batozskaya33, B. Batsukh63, A. Battig12, V. Battista45, A.
Bay45, F. Bedeschi26, I. Bediaga1, A. Beiter63, L.J. Bel29, S. Belin24, N.
Beliy4, V. Bellee45, N. Belloli22,i, K. Belous41, G. Bencivenni20, E. Ben-
Haim10, S. Benson29, S. Beranek11, A. Berezhnoy37, R. Bernet46, D.
Berninghoff14, E. Bertholet10, A. Bertolin25, C. Betancourt46, F. Betti17,e,
M.O. Bettler51, Ia. Bezshyiko46, S. Bhasin50, J. Bhom31, M.S. Bieker12, S.
Bifani49, P. Billoir10, A. Birnkraut12, A. Bizzeti19,u, M. Bjørn59, M.P.
Blago44, T. Blake52, F. Blanc45, S. Blusk63, D. Bobulska55, V. Bocci28, O.
Boente Garcia43, T. Boettcher60, A. Bondar40,x, N. Bondar35, S. Borghi58,44,
M. Borisyak39, M. Borsato14, M. Boubdir11, T.J.V. Bowcock56, C. Bozzi18,44, S.
Braun14, M. Brodski44, J. Brodzicka31, A. Brossa Gonzalo52, D. Brundu24,44, E.
Buchanan50, A. Buonaura46, C. Burr58, A. Bursche24, J. Buytaert44, W.
Byczynski44, S. Cadeddu24, H. Cai67, R. Calabrese18,g, S. Cali20, R.
Calladine49, M. Calvi22,i, M. Calvo Gomez42,m, A. Camboni42,m, P. Campana20,
D.H. Campora Perez44, L. Capriotti17,e, A. Carbone17,e, G. Carboni27, R.
Cardinale21, A. Cardini24, P. Carniti22,i, K. Carvalho Akiba2, G. Casse56, M.
Cattaneo44, G. Cavallero21, R. Cenci26,p, M.G. Chapman50, M. Charles10,44, Ph.
Charpentier44, G. Chatzikonstantinidis49, M. Chefdeville6, V. Chekalina39, C.
Chen3, S. Chen24, S.-G. Chitic44, V. Chobanova43, M. Chrzaszcz44, A.
Chubykin35, P. Ciambrone20, X. Cid Vidal43, G. Ciezarek44, F. Cindolo17,
P.E.L. Clarke54, M. Clemencic44, H.V. Cliff51, J. Closier44, V. Coco44, J.A.B.
Coelho9, J. Cogan8, E. Cogneras7, L. Cojocariu34, P. Collins44, T. Colombo44,
A. Comerma-Montells14, A. Contu24, G. Coombs44, S. Coquereau42, G. Corti44,
C.M. Costa Sobral52, B. Couturier44, G.A. Cowan54, D.C. Craik60, A.
Crocombe52, M. Cruz Torres1, R. Currie54, C.L. Da Silva78, E. Dall’Occo29, J.
Dalseno43,50, C. D’Ambrosio44, A. Danilina36, P. d’Argent14, A. Davis58, O. De
Aguiar Francisco44, K. De Bruyn44, S. De Capua58, M. De Cian45, J.M. De
Miranda1, L. De Paula2, M. De Serio16,d, P. De Simone20, J.A. de Vries29, C.T.
Dean55, W. Dean77, D. Decamp6, L. Del Buono10, B. Delaney51, H.-P.
Dembinski13, M. Demmer12, A. Dendek32, D. Derkach74, O. Deschamps7, F. Desse9,
F. Dettori24, B. Dey68, A. Di Canto44, P. Di Nezza20, S. Didenko73, H.
Dijkstra44, F. Dordei24, M. Dorigo26,y, A.C. dos Reis1, A. Dosil Suárez43, L.
Douglas55, A. Dovbnya47, K. Dreimanis56, L. Dufour44, G. Dujany10, P.
Durante44, J.M. Durham78, D. Dutta58, R. Dzhelyadin41,†, M. Dziewiecki14, A.
Dziurda31, A. Dzyuba35, S. Easo53, U. Egede57, V. Egorychev36, S.
Eidelman40,x, S. Eisenhardt54, U. Eitschberger12, R. Ekelhof12, L. Eklund55,
S. Ely63, A. Ene34, S. Escher11, S. Esen29, T. Evans61, A. Falabella17, C.
Färber44, N. Farley49, S. Farry56, D. Fazzini22,i, M. Féo44, P. Fernandez
Declara44, A. Fernandez Prieto43, F. Ferrari17,e, L. Ferreira Lopes45, F.
Ferreira Rodrigues2, S. Ferreres Sole29, M. Ferro-Luzzi44, S. Filippov38, R.A.
Fini16, M. Fiorini18,g, M. Firlej32, C. Fitzpatrick44, T. Fiutowski32, F.
Fleuret9,b, M. Fontana44, F. Fontanelli21,h, R. Forty44, V. Franco Lima56, M.
Frank44, C. Frei44, J. Fu23,q, W. Funk44, E. Gabriel54, A. Gallas Torreira43,
D. Galli17,e, S. Gallorini25, S. Gambetta54, Y. Gan3, M. Gandelman2, P.
Gandini23, Y. Gao3, L.M. Garcia Martin76, J. García Pardiñas46, B. Garcia
Plana43, J. Garra Tico51, L. Garrido42, D. Gascon42, C. Gaspar44, G. Gazzoni7,
D. Gerick14, E. Gersabeck58, M. Gersabeck58, T. Gershon52, D. Gerstel8, Ph.
Ghez6, V. Gibson51, O.G. Girard45, P. Gironella Gironell42, L. Giubega34, K.
Gizdov54, V.V. Gligorov10, C. Göbel65, D. Golubkov36, A. Golutvin57,73, A.
Gomes1,a, I.V. Gorelov37, C. Gotti22,i, E. Govorkova29, J.P. Grabowski14, R.
Graciani Diaz42, L.A. Granado Cardoso44, E. Graugés42, E. Graverini46, G.
Graziani19, A. Grecu34, R. Greim29, P. Griffith24, L. Grillo58, L. Gruber44,
B.R. Gruberg Cazon59, C. Gu3, E. Gushchin38, A. Guth11, Yu. Guz41,44, T.
Gys44, T. Hadavizadeh59, C. Hadjivasiliou7, G. Haefeli45, C. Haen44, S.C.
Haines51, P.M. Hamilton62, Q. Han68, X. Han14, T.H. Hancock59, S. Hansmann-
Menzemer14, N. Harnew59, T. Harrison56, C. Hasse44, M. Hatch44, J. He4, M.
Hecker57, K. Heinicke12, A. Heister12, K. Hennessy56, L. Henry76, M. Heß70, J.
Heuel11, A. Hicheur64, R. Hidalgo Charman58, D. Hill59, M. Hilton58, P.H.
Hopchev45, J. Hu14, W. Hu68, W. Huang4, Z.C. Huard61, W. Hulsbergen29, T.
Humair57, M. Hushchyn74, D. Hutchcroft56, D. Hynds29, P. Ibis12, M. Idzik32,
P. Ilten49, A. Inglessi35, A. Inyakin41, K. Ivshin35, R. Jacobsson44, S.
Jakobsen44, J. Jalocha59, E. Jans29, B.K. Jashal76, A. Jawahery62, F. Jiang3,
M. John59, D. Johnson44, C.R. Jones51, C. Joram44, B. Jost44, N. Jurik59, S.
Kandybei47, M. Karacson44, J.M. Kariuki50, S. Karodia55, N. Kazeev74, M.
Kecke14, F. Keizer51, M. Kelsey63, M. Kenzie51, T. Ketel30, B. Khanji44, A.
Kharisova75, C. Khurewathanakul45, K.E. Kim63, T. Kirn11, V.S. Kirsebom45, S.
Klaver20, K. Klimaszewski33, S. Koliiev48, M. Kolpin14, R. Kopecna14, P.
Koppenburg29, I. Kostiuk29,48, S. Kotriakhova35, M. Kozeiha7, L. Kravchuk38,
M. Kreps52, F. Kress57, S. Kretzschmar11, P. Krokovny40,x, W. Krupa32, W.
Krzemien33, W. Kucewicz31,l, M. Kucharczyk31, V. Kudryavtsev40,x, G.J.
Kunde78, A.K. Kuonen45, T. Kvaratskheliya36, D. Lacarrere44, G. Lafferty58, A.
Lai24, D. Lancierini46, G. Lanfranchi20, C. Langenbruch11, T. Latham52, C.
Lazzeroni49, R. Le Gac8, R. Lefèvre7, A. Leflat37, F. Lemaitre44, O. Leroy8,
T. Lesiak31, B. Leverington14, H. Li66, P.-R. Li4,ab, X. Li78, Y. Li5, Z.
Li63, X. Liang63, T. Likhomanenko72, R. Lindner44, F. Lionetto46, V.
Lisovskyi9, G. Liu66, X. Liu3, D. Loh52, A. Loi24, I. Longstaff55, J.H.
Lopes2, G. Loustau46, G.H. Lovell51, D. Lucchesi25,o, M. Lucio Martinez43, Y.
Luo3, A. Lupato25, E. Luppi18,g, O. Lupton52, A. Lusiani26, X. Lyu4, F.
Machefert9, F. Maciuc34, V. Macko45, P. Mackowiak12, S. Maddrell-Mander50, O.
Maev35,44, K. Maguire58, D. Maisuzenko35, M.W. Majewski32, S. Malde59, B.
Malecki44, A. Malinin72, T. Maltsev40,x, H. Malygina14, G. Manca24,f, G.
Mancinelli8, D. Marangotto23,q, J. Maratas7,w, J.F. Marchand6, U. Marconi17,
C. Marin Benito9, M. Marinangeli45, P. Marino45, J. Marks14, P.J. Marshall56,
G. Martellotti28, M. Martinelli44,22,i, D. Martinez Santos43, F. Martinez
Vidal76, A. Massafferri1, M. Materok11, R. Matev44, A. Mathad46, Z. Mathe44,
V. Matiunin36, C. Matteuzzi22, K.R. Mattioli77, A. Mauri46, E. Maurice9,b, B.
Maurin45, M. McCann57,44, A. McNab58, R. McNulty15, J.V. Mead56, B. Meadows61,
C. Meaux8, N. Meinert70, D. Melnychuk33, M. Merk29, A. Merli23,q, E.
Michielin25, D.A. Milanes69, E. Millard52, M.-N. Minard6, O. Mineev36, L.
Minzoni18,g, D.S. Mitzel14, A. Mödden12, A. Mogini10, R.D. Moise57, T.
Mombächer12, I.A. Monroy69, S. Monteil7, M. Morandin25, G. Morello20, M.J.
Morello26,t, J. Moron32, A.B. Morris8, R. Mountain63, F. Muheim54, M.
Mukherjee68, M. Mulder29, D. Müller44, J. Müller12, K. Müller46, V. Müller12,
C.H. Murphy59, D. Murray58, P. Naik50, T. Nakada45, R. Nandakumar53, A.
Nandi59, T. Nanut45, I. Nasteva2, M. Needham54, N. Neri23,q, S. Neubert14, N.
Neufeld44, R. Newcombe57, T.D. Nguyen45, C. Nguyen-Mau45,n, S. Nieswand11, R.
Niet12, N. Nikitin37, N.S. Nolte44, A. Oblakowska-Mucha32, V. Obraztsov41, S.
Ogilvy55, D.P. O’Hanlon17, R. Oldeman24,f, C.J.G. Onderwater71, J. D.
Osborn77, A. Ossowska31, J.M. Otalora Goicochea2, T. Ovsiannikova36, P.
Owen46, A. Oyanguren76, P.R. Pais45, T. Pajero26,t, A. Palano16, M. Palutan20,
G. Panshin75, A. Papanestis53, M. Pappagallo54, L.L. Pappalardo18,g, W.
Parker62, C. Parkes58,44, G. Passaleva19,44, A. Pastore16, M. Patel57, C.
Patrignani17,e, A. Pearce44, A. Pellegrino29, G. Penso28, M. Pepe Altarelli44,
S. Perazzini17, D. Pereima36, P. Perret7, L. Pescatore45, K. Petridis50, A.
Petrolini21,h, A. Petrov72, S. Petrucci54, M. Petruzzo23,q, B. Pietrzyk6, G.
Pietrzyk45, M. Pikies31, M. Pili59, D. Pinci28, J. Pinzino44, F. Pisani44, A.
Piucci14, V. Placinta34, S. Playfer54, J. Plews49, M. Plo Casasus43, F.
Polci10, M. Poli Lener20, M. Poliakova63, A. Poluektov8, N. Polukhina73,c, I.
Polyakov63, E. Polycarpo2, G.J. Pomery50, S. Ponce44, A. Popov41, D.
Popov49,13, S. Poslavskii41, E. Price50, C. Prouve43, V. Pugatch48, A. Puig
Navarro46, H. Pullen59, G. Punzi26,p, W. Qian4, J. Qin4, R. Quagliani10, B.
Quintana7, N.V. Raab15, B. Rachwal32, J.H. Rademacker50, M. Rama26, M. Ramos
Pernas43, M.S. Rangel2, F. Ratnikov39,74, G. Raven30, M. Ravonel Salzgeber44,
M. Reboud6, F. Redi45, S. Reichert12, F. Reiss10, C. Remon Alepuz76, Z. Ren3,
V. Renaudin59, S. Ricciardi53, S. Richards50, K. Rinnert56, P. Robbe9, A.
Robert10, A.B. Rodrigues45, E. Rodrigues61, J.A. Rodriguez Lopez69, M.
Roehrken44, S. Roiser44, A. Rollings59, V. Romanovskiy41, A. Romero Vidal43,
J.D. Roth77, M. Rotondo20, M.S. Rudolph63, T. Ruf44, J. Ruiz Vidal76, J.J.
Saborido Silva43, N. Sagidova35, B. Saitta24,f, V. Salustino Guimaraes65, C.
Sanchez Gras29, C. Sanchez Mayordomo76, B. Sanmartin Sedes43, R.
Santacesaria28, C. Santamarina Rios43, M. Santimaria20,44, E. Santovetti27,j,
G. Sarpis58, A. Sarti20,k, C. Satriano28,s, A. Satta27, M. Saur4, D.
Savrina36,37, S. Schael11, M. Schellenberg12, M. Schiller55, H. Schindler44,
M. Schmelling13, T. Schmelzer12, B. Schmidt44, O. Schneider45, A. Schopper44,
H.F. Schreiner61, M. Schubiger45, S. Schulte45, M.H. Schune9, R. Schwemmer44,
B. Sciascia20, A. Sciubba28,k, A. Semennikov36, E.S. Sepulveda10, A.
Sergi49,44, N. Serra46, J. Serrano8, L. Sestini25, A. Seuthe12, P. Seyfert44,
M. Shapkin41, T. Shears56, L. Shekhtman40,x, V. Shevchenko72, E. Shmanin73,
B.G. Siddi18, R. Silva Coutinho46, L. Silva de Oliveira2, G. Simi25,o, S.
Simone16,d, I. Skiba18, N. Skidmore14, T. Skwarnicki63, M.W. Slater49, J.G.
Smeaton51, E. Smith11, I.T. Smith54, M. Smith57, M. Soares17, l. Soares
Lavra1, M.D. Sokoloff61, F.J.P. Soler55, B. Souza De Paula2, B. Spaan12, E.
Spadaro Norella23,q, P. Spradlin55, F. Stagni44, M. Stahl14, S. Stahl44, P.
Stefko45, S. Stefkova57, O. Steinkamp46, S. Stemmle14, O. Stenyakin41, M.
Stepanova35, H. Stevens12, A. Stocchi9, S. Stone63, S. Stracka26, M.E.
Stramaglia45, M. Straticiuc34, U. Straumann46, S. Strokov75, J. Sun3, L.
Sun67, Y. Sun62, K. Swientek32, A. Szabelski33, T. Szumlak32, M. Szymanski4,
Z. Tang3, T. Tekampe12, G. Tellarini18, F. Teubert44, E. Thomas44, M.J.
Tilley57, V. Tisserand7, S. T’Jampens6, M. Tobin5, S. Tolk44, L.
Tomassetti18,g, D. Tonelli26, D.Y. Tou10, R. Tourinho Jadallah Aoude1, E.
Tournefier6, M. Traill55, M.T. Tran45, A. Trisovic51, A. Tsaregorodtsev8, G.
Tuci26,44,p, A. Tully51, N. Tuning29, A. Ukleja33, A. Usachov9, A.
Ustyuzhanin39,74, U. Uwer14, A. Vagner75, V. Vagnoni17, A. Valassi44, S.
Valat44, G. Valenti17, M. van Beuzekom29, H. Van Hecke78, E. van Herwijnen44,
C.B. Van Hulse15, J. van Tilburg29, M. van Veghel29, R. Vazquez Gomez44, P.
Vazquez Regueiro43, C. Vázquez Sierra29, S. Vecchi18, J.J. Velthuis50, M.
Veltri19,r, A. Venkateswaran63, M. Vernet7, M. Veronesi29, M. Vesterinen52,
J.V. Viana Barbosa44, D. Vieira4, M. Vieites Diaz43, H. Viemann70, X. Vilasis-
Cardona42,m, A. Vitkovskiy29, M. Vitti51, V. Volkov37, A. Vollhardt46, D. Vom
Bruch10, B. Voneki44, A. Vorobyev35, V. Vorobyev40,x, N. Voropaev35, R.
Waldi70, J. Walsh26, J. Wang5, M. Wang3, Y. Wang68, Z. Wang46, D.R. Ward51,
H.M. Wark56, N.K. Watson49, D. Websdale57, A. Weiden46, C. Weisser60, M.
Whitehead11, G. Wilkinson59, M. Wilkinson63, I. Williams51, M. Williams60,
M.R.J. Williams58, T. Williams49, F.F. Wilson53, M. Winn9, W. Wislicki33, M.
Witek31, G. Wormser9, S.A. Wotton51, K. Wyllie44, D. Xiao68, Y. Xie68, H.
Xing66, A. Xu3, M. Xu68, Q. Xu4, Z. Xu6, Z. Xu3, Z. Yang3, Z. Yang62, Y.
Yao63, L.E. Yeomans56, H. Yin68, J. Yu68,aa, X. Yuan63, O. Yushchenko41, K.A.
Zarebski49, M. Zavertyaev13,c, M. Zeng3, D. Zhang68, L. Zhang3, W.C. Zhang3,z,
Y. Zhang44, A. Zhelezov14, Y. Zheng4, X. Zhu3, V. Zhukov11,37, J.B.
Zonneveld54, S. Zucchelli17,e.

1Centro Brasileiro de Pesquisas Físicas (CBPF), Rio de Janeiro, Brazil
2Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil
3Center for High Energy Physics, Tsinghua University, Beijing, China
4University of Chinese Academy of Sciences, Beijing, China
5Institute Of High Energy Physics (ihep), Beijing, China
6Univ. Grenoble Alpes, Univ. Savoie Mont Blanc, CNRS, IN2P3-LAPP, Annecy,
France
7Université Clermont Auvergne, CNRS/IN2P3, LPC, Clermont-Ferrand, France
8Aix Marseille Univ, CNRS/IN2P3, CPPM, Marseille, France
9LAL, Univ. Paris-Sud, CNRS/IN2P3, Université Paris-Saclay, Orsay, France
10LPNHE, Sorbonne Université, Paris Diderot Sorbonne Paris Cité, CNRS/IN2P3,
Paris, France
11I. Physikalisches Institut, RWTH Aachen University, Aachen, Germany
12Fakultät Physik, Technische Universität Dortmund, Dortmund, Germany
13Max-Planck-Institut für Kernphysik (MPIK), Heidelberg, Germany
14Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg, Heidelberg,
Germany
15School of Physics, University College Dublin, Dublin, Ireland
16INFN Sezione di Bari, Bari, Italy
17INFN Sezione di Bologna, Bologna, Italy
18INFN Sezione di Ferrara, Ferrara, Italy
19INFN Sezione di Firenze, Firenze, Italy
20INFN Laboratori Nazionali di Frascati, Frascati, Italy
21INFN Sezione di Genova, Genova, Italy
22INFN Sezione di Milano-Bicocca, Milano, Italy
23INFN Sezione di Milano, Milano, Italy
24INFN Sezione di Cagliari, Monserrato, Italy
25INFN Sezione di Padova, Padova, Italy
26INFN Sezione di Pisa, Pisa, Italy
27INFN Sezione di Roma Tor Vergata, Roma, Italy
28INFN Sezione di Roma La Sapienza, Roma, Italy
29Nikhef National Institute for Subatomic Physics, Amsterdam, Netherlands
30Nikhef National Institute for Subatomic Physics and VU University Amsterdam,
Amsterdam, Netherlands
31Henryk Niewodniczanski Institute of Nuclear Physics Polish Academy of
Sciences, Kraków, Poland
32AGH - University of Science and Technology, Faculty of Physics and Applied
Computer Science, Kraków, Poland
33National Center for Nuclear Research (NCBJ), Warsaw, Poland
34Horia Hulubei National Institute of Physics and Nuclear Engineering,
Bucharest-Magurele, Romania
35Petersburg Nuclear Physics Institute NRC Kurchatov Institute (PNPI NRC KI),
Gatchina, Russia
36Institute of Theoretical and Experimental Physics NRC Kurchatov Institute
(ITEP NRC KI), Moscow, Russia, Moscow, Russia
37Institute of Nuclear Physics, Moscow State University (SINP MSU), Moscow,
Russia
38Institute for Nuclear Research of the Russian Academy of Sciences (INR RAS),
Moscow, Russia
39Yandex School of Data Analysis, Moscow, Russia
40Budker Institute of Nuclear Physics (SB RAS), Novosibirsk, Russia
41Institute for High Energy Physics NRC Kurchatov Institute (IHEP NRC KI),
Protvino, Russia, Protvino, Russia
42ICCUB, Universitat de Barcelona, Barcelona, Spain
43Instituto Galego de Física de Altas Enerxías (IGFAE), Universidade de
Santiago de Compostela, Santiago de Compostela, Spain
44European Organization for Nuclear Research (CERN), Geneva, Switzerland
45Institute of Physics, Ecole Polytechnique Fédérale de Lausanne (EPFL),
Lausanne, Switzerland
46Physik-Institut, Universität Zürich, Zürich, Switzerland
47NSC Kharkiv Institute of Physics and Technology (NSC KIPT), Kharkiv, Ukraine
48Institute for Nuclear Research of the National Academy of Sciences (KINR),
Kyiv, Ukraine
49University of Birmingham, Birmingham, United Kingdom
50H.H. Wills Physics Laboratory, University of Bristol, Bristol, United
Kingdom
51Cavendish Laboratory, University of Cambridge, Cambridge, United Kingdom
52Department of Physics, University of Warwick, Coventry, United Kingdom
53STFC Rutherford Appleton Laboratory, Didcot, United Kingdom
54School of Physics and Astronomy, University of Edinburgh, Edinburgh, United
Kingdom
55School of Physics and Astronomy, University of Glasgow, Glasgow, United
Kingdom
56Oliver Lodge Laboratory, University of Liverpool, Liverpool, United Kingdom
57Imperial College London, London, United Kingdom
58School of Physics and Astronomy, University of Manchester, Manchester,
United Kingdom
59Department of Physics, University of Oxford, Oxford, United Kingdom
60Massachusetts Institute of Technology, Cambridge, MA, United States
61University of Cincinnati, Cincinnati, OH, United States
62University of Maryland, College Park, MD, United States
63Syracuse University, Syracuse, NY, United States
64Laboratory of Mathematical and Subatomic Physics , Constantine, Algeria,
associated to 2
65Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio), Rio de
Janeiro, Brazil, associated to 2
66South China Normal University, Guangzhou, China, associated to 3
67School of Physics and Technology, Wuhan University, Wuhan, China, associated
to 3
68Institute of Particle Physics, Central China Normal University, Wuhan,
Hubei, China, associated to 3
69Departamento de Fisica , Universidad Nacional de Colombia, Bogota, Colombia,
associated to 10
70Institut für Physik, Universität Rostock, Rostock, Germany, associated to 14
71Van Swinderen Institute, University of Groningen, Groningen, Netherlands,
associated to 29
72National Research Centre Kurchatov Institute, Moscow, Russia, associated to
36
73National University of Science and Technology “MISIS”, Moscow, Russia,
associated to 36
74National Research University Higher School of Economics, Moscow, Russia,
associated to 39
75National Research Tomsk Polytechnic University, Tomsk, Russia, associated to
36
76Instituto de Fisica Corpuscular, Centro Mixto Universidad de Valencia -
CSIC, Valencia, Spain, associated to 42
77University of Michigan, Ann Arbor, United States, associated to 63
78Los Alamos National Laboratory (LANL), Los Alamos, United States, associated
to 63

aUniversidade Federal do Triângulo Mineiro (UFTM), Uberaba-MG, Brazil
bLaboratoire Leprince-Ringuet, Palaiseau, France
cP.N. Lebedev Physical Institute, Russian Academy of Science (LPI RAS),
Moscow, Russia
dUniversità di Bari, Bari, Italy
eUniversità di Bologna, Bologna, Italy
fUniversità di Cagliari, Cagliari, Italy
gUniversità di Ferrara, Ferrara, Italy
hUniversità di Genova, Genova, Italy
iUniversità di Milano Bicocca, Milano, Italy
jUniversità di Roma Tor Vergata, Roma, Italy
kUniversità di Roma La Sapienza, Roma, Italy
lAGH - University of Science and Technology, Faculty of Computer Science,
Electronics and Telecommunications, Kraków, Poland
mLIFAELS, La Salle, Universitat Ramon Llull, Barcelona, Spain
nHanoi University of Science, Hanoi, Vietnam
oUniversità di Padova, Padova, Italy
pUniversità di Pisa, Pisa, Italy
qUniversità degli Studi di Milano, Milano, Italy
rUniversità di Urbino, Urbino, Italy
sUniversità della Basilicata, Potenza, Italy
tScuola Normale Superiore, Pisa, Italy
uUniversità di Modena e Reggio Emilia, Modena, Italy
wMSU - Iligan Institute of Technology (MSU-IIT), Iligan, Philippines
xNovosibirsk State University, Novosibirsk, Russia
ySezione INFN di Trieste, Trieste, Italy
zSchool of Physics and Information Technology, Shaanxi Normal University
(SNNU), Xi’an, China
aaPhysics and Micro Electronic College, Hunan University, Changsha City, China
abLanzhou University, Lanzhou, China
†Deceased


# MULTI-LEVEL BATCH NORMALIZATION IN DEEP NETWORKS FOR INVASIVE DUCTAL
CARCINOMA CELL DISCRIMINATION IN HISTOPATHOLOGY IMAGES

###### Abstract

Breast cancer is the most diagnosed cancer and the most predominant cause of
death in women worldwide. Imaging techniques such as the breast cancer
pathology helps in the diagnosis and monitoring of the disease. However
identification of malignant cells can be challenging given the high
heterogeneity in tissue absorbotion from staining agents. In this work, we
present a novel approach for Invasive Ductal Carcinoma (IDC) cells
discrimination in histopathology slides. We propose a model derived from the
Inception architecture, proposing a multi-level batch normalization module
between each convolutional steps. This module was used as a base block for the
feature extraction in a CNN architecture. We used the open IDC dataset in
which we obtained a balanced accuracy of 0.89 and an F1 score of 0.90, thus
surpassing recent state of the art classification algorithms tested on this
public dataset.

Index Terms—  breast cancer, pathology, deep learning, batch normalization,
convolutional neural networks

##  1 Introduction

According to a recent census from 140 countries worldwide [1], breast cancer
is the most frequently diagnosed cancer in women, with an estimated incidence
of 1.7 million new cases per year. It represents 25% of all types of cancers
diagnosed in women. Furthermore, 15% of cancer deaths in women are due to
breast cancer [2]. In recent years, a combination of improved detection
techniques and earlier diagnosis, added to more effective treatments, has
caused the mortality rate to decrease in developed or emerging countries [1].

This has motivated groups to pursue efforts in improving detection methods at
early stages of the disease. Breast cancer detection through histopathology
images allows to quantify the cancer cell ratio with respect to healthy cells.
Oncologists use this information to determine cancer aggressiveness and select
the appropriate treatment option. Previously, researchers have worked on
methods for cancer/healthy cell classification [3, 4]. However, the scientific
community is still working to improve these classification methods by
identifying predominant features in cell images.

Convolutional neuronal networks (CNN) have become increasingly popular in the
processing of histopathological images [5]. The main difference with respect
to traditional machine learning approaches is that feature extraction is done
intrinsically and hierarchically. This allows the network to choose in a an
unsupervised fashion the features for optimal cell discrimination. In this
work, we present a new CNN-based model for cancer/healthy cell classification
on histopathology slides (HPS). We leverage these recent discoveries in deep
learning, particularly using the Inception architecture [6] by combining
regularization techniques such as batch normalization [7] in order to reduce
the intrinsic covariate shift.

##  2 Methods and materials

###  2.1 Dataset

We used a publicly available dataset 111Available at
http://andrewjanowczyk.com/wp-static/IDC_regular_ps50_idx5.zip that was first
introduced by Cruz-Roa et al. [3] for Invasive Ductal Carcinoma (IDC)
identification. The histopathological images of the dataset were not released
in their original format but in RGB patches of 50x50 pixels. The zoom factor
was 2.5x (4μm/pixel). The dataset contained 277,525 patches from 279 subjects.
The patches were comprised of 28.39% invasive ductal carcinoma cells, while
the remaining patches consisted of healthy tissue or non-invasive ductal
carcinoma (see Figure 1).

Fig. 1: Left: three IDC cell patches; Right: three normal tissue patches.

###  2.2 Proposed model

The Inception module [6] introduced by Szegedy C. et al. was initially
proposed to deal with the size variability problem in hierarchical
representations. The presented solution was simple and achieved excellent
results by using parallel filters with different kernel sizes, and allowed the
back-propagation process to select which filter would be used in each case.

Our model is composed of blocks derived from Inception that integrates batch
normalization (BN) [7] after each convolution step. For each Inception module,
the feature number per branch is determined during validation. Due to the
concatenation that takes place in each module, the feature output number is
four times greater than the feature number per branch value. In the branches
that contain the filters with 5x5 and 3x3 kernels, dimensionality reduction is
applied to minimize memory consumption through filters with 1x1 kernels. The
number of dimensionality reduction filters is set by the ($\alpha$) and
($\beta$) parameters, which vary depending on the feature number per branch
(see Figure 2).

Fig. 2: Schematic illustration of the proposed Inception module.

The batch normalization steps reduce the internal covariate shift, improving
network training stability. This aids in the training as the technique re-
parametrizes the underlying optimization problem thereby making the gradients
more predictable [8].

MaxPool layers are used to reduce the size of the internal network
representations. These layers increase the number of extracted features
without excessive memory consumption and provide a certain level of rotational
invariance. In the final stages where the classification is performed, we
include two fully connected layers of 512 neurons with batch normalization and
dropout of 0.4 each. The network output contains two neurons followed by a
SoftMax layer. See Table 1 for the detailed architecture.

Table 1: Details of the framework architecture. Block Type | Block Name | Feature #
---|---|---
Feature Extraction | Inception | 64
Feature Extraction | Inception | 64
Feature Extraction | MaxPool | -
Feature Extraction | Inception | 128
Feature Extraction | Inception | 126
Feature Extraction | MaxPool | -
Feature Extraction | Inception | 256
Feature Extraction | Inception | 256
Feature Extraction | MaxPool | -
Feature Extraction | Inception | 512
Feature Extraction | Inception | 512
Classification | Fully connected | 256
Classification | Batch Normalization | -
Classification | Dropout (0.4) | -
Classification | Fully connected | 256
Classification | Batch Normalization | -
Classification | Dropout (0.4) | -
Classification | Fully connected | 2
Classification | Softmax | -

###  2.3 Training Protocol

In this section, we present the training protocol with the parameter selection
strategies.

Padding: Most of the patch images were 50x50 pixels in size, but in some
isolated cases, images which contained 1-2 fewer rows or columns were zero-
padded.

Normalization: Mean centering and standard deviation normalization was applied
to each image. This decreases the variability of the input data, thus
improving the training stability.

Train/Validation/Test split: We train the network with a balanced training set
of 94,543 images. The validation set contained 31,514 images, also balanced.
For the final evaluation, our test set is unbalanced, the total number of
images were 151,465 images, of these 15,757 images contain cancer cells.

Batch size: A batch size of 32 images was used for training. This makes the
training faster as it takes advantage of the speed-up of matrix-matrix
products [9].

Optimization and Learning Rate Scheduling: Adam optimization algorithm [10]
was used with an initial learning rate of 10-3. This learning rate was reduced
by a factor of 2 after 10 epochs without improvements in the validation set
accuracy. The allowed minimum learning rate was 10-10.

Iterations: The initial number of epochs was 103, but due to the early
stopping criterion that was implemented, the final number of epochs was 55.
For early stopping, the validation set accuracy was monitored, and stopped
after 50 epochs if no improvements was detected.

Training time on an NVIDIA Titan Xp GPU was 170 minutes, while the inference
time per single patch was 3 ms (89 ms on an I7 7th generation CPU). The above
timings were calculated for the proposed model, include the Inception
architecture.

##  3 Results and Discussion

Table 2 shows a comparison between the balanced accuracy and F1-score obtained
with our method and previous methods using the same dataset. Both metrics are
designed to deal with unbalanced datasets because in these cases, metrics such
as accuracy can lead to misinterpretations in the classifier efficiency. Our
model showed an improvement in the results obtained by Cruz-Roa et al. [3],
that presented a custom deep neural network, and the results obtained by
Janowczyk and Madabhushi [4] who used AlexNet architecture [11] for the
classification task in the same dataset.

Table 2: Ensemble evaluation results comparing our model with others approaches in the same dataset. Model |  | Balanced
---
Accuracy
F1 score
2D-CNN [3] | 0.842 | 0.718
AlexNet with BN [4] | 0.847 | 0.765
Original InceptionNet [6] | 0.868 | 0.883
Proposed method | 0.890 | 0.897

Figures 4 and 4 shows the efficiency of our classifier. The area under the
curve (AUC) of the receiver operating characteristic (ROC) was 0.956 for
Inception+BN, while the original Inception module was 0.949. The improvement
in accuracy was 2.2% compared to the original model. Figure 4 shows the
confusion matrix of the obtained results for the proposed architecture.

Fig. 3: Models performance represented by the ROC curves.

Fig. 4: Model performance evaluated by the confusion matrix.

These results demonstrate the effectiveness of our Inception module variant.
Due to network’s multi-scale capability of processing information through
filters with different kernel sizes, the network is able to extract highly
discriminative features. Furthermore, the use of batch normalization aids the
training by reducing internal covariate shift, thus increasing the stability
of the deep neural network. Finally, the training strategy allowed us to
obtain the maximum benefit of each learning rate step, improving the
convergence of the network.

|
---|---
(a) | (b)
Fig. 5: IDC cell prediction results. (a) Original reconstructed pathology
slide. (b) Prediction heat map overlapped to the original slide.

Because the original patches in the database contain patch positions, this
enables to reconstruct the entire original slides. The trained models were
then applied at the patch level to generate entire processed slides. Because
our model processes patches of 50x50 pixels, the generated heatmap appeared
“pixelated”. To improve the final rendering, we applied a Gaussian filter with
a 25x25 kernel. Figure 5 (a) shows the slide number 14154, while Figure 5 (b)
shows an overlap of the predictions heat map generated by our model. Our
framework can be applied to HPS of other datasets, provided they have a
similar zoom factor of x2.5.

##  4 Conclusion

We presented a novel approach for Invasive Ductal Carcinoma cell
discrimination in histopathology slides. The model is composed of Inception
variant blocks with combined batch normalization steps to reduce internal
covariances. Results surpass state of the art classification methods which
were trained and tested on the same dataset. The quantitative evaluation of
the results was carried out through balanced accuracy and F1-score, metrics
used for unbalanced datasets. Future work will be focused on the inclusion of
residual blocks [12] to deal with the vanishing gradients during the training,
which in theory will lead to improved accuracy.

## References

  * [1] W.B. Stewart and C.P. Wild,  “World cancer report 2014. lyon, france: International agency for research on cancer,”  World Health Organization, p. 630, 2014.
  * [2] J. Ferlay, I. Soerjomataram, R. Dikshit, S. Eser, C. Mathers, M. Rebelo, M.D. Parkin, D. Forman, and F. Bray,  “Cancer incidence and mortality worldwide: sources, methods and major patterns in globocan 2012,”  International journal of cancer, vol. 136, no. 5, pp. E359–E386, 2015.
  * [3] A. Cruz-Roa, A. Basavanhally, F. Gonz’alez, H. Gilmore, M. Feldman, S. Ganesan, N. Shih, J. Tomaszewski, and A. Madabhushi,  “Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks,”  in Medical Imaging 2014: Digital Pathology. International Society for Optics and Photonics, 2014, vol. 9041, p. 904103.
  * [4] A. Janowczyk and A. Madabhushi,  “Deep learning for digital pathology image analysis: A comprehensive tutorial with selected use cases,”  Journal of pathology informatics, vol. 7, 2016.
  * [5] G. Litjens, T. Kooi, E.B. Bejnordi, A.A. Setio, F. Ciompi, M. Ghafoorian, A.J. van der Laak, B. Van Ginneken, and I.C. S’anchez,  “A survey on deep learning in medical image analysis,”  Medical image analysis, vol. 42, pp. 60–88, 2017.
  * [6] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich,  “Going deeper with convolutions,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1–9.
  * [7] S. Ioffe and C. Szegedy,  “Batch normalization: Accelerating deep network training by reducing internal covariate shift,”  arXiv preprint arXiv:1502.03167, 2015.
  * [8] S. Santurkar, D. Tsipras, A. Ilyas, and A. Madry,  “How does batch normalization help optimization?(no, it is not about internal covariate shift),”  arXiv preprint arXiv:1805.11604, 2018.
  * [9] Y. Bengio,  “Practical recommendations for gradient-based training of deep architectures,”  in Neural networks: Tricks of the trade, pp. 437–478. Springer, 2012.
  * [10] P.D. Kingma and J. Ba,  “Adam: A method for stochastic optimization,”  arXiv preprint arXiv:1412.6980, 2014.
  * [11] A. Krizhevsky, I. Sutskever, and E.G. Hinton,  “Imagenet classification with deep convolutional neural networks,”  in Advances in neural information processing systems, 2012, pp. 1097–1105.
  * [12] K. He, X. Zhang, S. Ren, and J. Sun,  “Deep residual learning for image recognition,”  in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.