EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH (CERN)

 CERN-PH-EP-2015-144 LHCb-PAPER-2015-018 4 January 2016

First observation of the decay $B_{s}^{0}\rightarrow
K_{\rm{S}}^{0}K^{*}(892)^{0}$

The LHCb collaboration†††Authors are listed at the end of this paper.

A search for $B_{(s)}^{0}\rightarrow K_{S}^{0}K^{*}(892)^{0}$ decays is
performed using $pp$ collision data, corresponding to an integrated luminosity
of $1.0~{}\text{fb}^{-1}$, collected with the LHCb detector at a centre-of-
mass energy of $7~{}\text{TeV}$. The $B_{s}^{0}\rightarrow
K_{S}^{0}K^{*}(892)^{0}$ decay is observed for the first time, with a
significance of 7.1 standard deviations. The branching fraction is measured to
be

$\mathcal{B}(B_{s}^{0}\rightarrow\bar{K}^{0}K^{*}(892)^{0})+\mathcal{B}(B_{s}^{0}\rightarrow
K^{0}\bar{K}^{*}(892)^{0})=(16.4\pm 3.4\pm 2.3)\times 10^{-6},\\\ $

where the first uncertainty is statistical and the second is systematic. No
evidence is found for the decay $B^{0}\rightarrow K_{S}^{0}K^{*}(892)^{0}$ and
an upper limit is set on the branching fraction,
$\mathcal{B}(B^{0}\rightarrow\bar{K}^{0}K^{*}(892)^{0})+\mathcal{B}(B^{0}\rightarrow
K^{0}\bar{K}^{*}(892)^{0})<0.96\ \times 10^{-6},$ at $90\,\%$ confidence
level. All results are consistent with Standard Model predictions.

Published in JHEP 2016(1), 1-17

© CERN on behalf of the LHCb collaboration, licence CC-BY-4.0.

##  1 Introduction

Violation of the combined charge-conjugation and parity symmetry ($C\\!P$) is
one of the fundamental ingredients to explain a dynamical generation of the
observed matter-antimatter asymmetry in the universe[1]. In the Standard Model
of particle physics (SM), $C\\!P$ violation in the quark sector is generated
by a single complex phase in the Cabibbo-Kobayashi-Maskawa matrix [2, 3].
However, the observed baryon asymmetry is too large to be explained by the SM
mechanism alone [4]. Non-leptonic $B$ meson decays dominated by amplitudes
involving a quark and a $W$ boson in a loop (penguin) are sensitive to the
presence of non-SM physics processes. These processes could provide additional
sources of $C\\!P$ violation that might explain the observed baryon asymmetry.
The ${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}h^{\pm}h^{{}^{\prime}\mp}$ ($h,h^{{}^{\prime}}=\pi,K$) decays are
interesting for $C\\!P$ violation measurements111Charge-conjugate modes are
implicitly included throughout this paper.. Knowledge of the branching
fractions of the various sub-modes, as reported in this paper, is an important
input to the theory of $C\\!P$-violation, particularly models of new-physics
contributions to $b\rightarrow s$ transitions Ref. [5] . The measurements also
allow tests of QCD models (see, for example, the predictions in Refs. [6, 7,
8]). If sufficient data are available, a common approach for three-body decays
is to perform an amplitude analysis by studying the structure of the Dalitz
plot [9]. If data are less abundant and the decay products originate from
intermediate resonances, as in the present analysis, a quasi two-body approach
can be used.

The LHCb collaboration has provided results for inclusive
${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}h^{\pm}h^{{}^{\prime}\mp}$ decays [10], and more recently the first
measurements of $B_{s}^{0}$ meson decays to $K^{*}(892)^{-}\pi^{+}$ and
$K^{*}(892)^{-}K^{+}$ final states [11]. An initial search for the neutral
decay ${{B}^{0}}\rightarrow K^{*}(892)^{0}{{K}^{0}_{\rm\scriptscriptstyle S}}$
has been reported by the BaBar experiment [12]. In this paper a search for
${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{K}^{*}}(892)^{0}$ decays is reported, where the ${{K}^{*}}(892)^{0}$
meson, hereafter denoted by ${K}^{*0}$, decays to the ${K}^{+}$ ${\pi}^{-}$
final state. The resonant structure in the ${K}^{+}$ ${\pi}^{-}$ invariant
mass region around 1${\mathrm{\,Ge\kern-1.00006ptV\\!/}c^{2}}$ is analysed to
determine the number of decays that proceed through an intermediate ${K}^{*0}$
resonance. The branching fraction is measured relative to the
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}$ decay [13], using the relation

$\frac{\mathcal{B}({B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}K^{*0})}{\mathcal{B}({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}})}=\frac{N_{{B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}K^{*0}}}{N_{{{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}}}\cdot\frac{\epsilon_{{{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}}}{\epsilon_{{B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}K^{*0}}}\cdot\frac{f_{d}}{f_{d(s)}}\cdot\frac{1}{\mathcal{B}(K^{*0}\rightarrow
K^{+}\pi^{-})},$ (1)

where $N$ represents the number of observed decays, $\epsilon$ the total
efficiency, and $f_{s}/f_{d}$ the ratio of the fragmentation fractions of a
$b$ quark into a ${B}^{0}_{s}$ or a ${B}^{0}$ meson [14, 15, 16] and
$\mathcal{B}(K^{*0}\rightarrow K^{+}\pi^{-})$ is the branching fraction of the
$K^{*0}$ meson into $K^{+}\pi^{-}$ final state. In the following, the
${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ and
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}$ decays are referred to as signal and normalisation
channels, respectively.

##  2 Detector and simulation

The analysis is performed using $pp$ collision data recorded with the LHCb
detector, corresponding to an integrated luminosity of
$1.0$$\mbox{\,fb}^{-1}$, at a centre-of-mass energy of
$7$$\mathrm{\,Te\kern-1.00006ptV}$. The LHCb detector [17, 18] is a single-arm
forward spectrometer covering the pseudorapidity range $2<\eta<5$, designed
for the study of particles containing $b$ or $c$ quarks. The detector includes
a high-precision tracking system consisting of a silicon-strip vertex detector
surrounding the $pp$ interaction region, a large-area silicon-strip detector
located upstream of a dipole magnet with a bending power of about
$4{\rm\,Tm}$, and three stations of silicon-strip detectors and straw drift
tubes placed downstream of the magnet. The tracking system provides a
measurement of momentum, $p$, of charged particles with a relative uncertainty
that varies from 0.5% at low momentum to 1.0% at
200${\mathrm{\,Ge\kern-1.00006ptV\\!/}c}$. The minimum distance of a track to
a primary vertex (PV), the impact parameter, is measured with a resolution of
$(15+29/\mbox{$p_{\rm T}$}){\,\upmu\rm m}$, where $p_{\rm T}$ is the component
of the momentum transverse to the beam, in
${\mathrm{\,Ge\kern-1.00006ptV\\!/}c}$. Different types of charged hadrons are
distinguished using information from two ring-imaging Cherenkov detectors.
Photons, electrons and hadrons are identified by a calorimeter system
consisting of scintillating-pad and preshower detectors, an electromagnetic
calorimeter and a hadronic calorimeter. Muons are identified by a system
composed of alternating layers of iron and multiwire proportional chambers.

Simulated events are used to determine the efficiency of the selection
requirements, to study possible sources of background and to determine the
parametrisations used to model the data. In the simulation, $pp$ collisions
are generated using Pythia 6 [19] with a specific LHCb configuration [20].
Decays of hadronic particles are described by EvtGen [21], in which final-
state radiation is generated using Photos [22]. The interaction of the
generated particles with the detector, and its response, are implemented using
the Geant4 toolkit [23, *Agostinelli:2002hh] as described in Ref. [25].

##  3 Event selection

The online event selection system (trigger) [26] consists of a hardware stage,
based on information from the calorimeter and muon systems, followed by a
software stage, in which all charged particles with $\mbox{$p_{\rm
T}$}>500{\mathrm{\,Me\kern-1.00006ptV\\!/}c}$ are reconstructed. The hardware
hadron trigger requires a calorimeter cluster with transverse energy greater
than $3.5\mathrm{\,Ge\kern-1.00006ptV}$. In the offline selection, candidates
are divided into two non mutually exclusive categories based on the hardware
trigger decision. One category consists of candidates whose decay products
satisfy the hadron trigger requirements, while the other consists of
candidates from events in which other particles meet the hardware trigger
requirements. Only events that fall into either of these categories are used
in the subsequent analysis. The software trigger requires a two-, three- or
four-particle secondary vertex with a significant displacement from the
primary $pp$ interaction points. At least one charged particle must have
$\mbox{$p_{\rm T}$}>1.7{\mathrm{\,Ge\kern-1.00006ptV\\!/}c}$ and be
inconsistent with originating from any PV. A multivariate algorithm [27] is
used for the identification of secondary vertices consistent with the decay of
a $b$ hadron.

In the offline selection the
${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ decays
are reconstructed through the ${K}^{*0}$ $\rightarrow$ ${K}^{+}$ ${\pi}^{-}$
and ${K}^{0}_{\rm\scriptscriptstyle S}$ $\rightarrow$ ${\pi}^{+}$ ${\pi}^{-}$
decay modes, where the ${K}^{0}_{\rm\scriptscriptstyle S}$ candidate is
constrained to its known mass [13] and the $B$ candidate is constrained to
originate from a PV. Decays of ${K}^{0}_{\rm\scriptscriptstyle S}$ mesons are
reconstructed in two mutually exclusive categories: long
${K}^{0}_{\rm\scriptscriptstyle S}$ candidates, which decay sufficiently early
that their daughter pions are reconstructed in the vertex detector; and
downstream ${K}^{0}_{\rm\scriptscriptstyle S}$ candidates, which have daughter
particles that are only reconstructed in the rest of the tracking system. As
these two categories have different backgrounds, and the long
${K}^{0}_{\rm\scriptscriptstyle S}$ mesons have better momentum and vertex
resolutions, the selection criteria for long and downstream
${K}^{0}_{\rm\scriptscriptstyle S}$ candidates differ. The selection criteria
follow those in Ref. [10].

Fully reconstructed background decays that have the same final state as the
signal include contributions from $B$ decays to final states involving charm
mesons, such as $Dh$, with a ${{K}^{0}_{\rm\scriptscriptstyle S}}h^{+}h^{-}$
final state, or $\Lambda_{b}^{0}$ decays to $\Lambda_{c}^{+}h^{-}$, with
$\Lambda_{c}^{+}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}p$, where the
proton is misidentified as a ${\pi}^{+}$ or ${K}^{+}$. In addition, $B$ decays
with an intermediate charmonium state like ${B}^{0}$ $\rightarrow$
$J/\psi$${K}^{0}_{\rm\scriptscriptstyle S}$, with
$J/\psi\\!\rightarrow{{\pi}^{+}}{{\pi}^{-}},{{K}^{+}}{{K}^{-}},{\mu^{+}}{\mu^{-}}$,
can be present in the mass region of the normalisation channel. To reduce the
contamination from these backgrounds, a veto is applied on the invariant mass
of each of the possible intermediate states reconstructed under the
corresponding hypothesis. Candidates are excluded if the reconstructed mass of
a two-body intermediate state is within
30${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$
(48${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$) of the known mass of the
relevant intermediate charm (charmonium) resonance [13] of one of the
backgrounds considered. No particle identification information is used at this
stage.

If a final-state hadron is misidentified, signal yields can potentially be
affected by decays into any ${{K}^{0}_{\rm\scriptscriptstyle
S}}h^{\pm}h^{\prime\mp}$ final state, especially when the
$h^{\pm}h^{\prime\mp}$ proceeds through a resonance. Particle identification
requirements on the two tracks originating from the $B$ decay vertex are used
to separate pions, kaons and protons, and to reduce this background to a
negligible level. The largest source of background is due to random tracks
that form candidate $B$ or ${K}^{0}_{\rm\scriptscriptstyle S}$ decay vertices.
A multivariate discriminant based on a boosted decision tree (BDT) algorithm
[28, 29] is used to reduce this background. The greatest discrimination in the
BDT is provided by kinematic properties of the $B$ meson, its flight direction
with respect to the PV, and variables defined analogously for its decay
products. The optimisation of the BDT is described in Ref. [10]; the selection
requirement on the BDT response for this analysis is chosen to maximise
$\epsilon/(a/2+\sqrt{N_{B}})$ [30]. Here, $\epsilon$ is the signal efficiency,
$B$ represents the number of background events in the signal mass interval,
which is estimated using data by extrapolating the number of background events
from the upper mass sideband into the signal region, and $a=5$ is the chosen
target signal significance.

The efficiencies are determined from simulation, except for the particle
identification efficiencies. The latter are determined from data using samples
of kinematically identified charged particles from
${{D}^{*+}}\\!\rightarrow{{D}^{0}}{{\pi}^{+}}$ with
${{D}^{0}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}$, and
$\Lambda\\!\rightarrow{{\pi}^{-}}{p}$ decays, reweighted to match the
kinematic properties of the signal. The BDT selection efficiency for signal is
approximately 85% (90%) for downstream (long) signal decays; approximately 88%
(95%) of backgrounds in the respective categories are rejected. The
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}$ decay selection efficiency is taken from Ref. [10].
The efficiencies for the normalisation channel are determined in bins of the
Dalitz plane and are reweighted from data using the sPlot method [31].

##  4 Fit model

Two-dimensional extended maximum likelihood fits to the unbinned
${{K}^{0}_{\rm\scriptscriptstyle S}}$ ${K}^{+}$ ${\pi}^{-}$ and ${K}^{+}$
${\pi}^{-}$ mass distributions are used to determine the event yields for the
signal channel, while an independent one-dimensional fit to the
${{K}^{0}_{\rm\scriptscriptstyle S}}$ ${\pi}^{+}$ ${\pi}^{-}$ mass
distribution is used for the normalisation channel. The correlation between
the two signal mass distributions is checked on simulation. The results do not
show significant correlations and therefore the correlation terms are
neglected in the fit. Candidates in the long and downstream categories are
fitted simultaneously. The signal fit is restricted to candidates in the mass
regions $5000$ $<$ $m$(${{K}^{0}_{\rm\scriptscriptstyle S}}$ ${K}^{+}$
${\pi}^{-}$) $<$ $5800$ ${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$ and $650$
$<$ $m$(${K}^{+}$ ${\pi}^{-}$) $<$ $1200$
${\mathrm{\,Me\kern-1.00006ptV\\!/}c^{2}}$. The fit model consists of signal,
non-resonant background, partially reconstructed background and combinatorial
background components.

The ${B}^{0}$ and the ${B}^{0}_{s}$ components of the signal are both
parametrised as two Gaussian distributions with a power-law tail on each side.
For each component the two functions share the peak position and the width
parameters. The parameters describing the tails are determined by fits to
simulated samples and subsequently fixed in the fit to data. The systematic
uncertainty associated with this choice is found to be negligible. The
${B}^{0}$-${B}^{0}_{s}$ mass difference is fixed to the known value[13]. The
${K}^{*0}$ mass distribution is parametrised by a relativistic Breit-Wigner
function with the peak position and width allowed to vary in the fit.

The components in the $B$ mass model that are non-resonant in ${K}^{+}$
${\pi}^{-}$ are parametrised by the same function as the signal, sharing their
peak positions and widths with the signal functions. The tail parameters are
fixed according to the values obtained from simulation. The non-resonant
component of the ${K}^{+}$ ${\pi}^{-}$ mass distribution is approximated by a
normalised linear function as in Ref.[11], with the zero point of the function
on the abscissa determined by the fit. While the ratio between the non-
resonant and the signal components is fixed to be the same for the two
${K}^{0}_{\rm\scriptscriptstyle S}$ meson categories, it is independent for
the ${B}^{0}$ and the ${B}^{0}_{s}$ candidates.

Backgrounds from partially reconstructed decays are classified into two
categories. Decays such as ${B}\rightarrow{D}h$ are parametrised by means of
ARGUS functions convolved with a Gaussian function in the $B$ candidate mass,
and linear functions in the ${K}^{*0}$ candidate mass. The choice is based on
simulation studies and previous findings [10, 11]. In decays such as
${{B}^{0}_{s}}\\!\rightarrow{{K}^{*0}}\overline{K}^{*0}$, where one resonance
decays as ${{K}^{*0}}\\!\rightarrow{{K}^{+}}{{\pi}^{-}}$ while the other
decays as $\overline{K}^{*0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{0}}$, the ${B}^{0}_{s}$ mass distribution is described using the
same parametrisation as for the previous background, while the invariant mass
distribution for ${K}^{*0}$ candidates is described by a relativistic Breit-
Wigner function sharing the peak position and widths with the signal
component. The yield for these components are determined in the fit to data.

The combinatorial background is modelled by an exponential function in the $B$
candidate mass distribution and a linear function in the ${K}^{*0}$ candidate
mass distribution. These functions are found to give good agreement with the
distributions in the appropriate data sidebands. The slopes of the exponential
functions are independent for the long and downstream categories, while the
abscissae of the linear functions are the same. All these parameters are
allowed to vary in the fit.

The parametrisation used to model the
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}$ normalisation and the background follow those used
to fit the signal mode. In addition two other categories of partially
reconstructed backgrounds are included: decays such as
${{B}^{0}}\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}\gamma$ or
${{B}^{0}}\rightarrow\eta^{\prime}{{K}^{0}_{\rm\scriptscriptstyle S}}$, with
$\eta\\!\rightarrow\rho^{0}\gamma$; and misidentified
${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{K}^{\pm}}{{\pi}^{\mp}}$ decays. Their parameters are fixed in the fit to
the values derived from simulated samples.

The observed ${K}^{0}_{\rm\scriptscriptstyle S}$ ${K}^{*0}$ and
${K}^{0}_{\rm\scriptscriptstyle S}$ ${\pi}^{+}$ ${\pi}^{-}$ mass distributions
and the corresponding fits are shown in Figs. 1 and 2, respectively. The
signal yields are reported in Table 1. The ${B}^{0}$ mode is dominated by the
non-resonant component. The statistical significance of the ${B}^{0}_{s}$
signal is determined using Wilks’ theorem [32] and by combining the long and
the downstream samples. The significance including relevant systematic
uncertainties, estimated by repeating the procedure with the signal likelihood
convolved with a Gaussian function of width equal to the sum in quadrature of
the systematic uncertainties, is $7.1$ standard deviations.

Table 1: Signal yields obtained from the fits to ${{K}^{0}_{\rm\scriptscriptstyle S}}{{K}^{\pm}}{{\pi}^{\mp}}$ and ${{K}^{0}_{\rm\scriptscriptstyle S}}{{\pi}^{+}}{{\pi}^{-}}$ mass distributions and corresponding efficiencies. Only statistical contributions to the uncertainty are reported.  Decay | Downstream | Long
---|---|---
| Yield | Efficiency ($\%$) | Yield | Efficiency ($\%$)
${{B}^{0}_{s}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ | $\phantom{0}21\pm\phantom{0}6$ | $0.0174\pm 0.0012$ | $\phantom{0}25\pm\phantom{0}6$ | $0.0121\pm 0.0008$
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ | $\phantom{00}2\pm\phantom{0}3$ | $0.0183\pm 0.0013$ | $\phantom{00}1\pm\phantom{0}2$ | $0.0125\pm 0.0009$
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}{{\pi}^{+}}{{\pi}^{-}}$ | $828\pm 41$ | $0.0336\pm 0.0010$ | $341\pm 23$ | $0.0117\pm 0.0009$

Figure 1: Distribution of (left) ${{K}^{0}_{\rm\scriptscriptstyle
S}}{{K}^{\pm}}{{\pi}^{\mp}}$ mass and (right) ${K}^{\pm}$${\pi}^{\mp}$ mass
for signal candidates with fit results overlaid for (top) downstream and
(bottom) long categories. The data are shown as black points with error bars.
The overall fit is represented by the solid black line. The ${B}^{0}$ and
${B}^{0}_{s}$ signal components are the black short-dashed and dotted lines
respectively, while the non-resonant components are the magenta short-dashed
and dotted lines. The partially reconstructed backgrounds are the red triple-
dotted line (${B}\rightarrow Dh$) and the blue triple-dotted line
(${{B}^{0}_{s}}\\!\rightarrow{{K}^{*0}}\overline{K}^{*0}$). The combinatorial
background is the green long-dash dotted line.

Figure 2:  Distribution of ${{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}$ mass for signal candidates with fit results
overlaid for (left) downstream and (right) long categories. The data are
described by the black points with error bars. The overall fit is represented
by the solid black line. The ${B}^{0}$ and ${B}^{0}_{s}$ signal components are
the black short-dashed and dotted lines. The misidentified ${B}^{0}_{s}$ decay
is the black dashed line, respectively. The partially reconstructed
backgrounds are the red triple-dotted line (${B}\rightarrow Dh$), the blue
triple-dotted line (${{B}^{0}}\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}X$), the violet dash single-dotted line
(${{B}^{0}}\rightarrow\eta^{\prime}{{K}^{0}_{\rm\scriptscriptstyle S}}$) and
the pink short-dash single dotted line
(${{B}^{0}}\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}\gamma$). The combinatorial background is the green
long-dash dotted line. Some of the contributions are small in the figures.

##  5 Systematic uncertainties

The model used to fit data and the limited knowledge in the efficiency
determination are possible sources of systematic uncertainty. Many parameters
in the fit are fixed to values obtained from fits to simulated data. The
associated systematic uncertainties are determined from fits to
pseudoexperiments generated assuming alternative values of the relevant
parameters, corresponding to variations within uncertainties around their
default values. The average difference between the yields determined in the
pseudoexperiments and the nominal value is taken as a systematic uncertainty.

The fit model does not account for the possible interference between the P
wave of the $K^{*}(892)^{0}$ resonance and the S wave from other intermediate
states, e.g. the non-resonant component or the $K^{*}(1430)^{0}$ resonance.
The associated systematic uncertainty is determined by exploiting the
distribution of $\theta_{{{K}^{*0}}}$, defined as the angle between the flight
direction of the $K^{+}$ in the ${K}^{*0}$ rest frame with respect to the
direction of the boost from the laboratory frame to the ${K}^{*0}$ rest frame.
The $\cos\theta_{{{K}^{*0}}}$ distribution is described by a parabola, where
the second-order term represents the signal P wave, the constant term is
related to the S wave and the first-order term accounts for the interference.
Using the sPlot technique [31], the $\cos\theta_{{{K}^{*0}}}$ distribution of
the signal P and S wave is unfolded from the other background components. A
fit in the region of positive $\cos\theta_{{{K}^{*0}}}$ is performed using a
second-order polynomial and the systematic uncertainty is determined as the
relative difference between the integral of the function when the constant
coefficient is allowed to vary and when this coefficient is fixed to zero. Due
to the limited size of the
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ sample, the
relative uncertainty obtained for the
${{B}^{0}_{s}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ decays
is also applied to the ${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}K^{*0}$ decays.

Potential biases that may be associated with the maximum likelihood estimator
are investigated using pseudoexperiments. The systematic uncertainty is
determined as the average difference between the nominal value and the fitted
yields in the pseudoexperiments.

The impact of the limited size of the simulated samples, used to determine the
selection and particle identification efficiencies, is considered as
systematic uncertainty. In addition, the hardware trigger is a potential
source of systematic uncertainty due to imperfections in the description of
data by simulation. A data sample of $D^{*+}\rightarrow D^{0}\pi^{\pm}$, with
${{D}^{0}}\\!\rightarrow K^{-}\pi^{+}$, decays is used to characterise the
trigger efficiencies of the pions and kaons, separated according to particle
charge, as a function of the transverse energy of the associated cluster in
the hadron calorimeter [26, 33]. These data-driven calibration curves are used
to weight simulated events in order to determine the efficiency of the hadron
trigger.

The effective lifetimes of ${B}^{0}_{s}$ meson reconstructed in a particular
decay depend on the $C\\!P$-admixture of the final state because $C\\!P$-even
and $C\\!P$-odd eigenstates may have different lifetimes [34]. Since the
selection efficiency depends on decay time, this might lead to a source of
uncertainty in the measurement. The relative change in efficiency with respect
to the nominal value, estimated for the extreme ranges of possible effective
lifetime distributions, is assigned as the systematic uncertainty.

Finally, the uncertainty from the measurement of the fragmentation fractions
ratio, $f_{s}/f_{d}$ [14, 15, 16], is taken into account. A summary of the
relative uncertainties on the ratio of branching fractions is given in Table
2. The final results reported in Sec. 6 take into account correlations between
the two samples; thus the systematic uncertainty for the combined measurement
is reduced.

Table 2: Systematic uncertainties on the relative branching fraction measurement for the two ${K}^{0}_{\rm\scriptscriptstyle S}$ categories. The uncertainties are quoted as fractional contributions of the relative branching fraction and the total is the sum in quadrature of all contributions.  Source | $\frac{{\cal B}({{B}^{0}_{s}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0})}{{\cal B}({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}{{\pi}^{+}}{{\pi}^{-}})}$ | $\frac{{\cal B}({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0})}{{\cal B}({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}{{\pi}^{+}}{{\pi}^{-}})}$
---|---|---
| Downstream | Long | Downstream | Long
Fit | 0.05 | 0.03 | 0.20 | 0.28
Selection efficiency | 0.08 | 0.10 | 0.08 | 0.11
PID efficiency | 0.01 | 0.01 | 0.01 | 0.01
Trigger | 0.07 | 0.07 | 0.02 | 0.09
Lifetime | 0.05 | 0.05 | - | -
Total | 0.13 | 0.14 | 0.22 | 0.31
$f_{s}/f_{d}$ | 0.06 | 0.06 | - | -

##  6 Summary and conclusion

A search for ${B}_{(s)}^{0}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}K^{*0}$ decays is performed by the LHCb experiment using $pp$ data recorded
at a centre-of-mass energy of $7\mathrm{\,Te\kern-1.00006ptV}$, corresponding
to an integrated luminosity of $1.0$$\mbox{\,fb}^{-1}$. The branching ratios
are determined using the
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}}$ decay as a normalisation mode. The measurements are
performed separately for the downstream and long
${K}^{0}_{\rm\scriptscriptstyle S}$ categories and then combined following
Refs. [35, 36].

The ${B}^{0}_{s}$ decay is observed for the first time, with a total
significance of $7.1$ standard deviations. The relative branching fraction is

$\displaystyle\frac{{\cal
B}{({{B}^{0}_{s}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}K^{*0})}}{{\cal B}{({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}})}}$ $\displaystyle=$ $\displaystyle 0.33\pm 0.07\
(\text{stat})\pm 0.04\ (\text{syst})\pm 0.02\ (f_{s}/f_{d}).$

For the ${B}^{0}$ decay, an upper limit at 90% (95%) confidence level (CL) is
determined. The likelihood function is convolved with a Gaussian function with
standard deviation equal to the total systematic uncertainty, and the upper
limit is taken to be the value of the relative branching fraction below which
90% (95%) of the total integral of the likelihood function over non-negative
branching ratio values is found. The central value and the upper limit on the
relative branching fraction of the decay
${{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0}$ are

$\displaystyle\frac{{\cal
B}{({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle S}}K^{*0})}}{{\cal
B}{({{B}^{0}}\\!\rightarrow{{K}^{0}_{\rm\scriptscriptstyle
S}}{{\pi}^{+}}{{\pi}^{-}})}}$ $\displaystyle=$ $\displaystyle 0.005\pm 0.007\
(\text{stat})\pm 0.001\ (\text{syst}),$ $\displaystyle<$ $\displaystyle 0.020\
(0.021)\;{\rm at}\;90\%\;(95\%)\;\text{CL}.$

The absolute branching fractions, calculated using the reference value of
${\cal B}({{B}^{0}}\\!\rightarrow{{K}^{0}}{{\pi}^{+}}{{\pi}^{-}})=(4.96\pm
0.20)\times 10^{-5}$ [37], determined without using the correlated LHCb
measurement. The results are expressed in terms of the sum of final states
containing either $K^{0}$ or $\bar{K}^{0}$ mesons

$\displaystyle\mathcal{B}(B_{s}^{0}\rightarrow\bar{K}^{0}K^{*}(892)^{0})+\mathcal{B}(B_{s}^{0}\rightarrow
K^{0}\bar{K}^{*}(892)^{0})$ $\displaystyle=$ $\displaystyle(16.4\pm 3.4\pm
1.9\pm 1.0\pm 0.7)\times 10^{-6},$
$\displaystyle\mathcal{B}(B^{0}\rightarrow\bar{K}^{0}K^{*}(892)^{0})+\mathcal{B}(B^{0}\rightarrow
K^{0}\bar{K}^{*}(892)^{0})$ $\displaystyle=$ $\displaystyle(0.25\pm 0.34\pm
0.05\pm 0.01)\times 10^{-6},$ $\displaystyle<$ $\displaystyle 0.96\
(1.04)\times 10^{-6}\;{\rm at}\;90\%\;(95\%)\;\text{CL},$

where the first uncertainty is statistical, the second systematic, the third
due to the ratio of the fragmentation fractions and the fourth due to the
uncertainty on the branching fraction of the normalisation decay. These
results are in agreement with theoretical predictions [6, 7, 8] and can be
used to further constrain phenomenological models.

## Acknowledgements

We express our gratitude to our colleagues in the CERN accelerator departments
for the excellent performance of the LHC. We thank the technical and
administrative staff at the LHCb institutes. We acknowledge support from CERN
and from the national agencies: CAPES, CNPq, FAPERJ and FINEP (Brazil); NSFC
(China); CNRS/IN2P3 (France); BMBF, DFG, HGF and MPG (Germany); INFN (Italy);
FOM and NWO (The Netherlands); MNiSW and NCN (Poland); MEN/IFA (Romania);
MinES and FANO (Russia); MinECo (Spain); SNSF and SER (Switzerland); NASU
(Ukraine); STFC (United Kingdom); NSF (USA). The Tier1 computing centres are
supported by IN2P3 (France), KIT and BMBF (Germany), INFN (Italy), NWO and
SURF (The Netherlands), PIC (Spain), GridPP (United Kingdom). We are indebted
to the communities behind the multiple open source software packages on which
we depend. We are also thankful for the computing resources and the access to
software R&D tools provided by Yandex LLC (Russia). Individual groups or
members have received support from EPLANET, Marie Skłodowska-Curie Actions and
ERC (European Union), Conseil général de Haute-Savoie, Labex ENIGMASS and
OCEVU, Région Auvergne (France), RFBR (Russia), XuntaGal and GENCAT (Spain),
Royal Society and Royal Commission for the Exhibition of 1851 (United
Kingdom). We acknowledge Ulrich Nierste from KIT (Germany) for his assistance
on theoretical aspects of the analysis.

## References

  * [1] A. D. Sakharov, _Violation of $C\\!P$ invariance, C asymmetry, and baryon asymmetry of the Universe_, J. Exp. Theor. Phys. Lett. 5 (1967) 24
  * [2] N. Cabibbo, _Unitary symmetry and leptonic decays_ , Phys. Rev. Lett. 10 (1963) 531
  * [3] M. Kobayashi and T. Maskawa, _$C\\!P$ violation in the renormalizable theory of weak interaction_, Prog. Theor. Phys. 49 (1973) 652
  * [4] A. Riotto and M. Trodden, _Recent progress in baryogenesis_ , Annu. Rev. Nucl. Part. Sci. 49 (1999) 35, arXiv:hep-ph/9901362
  * [5] M. Ciuchini, M. Pierini, and L. Silvestrini, _$B_{s}^{0}\rightarrow K^{(*)0}\overline{K}^{(*)0}$ $C\\!P$ asymmetries: golden channels for New Physics Searches_, Phys. Rev. Lett. 100 (2008) 031802
  * [6] H.-Y. Cheng and C.-K. Chua, _QCD factorization for charmless hadronic ${B}^{0}_{s}$ decays revisited_, Phys. Rev. D80 (2009) 114026, arXiv:0910.5237
  * [7] A. Ali et al., _Charmless nonleptonic ${{B}^{0}_{s}}$ decays to $PP$, $PV$, and $VV$ final states in the perturbative QCD approach_, Phys. Rev. D76 (2007) 074018, arXiv:hep-ph/0703162
  * [8] F. Su, Y.-L. Wu, C. Zhuang, and Y.-B. Yang, _Charmless ${{B}^{0}_{s}}\rightarrow PP,PV,VV$ decays based on the six-quark effective Hamiltonian with strong phase effects II_, Eur. Phys. J. C72 (2012) 1914, arXiv:1107.0136
  * [9] R. H. Dalitz, _On the analysis of $\tau$-meson data and the nature of the $\tau$-meson_, Phil. Mag. 44 (1953) 1068
  * [10] LHCb collaboration, R. Aaij et al., _Study of $B_{(s)}^{0}\rightarrow K^{0}_{S}h^{+}h^{\prime-}$ decays with first observation of $B_{s}^{0}\rightarrow K^{0}_{S}K^{\pm}\pi^{\mp}$ and $B_{s}^{0}\rightarrow K^{0}_{S}\pi^{+}\pi^{-}$_, JHEP 10 (2013) 143, arXiv:1307.7648
  * [11] LHCb collaboration, R. Aaij et al., _Observation of $B^{0}_{s}\rightarrow K^{*\pm}K^{\mp}$ and evidence of $B^{0}_{s}\rightarrow K^{*-}\pi^{+}$ decays_, New J. Phys. 16 (2014) 123001, arXiv:1407.7704
  * [12] BaBar collaboration, B. Aubert et al., _Search for the decay of a ${B}^{0}$ or ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ meson to ${{\kern 1.99997pt\overline{\kern-1.99997ptK}{}}{}^{*0}}{{K}^{0}}$ or ${{K}^{*0}}{{\kern 1.99997pt\overline{\kern-1.99997ptK}{}}{}^{0}}$_, Phys. Rev. D74 (2006) 072008, arXiv:hep-ex/0606050
  * [13] Particle Data Group, K. A. Olive et al., _Review of particle physics_ , Chin. Phys. C38 (2014) 090001
  * [14] LHCb collaboration, R. Aaij et al., _Measurement of $b$ hadron production fractions in 7 TeV $pp$ collisions_, Phys. Rev. D85 (2012) 032008, arXiv:1111.2357
  * [15] LHCb collaboration, R. Aaij et al., _Measurement of the fragmentation fraction ratio $f_{s}/f_{d}$ and its dependence on $B$ meson kinematics_, JHEP 04 (2013) 001, arXiv:1301.5286
  * [16] LHCb collaboration, _Updated average $f_{s}/f_{d}$ $b$-hadron production fraction ratio for $7~{}T\kern-0.50003pteV$ $pp$ collisions_, LHCb-CONF-2013-011
  * [17] LHCb collaboration, A. A. Alves Jr. et al., _The LHCb detector at the LHC_, JINST 3 (2008) S08005
  * [18] LHCb collaboration, R. Aaij et al., _LHCb detector performance_ , Int. J. Mod. Phys. A30 (2015) 1530022, arXiv:1412.6352
  * [19] T. Sjöstrand, S. Mrenna, and P. Skands, _PYTHIA 6.4 physics and manual_ , JHEP 05 (2006) 026, arXiv:hep-ph/0603175
  * [20] I. Belyaev et al., _Handling of the generation of primary events in Gauss, the LHCb simulation framework_ , J. Phys. Conf. Ser. 331 (2011) 032047
  * [21] D. J. Lange, _The EvtGen particle decay simulation package_ , Nucl. Instrum. Meth. A462 (2001) 152
  * [22] P. Golonka and Z. Was, _PHOTOS Monte Carlo: A precision tool for QED corrections in $Z$ and $W$ decays_, Eur. Phys. J. C45 (2006) 97, arXiv:hep-ph/0506026
  * [23] Geant4 collaboration, J. Allison et al., _Geant4 developments and applications_ , IEEE Trans. Nucl. Sci. 53 (2006) 270
  * [24] Geant4 collaboration, S. Agostinelli et al., _Geant4: a simulation toolkit_ , Nucl. Instrum. Meth. A506 (2003) 250
  * [25] M. Clemencic et al., _The LHCb simulation application, Gauss: Design, evolution and experience_, J. Phys. Conf. Ser. 331 (2011) 032023
  * [26] R. Aaij et al., _The LHCb trigger and its performance in 2011_, JINST 8 (2013) P04022, arXiv:1211.3055
  * [27] V. V. Gligorov and M. Williams, _Efficient, reliable and fast high-level triggering using a bonsai boosted decision tree_ , JINST 8 (2013) P02013, arXiv:1210.6861
  * [28] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and regression trees, Wadsworth international group, Belmont, California, USA, 1984
  * [29] R. E. Schapire and Y. Freund, _A decision-theoretic generalization of on-line learning and an application to boosting_ , Jour. Comp. and Syst. Sc. 55 (1997) 119
  * [30] G. Punzi, _Sensitivity of searches for new signals and its optimization_ , in Statistical Problems in Particle Physics, Astrophysics, and Cosmology (L. Lyons, R. Mount, and R. Reitmeyer, eds.), p. 79, 2003.  arXiv:physics/0308063
  * [31] M. Pivk and F. R. Le Diberder, _sPlot: A statistical tool to unfold data distributions_ , Nucl. Instrum. Meth. A555 (2005) 356, arXiv:physics/0402083
  * [32] S. S. Wilks, _The large-sample distribution of the likelihood ratio for testing composite hypotheses_ , Ann. Math. Statist. 9 (1938) 60
  * [33] A. Martin Sanchez, CP violation studies on the $B^{0}\rightarrow DK^{*0}$ decays and hadronic trigger performance with the LHCb detector at CERN, PhD thesis, Université Paris-sud XI, 2013, CERN-THESIS-2013-311
  * [34] I. Dunietz, R. Fleischer, and U. Nierste, _In pursuit of new physics with ${{B}^{0}_{s}}$ decays_, Phys. Rev. D 63 (2001) 114015, arXiv:0012219
  * [35] L. Lyons, D. Gibaut, and P. Clifford, _How to combine correlated estimates of a single physical quantity_ , Nucl. Instrum. Meth. A270 (1988) 110
  * [36] A. Valassi, _Combining correlated measurements of several different physical quantities_ , Nucl. Instrum. Meth. A500 (2003) 391
  * [37] Particle Data Group, J. Beringer et al., _Review of particle physics_ , Phys. Rev. D86 (2012) 010001

LHCb collaboration

R. Aaij38, B. Adeva37, M. Adinolfi46, A. Affolder52, Z. Ajaltouni5, S. Akar6,
J. Albrecht9, F. Alessio38, M. Alexander51, S. Ali41, G. Alkhazov30, P.
Alvarez Cartelle53, A.A. Alves Jr57, S. Amato2, S. Amerio22, Y. Amhis7, L.
An3, L. Anderlini17,g, J. Anderson40, M. Andreotti16,f, J.E. Andrews58, R.B.
Appleby54, O. Aquines Gutierrez10, F. Archilli38, P. d’Argent11, A.
Artamonov35, M. Artuso59, E. Aslanides6, G. Auriemma25,n, M. Baalouch5, S.
Bachmann11, J.J. Back48, A. Badalov36, C. Baesso60, W. Baldini16,38, R.J.
Barlow54, C. Barschel38, S. Barsuk7, W. Barter38, V. Batozskaya28, V.
Battista39, A. Bay39, L. Beaucourt4, J. Beddow51, F. Bedeschi23, I. Bediaga1,
L.J. Bel41, I. Belyaev31, E. Ben-Haim8, G. Bencivenni18, S. Benson38, J.
Benton46, A. Berezhnoy32, R. Bernet40, A. Bertolin22, M.-O. Bettler38, M. van
Beuzekom41, A. Bien11, S. Bifani45, T. Bird54, A. Birnkraut9, A. Bizzeti17,i,
T. Blake48, F. Blanc39, J. Blouw10, S. Blusk59, V. Bocci25, A. Bondar34, N.
Bondar30,38, W. Bonivento15, S. Borghi54, M. Borsato7, T.J.V. Bowcock52, E.
Bowen40, C. Bozzi16, S. Braun11, D. Brett54, M. Britsch10, T. Britton59, J.
Brodzicka54, N.H. Brook46, A. Bursche40, J. Buytaert38, S. Cadeddu15, R.
Calabrese16,f, M. Calvi20,k, M. Calvo Gomez36,p, P. Campana18, D. Campora
Perez38, L. Capriotti54, A. Carbone14,d, G. Carboni24,l, R. Cardinale19,j, A.
Cardini15, P. Carniti20, L. Carson50, K. Carvalho Akiba2,38, R. Casanova
Mohr36, G. Casse52, L. Cassina20,k, L. Castillo Garcia38, M. Cattaneo38, Ch.
Cauet9, G. Cavallero19, R. Cenci23,t, M. Charles8, Ph. Charpentier38, M.
Chefdeville4, S. Chen54, S.-F. Cheung55, N. Chiapolini40, M. Chrzaszcz40, X.
Cid Vidal38, G. Ciezarek41, P.E.L. Clarke50, M. Clemencic38, H.V. Cliff47, J.
Closier38, V. Coco38, J. Cogan6, E. Cogneras5, V. Cogoni15,e, L. Cojocariu29,
G. Collazuol22, P. Collins38, A. Comerma-Montells11, A. Contu15,38, A. Cook46,
M. Coombes46, S. Coquereau8, G. Corti38, M. Corvo16,f, B. Couturier38, G.A.
Cowan50, D.C. Craik48, A. Crocombe48, M. Cruz Torres60, S. Cunliffe53, R.
Currie53, C. D’Ambrosio38, J. Dalseno46, P.N.Y. David41, A. Davis57, K. De
Bruyn41, S. De Capua54, M. De Cian11, J.M. De Miranda1, L. De Paula2, W. De
Silva57, P. De Simone18, C.-T. Dean51, D. Decamp4, M. Deckenhoff9, L. Del
Buono8, N. Déléage4, D. Derkach55, O. Deschamps5, F. Dettori38, B. Dey40, A.
Di Canto38, F. Di Ruscio24, H. Dijkstra38, S. Donleavy52, F. Dordei11, M.
Dorigo39, A. Dosil Suárez37, D. Dossett48, A. Dovbnya43, K. Dreimanis52, L.
Dufour41, G. Dujany54, F. Dupertuis39, P. Durante38, R. Dzhelyadin35, A.
Dziurda26, A. Dzyuba30, S. Easo49,38, U. Egede53, V. Egorychev31, S.
Eidelman34, S. Eisenhardt50, U. Eitschberger9, R. Ekelhof9, L. Eklund51, I. El
Rifai5, Ch. Elsasser40, S. Ely59, S. Esen11, H.M. Evans47, T. Evans55, A.
Falabella14, C. Färber11, C. Farinelli41, N. Farley45, S. Farry52, R. Fay52,
D. Ferguson50, V. Fernandez Albor37, F. Ferrari14, F. Ferreira Rodrigues1, M.
Ferro-Luzzi38, S. Filippov33, M. Fiore16,38,f, M. Fiorini16,f, M. Firlej27, C.
Fitzpatrick39, T. Fiutowski27, K. Fohl38, P. Fol53, M. Fontana10, F.
Fontanelli19,j, R. Forty38, O. Francisco2, M. Frank38, C. Frei38, M.
Frosini17, J. Fu21, E. Furfaro24,l, A. Gallas Torreira37, D. Galli14,d, S.
Gallorini22,38, S. Gambetta50, M. Gandelman2, P. Gandini55, Y. Gao3, J. García
Pardiñas37, J. Garofoli59, J. Garra Tico47, L. Garrido36, D. Gascon36, C.
Gaspar38, R. Gauld55, L. Gavardi9, G. Gazzoni5, A. Geraci21,v, D. Gerick11, E.
Gersabeck11, M. Gersabeck54, T. Gershon48, Ph. Ghez4, A. Gianelle22, S.
Gianì39, V. Gibson47, O. G. Girard39, L. Giubega29, V.V. Gligorov38, C.
Göbel60, D. Golubkov31, A. Golutvin53,31,38, A. Gomes1,a, C. Gotti20,k, M.
Grabalosa Gándara5, R. Graciani Diaz36, L.A. Granado Cardoso38, E. Graugés36,
E. Graverini40, G. Graziani17, A. Grecu29, E. Greening55, S. Gregson47, P.
Griffith45, L. Grillo11, O. Grünberg63, B. Gui59, E. Gushchin33, Yu. Guz35,38,
T. Gys38, C. Hadjivasiliou59, G. Haefeli39, C. Haen38, S.C. Haines47, S.
Hall53, B. Hamilton58, T. Hampson46, X. Han11, S. Hansmann-Menzemer11, N.
Harnew55, S.T. Harnew46, J. Harrison54, J. He38, T. Head39, V. Heijne41, K.
Hennessy52, P. Henrard5, L. Henry8, J.A. Hernando Morata37, E. van
Herwijnen38, M. Heß63, A. Hicheur2, D. Hill55, M. Hoballah5, C. Hombach54, W.
Hulsbergen41, T. Humair53, N. Hussain55, D. Hutchcroft52, D. Hynds51, M.
Idzik27, P. Ilten56, R. Jacobsson38, A. Jaeger11, J. Jalocha55, E. Jans41, A.
Jawahery58, F. Jing3, M. John55, D. Johnson38, C.R. Jones47, C. Joram38, B.
Jost38, N. Jurik59, S. Kandybei43, W. Kanso6, M. Karacson38, T.M. Karbach38,†,
S. Karodia51, M. Kelsey59, I.R. Kenyon45, M. Kenzie38, T. Ketel42, B.
Khanji20,38,k, C. Khurewathanakul39, S. Klaver54, K. Klimaszewski28, O.
Kochebina7, M. Kolpin11, I. Komarov39, R.F. Koopman42, P. Koppenburg41,38, L.
Kravchuk33, K. Kreplin11, M. Kreps48, G. Krocker11, P. Krokovny34, F. Kruse9,
W. Kucewicz26,o, M. Kucharczyk26, V. Kudryavtsev34, A. K. Kuonen39, K.
Kurek28, T. Kvaratskheliya31, V.N. La Thi39, D. Lacarrere38, G. Lafferty54, A.
Lai15, D. Lambert50, R.W. Lambert42, G. Lanfranchi18, C. Langenbruch48, B.
Langhans38, T. Latham48, C. Lazzeroni45, R. Le Gac6, J. van Leerdam41, J.-P.
Lees4, R. Lefèvre5, A. Leflat32,38, J. Lefrançois7, O. Leroy6, T. Lesiak26, B.
Leverington11, Y. Li7, T. Likhomanenko65,64, M. Liles52, R. Lindner38, C.
Linn38, F. Lionetto40, B. Liu15, X. Liu3, S. Lohn38, I. Longstaff51, J.H.
Lopes2, D. Lucchesi22,r, M. Lucio Martinez37, H. Luo50, A. Lupato22, E.
Luppi16,f, O. Lupton55, F. Machefert7, F. Maciuc29, O. Maev30, K. Maguire54,
S. Malde55, A. Malinin64, G. Manca7, G. Mancinelli6, P. Manning59, A.
Mapelli38, J. Maratas5, J.F. Marchand4, U. Marconi14, C. Marin Benito36, P.
Marino23,38,t, R. Märki39, J. Marks11, G. Martellotti25, M. Martinelli39, D.
Martinez Santos42, F. Martinez Vidal66, D. Martins Tostes2, A. Massafferri1,
R. Matev38, A. Mathad48, Z. Mathe38, C. Matteuzzi20, K. Matthieu11, A.
Mauri40, B. Maurin39, A. Mazurov45, M. McCann53, J. McCarthy45, A. McNab54, R.
McNulty12, B. Meadows57, F. Meier9, M. Meissner11, M. Merk41, D.A. Milanes62,
M.-N. Minard4, D.S. Mitzel11, J. Molina Rodriguez60, S. Monteil5, M.
Morandin22, P. Morawski27, A. Mordà6, M.J. Morello23,t, J. Moron27, A.B.
Morris50, R. Mountain59, F. Muheim50, J. Müller9, K. Müller40, V. Müller9, M.
Mussini14, B. Muster39, P. Naik46, T. Nakada39, R. Nandakumar49, I. Nasteva2,
M. Needham50, N. Neri21, S. Neubert11, N. Neufeld38, M. Neuner11, A.D.
Nguyen39, T.D. Nguyen39, C. Nguyen-Mau39,q, V. Niess5, R. Niet9, N. Nikitin32,
T. Nikodem11, D. Ninci23, A. Novoselov35, D.P. O’Hanlon48, A. Oblakowska-
Mucha27, V. Obraztsov35, S. Ogilvy51, O. Okhrimenko44, R. Oldeman15,e, C.J.G.
Onderwater67, B. Osorio Rodrigues1, J.M. Otalora Goicochea2, A. Otto38, P.
Owen53, A. Oyanguren66, A. Palano13,c, F. Palombo21,u, M. Palutan18, J.
Panman38, A. Papanestis49, M. Pappagallo51, L.L. Pappalardo16,f, C. Parkes54,
G. Passaleva17, G.D. Patel52, M. Patel53, C. Patrignani19,j, A. Pearce54,49,
A. Pellegrino41, G. Penso25,m, M. Pepe Altarelli38, S. Perazzini14,d, P.
Perret5, L. Pescatore45, K. Petridis46, A. Petrolini19,j, E. Picatoste
Olloqui36, B. Pietrzyk4, T. Pilař48, D. Pinci25, A. Pistone19, A. Piucci11, S.
Playfer50, M. Plo Casasus37, T. Poikela38, F. Polci8, A. Poluektov48,34, I.
Polyakov31, E. Polycarpo2, A. Popov35, D. Popov10,38, B. Popovici29, C.
Potterat2, E. Price46, J.D. Price52, J. Prisciandaro39, A. Pritchard52, C.
Prouve46, V. Pugatch44, A. Puig Navarro39, G. Punzi23,s, W. Qian4, R.
Quagliani7,46, B. Rachwal26, J.H. Rademacker46, B. Rakotomiaramanana39, M.
Rama23, M.S. Rangel2, I. Raniuk43, N. Rauschmayr38, G. Raven42, F. Redi53, S.
Reichert54, M.M. Reid48, A.C. dos Reis1, S. Ricciardi49, S. Richards46, M.
Rihl38, K. Rinnert52, V. Rives Molina36, P. Robbe7,38, A.B. Rodrigues1, E.
Rodrigues54, J.A. Rodriguez Lopez62, P. Rodriguez Perez54, S. Roiser38, V.
Romanovsky35, A. Romero Vidal37, M. Rotondo22, J. Rouvinet39, T. Ruf38, H.
Ruiz36, P. Ruiz Valls66, J.J. Saborido Silva37, N. Sagidova30, P. Sail51, B.
Saitta15,e, V. Salustino Guimaraes2, C. Sanchez Mayordomo66, B. Sanmartin
Sedes37, R. Santacesaria25, C. Santamarina Rios37, M. Santimaria18, E.
Santovetti24,l, A. Sarti18,m, C. Satriano25,n, A. Satta24, D.M. Saunders46, D.
Savrina31,32, M. Schiller38, H. Schindler38, M. Schlupp9, M. Schmelling10, T.
Schmelzer9, B. Schmidt38, O. Schneider39, A. Schopper38, M. Schubiger39, M.-H.
Schune7, R. Schwemmer38, B. Sciascia18, A. Sciubba25,m, A. Semennikov31, I.
Sepp53, N. Serra40, J. Serrano6, L. Sestini22, P. Seyfert11, M. Shapkin35, I.
Shapoval16,43,f, Y. Shcheglov30, T. Shears52, L. Shekhtman34, V. Shevchenko64,
A. Shires9, R. Silva Coutinho48, G. Simi22, M. Sirendi47, N. Skidmore46, I.
Skillicorn51, T. Skwarnicki59, E. Smith55,49, E. Smith53, I. T. Smith50, J.
Smith47, M. Smith54, H. Snoek41, M.D. Sokoloff57,38, F.J.P. Soler51, F.
Soomro39, D. Souza46, B. Souza De Paula2, B. Spaan9, P. Spradlin51, S.
Sridharan38, F. Stagni38, M. Stahl11, S. Stahl38, O. Steinkamp40, O.
Stenyakin35, F. Sterpka59, S. Stevenson55, S. Stoica29, S. Stone59, B.
Storaci40, S. Stracka23,t, M. Straticiuc29, U. Straumann40, L. Sun57, W.
Sutcliffe53, K. Swientek27, S. Swientek9, V. Syropoulos42, M. Szczekowski28,
P. Szczypka39,38, T. Szumlak27, S. T’Jampens4, T. Tekampe9, M. Teklishyn7, G.
Tellarini16,f, F. Teubert38, C. Thomas55, E. Thomas38, J. van Tilburg41, V.
Tisserand4, M. Tobin39, J. Todd57, S. Tolk42, L. Tomassetti16,f, D. Tonelli38,
S. Topp-Joergensen55, N. Torr55, E. Tournefier4, S. Tourneur39, K. Trabelsi39,
M.T. Tran39, M. Tresch40, A. Trisovic38, A. Tsaregorodtsev6, P. Tsopelas41, N.
Tuning41,38, A. Ukleja28, A. Ustyuzhanin65,64, U. Uwer11, C. Vacca15,e, V.
Vagnoni14, G. Valenti14, A. Vallier7, R. Vazquez Gomez18, P. Vazquez
Regueiro37, C. Vázquez Sierra37, S. Vecchi16, J.J. Velthuis46, M. Veltri17,h,
G. Veneziano39, M. Vesterinen11, B. Viaud7, D. Vieira2, M. Vieites Diaz37, X.
Vilasis-Cardona36,p, A. Vollhardt40, D. Volyanskyy10, D. Voong46, A.
Vorobyev30, V. Vorobyev34, C. Voß63, J.A. de Vries41, R. Waldi63, C.
Wallace48, R. Wallace12, J. Walsh23, S. Wandernoth11, J. Wang59, D.R. Ward47,
N.K. Watson45, D. Websdale53, A. Weiden40, M. Whitehead48, D. Wiedner11, G.
Wilkinson55,38, M. Wilkinson59, M. Williams38, M.P. Williams45, M. Williams56,
T. Williams45, F.F. Wilson49, J. Wimberley58, J. Wishahi9, W. Wislicki28, M.
Witek26, G. Wormser7, S.A. Wotton47, S. Wright47, K. Wyllie38, Y. Xie61, Z.
Xu39, Z. Yang3, J. Yu61, X. Yuan34, O. Yushchenko35, M. Zangoli14, M.
Zavertyaev10,b, L. Zhang3, Y. Zhang3, A. Zhelezov11, A. Zhokhov31, L. Zhong3.

1Centro Brasileiro de Pesquisas Físicas (CBPF), Rio de Janeiro, Brazil
2Universidade Federal do Rio de Janeiro (UFRJ), Rio de Janeiro, Brazil
3Center for High Energy Physics, Tsinghua University, Beijing, China
4LAPP, Université Savoie Mont-Blanc, CNRS/IN2P3, Annecy-Le-Vieux, France
5Clermont Université, Université Blaise Pascal, CNRS/IN2P3, LPC, Clermont-
Ferrand, France
6CPPM, Aix-Marseille Université, CNRS/IN2P3, Marseille, France
7LAL, Université Paris-Sud, CNRS/IN2P3, Orsay, France
8LPNHE, Université Pierre et Marie Curie, Université Paris Diderot,
CNRS/IN2P3, Paris, France
9Fakultät Physik, Technische Universität Dortmund, Dortmund, Germany
10Max-Planck-Institut für Kernphysik (MPIK), Heidelberg, Germany
11Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg, Heidelberg,
Germany
12School of Physics, University College Dublin, Dublin, Ireland
13Sezione INFN di Bari, Bari, Italy
14Sezione INFN di Bologna, Bologna, Italy
15Sezione INFN di Cagliari, Cagliari, Italy
16Sezione INFN di Ferrara, Ferrara, Italy
17Sezione INFN di Firenze, Firenze, Italy
18Laboratori Nazionali dell’INFN di Frascati, Frascati, Italy
19Sezione INFN di Genova, Genova, Italy
20Sezione INFN di Milano Bicocca, Milano, Italy
21Sezione INFN di Milano, Milano, Italy
22Sezione INFN di Padova, Padova, Italy
23Sezione INFN di Pisa, Pisa, Italy
24Sezione INFN di Roma Tor Vergata, Roma, Italy
25Sezione INFN di Roma La Sapienza, Roma, Italy
26Henryk Niewodniczanski Institute of Nuclear Physics Polish Academy of
Sciences, Kraków, Poland
27AGH - University of Science and Technology, Faculty of Physics and Applied
Computer Science, Kraków, Poland
28National Center for Nuclear Research (NCBJ), Warsaw, Poland
29Horia Hulubei National Institute of Physics and Nuclear Engineering,
Bucharest-Magurele, Romania
30Petersburg Nuclear Physics Institute (PNPI), Gatchina, Russia
31Institute of Theoretical and Experimental Physics (ITEP), Moscow, Russia
32Institute of Nuclear Physics, Moscow State University (SINP MSU), Moscow,
Russia
33Institute for Nuclear Research of the Russian Academy of Sciences (INR RAN),
Moscow, Russia
34Budker Institute of Nuclear Physics (SB RAS) and Novosibirsk State
University, Novosibirsk, Russia
35Institute for High Energy Physics (IHEP), Protvino, Russia
36Universitat de Barcelona, Barcelona, Spain
37Universidad de Santiago de Compostela, Santiago de Compostela, Spain
38European Organization for Nuclear Research (CERN), Geneva, Switzerland
39Ecole Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland
40Physik-Institut, Universität Zürich, Zürich, Switzerland
41Nikhef National Institute for Subatomic Physics, Amsterdam, The Netherlands
42Nikhef National Institute for Subatomic Physics and VU University Amsterdam,
Amsterdam, The Netherlands
43NSC Kharkiv Institute of Physics and Technology (NSC KIPT), Kharkiv, Ukraine
44Institute for Nuclear Research of the National Academy of Sciences (KINR),
Kyiv, Ukraine
45University of Birmingham, Birmingham, United Kingdom
46H.H. Wills Physics Laboratory, University of Bristol, Bristol, United
Kingdom
47Cavendish Laboratory, University of Cambridge, Cambridge, United Kingdom
48Department of Physics, University of Warwick, Coventry, United Kingdom
49STFC Rutherford Appleton Laboratory, Didcot, United Kingdom
50School of Physics and Astronomy, University of Edinburgh, Edinburgh, United
Kingdom
51School of Physics and Astronomy, University of Glasgow, Glasgow, United
Kingdom
52Oliver Lodge Laboratory, University of Liverpool, Liverpool, United Kingdom
53Imperial College London, London, United Kingdom
54School of Physics and Astronomy, University of Manchester, Manchester,
United Kingdom
55Department of Physics, University of Oxford, Oxford, United Kingdom
56Massachusetts Institute of Technology, Cambridge, MA, United States
57University of Cincinnati, Cincinnati, OH, United States
58University of Maryland, College Park, MD, United States
59Syracuse University, Syracuse, NY, United States
60Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio), Rio de
Janeiro, Brazil, associated to 2
61Institute of Particle Physics, Central China Normal University, Wuhan,
Hubei, China, associated to 3
62Departamento de Fisica , Universidad Nacional de Colombia, Bogota, Colombia,
associated to 8
63Institut für Physik, Universität Rostock, Rostock, Germany, associated to 11
64National Research Centre Kurchatov Institute, Moscow, Russia, associated to
31
65Yandex School of Data Analysis, Moscow, Russia, associated to 31
66Instituto de Fisica Corpuscular (IFIC), Universitat de Valencia-CSIC,
Valencia, Spain, associated to 36
67Van Swinderen Institute, University of Groningen, Groningen, The
Netherlands, associated to 41

aUniversidade Federal do Triângulo Mineiro (UFTM), Uberaba-MG, Brazil
bP.N. Lebedev Physical Institute, Russian Academy of Science (LPI RAS),
Moscow, Russia
cUniversità di Bari, Bari, Italy
dUniversità di Bologna, Bologna, Italy
eUniversità di Cagliari, Cagliari, Italy
fUniversità di Ferrara, Ferrara, Italy
gUniversità di Firenze, Firenze, Italy
hUniversità di Urbino, Urbino, Italy
iUniversità di Modena e Reggio Emilia, Modena, Italy
jUniversità di Genova, Genova, Italy
kUniversità di Milano Bicocca, Milano, Italy
lUniversità di Roma Tor Vergata, Roma, Italy
mUniversità di Roma La Sapienza, Roma, Italy
nUniversità della Basilicata, Potenza, Italy
oAGH - University of Science and Technology, Faculty of Computer Science,
Electronics and Telecommunications, Kraków, Poland
pLIFAELS, La Salle, Universitat Ramon Llull, Barcelona, Spain
qHanoi University of Science, Hanoi, Viet Nam
rUniversità di Padova, Padova, Italy
sUniversità di Pisa, Pisa, Italy
tScuola Normale Superiore, Pisa, Italy
uUniversità degli Studi di Milano, Milano, Italy
vPolitecnico di Milano, Milano, Italy
†Deceased


11institutetext:  Hong Kong Baptist University, Hong Kong, China
22institutetext: University of Southern California, CA, USA 33institutetext:
Hangzhou Dianzi University, Hang Zhou, China 44institutetext: The Hong Kong
University of Science and Technology (Guangzhou), China

# Evolutionary Multi-objective Architecture Search Framework: Application to
COVID-19 3D CT Classification

Xin He 11 Guohao Ying 22 Jiyong Zhang$\dagger$ 33 Xiaowen Chu$\dagger$
$\dagger$Corresponding authors<EMAIL_ADDRESS>jzhang@hdu.edu.cn).4411

###### Abstract

The COVID-19 pandemic has threatened global health. Many studies have applied
deep convolutional neural networks (CNN) to recognize COVID-19 based on chest
3D computed tomography (CT). Recent works show that no model generalizes well
across CT datasets from different countries, and manually designing models for
specific datasets requires expertise; thus, neural architecture search (NAS)
that aims to search models automatically has become an attractive solution. To
reduce the search cost on large 3D CT datasets, most NAS-based works use the
weight-sharing (WS) strategy to make all models share weights within a
supernet; however, WS inevitably incurs search instability, leading to
inaccurate model estimation. In this work, we propose an efficient
Evolutionary Multi-objective ARchitecture Search (EMARS) framework. We propose
a new objective, namely potential, which can help exploit promising models to
indirectly reduce the number of models involved in weights training, thus
alleviating search instability. We demonstrate that under objectives of
accuracy and potential, EMARS can balance exploitation and exploration, i.e.,
reducing search time and finding better models. Our searched models are small
and perform better than prior works on three public COVID-19 3D CT datasets.

###### Keywords:

COVID-19 Neural Architecture Search (NAS) Weight-sharing Evolutionary
Algorithm (EA) 3D Computed Tomograph (CT)

##  1 Introduction

The rapid spread of coronavirus disease 2019 (COVID-19) pandemic has
threatened global health. Isolating infected patients is an effective way to
block the transmission of the virus. Thus, fast and accurate methods to detect
infected patients are crucial. Chest CT is relatively easy to perform and has
been proved an important complement to nucleic acid test [7]. However, there
is a serious lack of radiologists during the pandemic. Many researchers have
applied deep learning (DL) techniques to assist CT diagnosis. For COVID-19 3D
CT classification, there are two mainstream CNN-based methods: 1) multiview-
based methods [22, 15] use 2D CNN to extract features for each 2D CT slice and
then fuse these features to make predictions; and 2) voxel-based methods [8,
32] feed 3D CNNs with 3D CT scans to make full use of the geometric
information. He et al. [9] benchmark a series of hand-crafted 2D and 3D CNNs
and demonstrate that 3D CNNs generally outperform 2D CNNs.

Some recent works [8, 11] benchmark multiple COVID-19 datasets from different
countries and find that no model can maintain absolute advantages on different
datasets. However, since it is difficult to design models manually for
specific datasets, the neural architecture search (NAS) [10, 6] has become an
attractive solution to discover superior models without human assistance.
Reinforcement learning [33, 21], gradient descent (GD) [18], and evolutionary
algorithm (EA) [30, 23] are three mainstream NAS methods. The comparative
results of a recent survey [10] show that the EA-based NAS can discover better
networks than other types of NAS methods. However, the better performance of
EA-based NAS is at the cost of more computing resources because they need to
retrain all searched models to compare their performance, e.g., AmoebaNet [23]
took 3,150 GPU days to search. Thanks to the weight-sharing method [21, 29],
any model can be evaluated without retraining, and Yang et al. [30] reduced
the search time of the EA-based NAS to 0.4 GPU days. NAS was originally
proposed for large-scale 2D image tasks. Although some works [9, 8] have
extended NAS to search 3D models for COVID-19 3D datasets, they suffered from
the search instability (analyzed in Sec. 3.1) incurred by weight-sharing,
which leads to fluctuation in the search process and even worse results than
random search in some cases. In this work, we propose an efficient
Evolutionary Multi-objective ARchitecture Search framework, dubbed as EMARS.
We summarize our contributions below.

  1. 1.

We propose a new objective, i.e., potential, which can help exploit promising
models and indirectly reduce the number of models involved in weights
training, thereby alleviating search instability.

  2. 2.

We demonstrate that compared to conventional objective settings (e.g., only
considering accuracy), EMARS that aims at accuracy, potential, and small size
objectives can trade-off between exploitation and exploration, reducing search
time by 22% on average and discovering better models.

  3. 3.

Our searched models are small in size and outperform prior works [9, 8] on
three public datasets: CC-CCII [31], MosMed [20], and Covid-CTset [22].

##  2 Preliminaries

In this section, we describe the common basis of weight-sharing neural
architecture search (NAS) [29]. NAS is formulated as a bi-level optimization
problem:

$\begin{array}[]{cl}\min_{\alpha}&L_{\text{val }}\left(w^{*},\alpha\right)\\\
\text{ s.t. }&w^{*}=\operatorname{argmin}_{w}L_{\text{train
}}(w,\alpha)\end{array}$ (1)

where $L_{\text{train}}$ and $L_{\text{val}}$ indicate the training and
validation loss; $w$ and $\alpha$ indicate the weights and architecture of a
candidate model. The early NAS methods [33, 23] search and evaluate the
networks by retraining them from scratch, resulting in huge computational
cost. To reduce the burden, the weight-sharing strategy [29] was proposed, in
which the SuperNet $\mathcal{N}$ contains all possible architectures (subnets)
and its weights $\mathcal{W}$ are shared among these subnets. The architecture
and weights of each subnet are denoted by $\mathcal{N}(\alpha)$ and
$\mathcal{W}(\alpha)$, respectively, where $\alpha$ is the subnet
architecture, encoded by one-hot sequences (described in Sec. 3.3). The loss
of a subnet is expressed as
$L(\alpha)=L(\mathcal{N(\alpha)},\mathcal{W(\alpha)},X,Y)$, where $L,X,Y$
indicate the loss function, input data, and target, respectively, and the
gradient of subnet weights is $\nabla_{\mathcal{W}(\alpha)}=\frac{\partial
L(\alpha)}{\partial\mathcal{W}}$. Then gradients of SuperNet weights
$\mathcal{W}$ can be calculated as the average gradient of all subnets, i.e.,
$\nabla_{\mathcal{W}}=\frac{1}{N}\sum_{i=1}^{N}\nabla_{\mathcal{W}(\alpha_{i})}=\frac{1}{N}\sum_{i=1}^{N}\frac{\partial
L(\alpha_{i})}{\partial\mathcal{W}}$, where $N$ is the total number of
subnets. Obviously, it is not practical to use all subnets to update SuperNet
weights at each time. Therefore, we use a mini-batch of subnets for training,
detailed as Eq. 2

$\nabla_{\mathcal{W}}\approx\frac{1}{M}\sum_{i=1}^{M}\nabla_{\mathcal{W}(\alpha_{i})}$
(2)

where $M$ is the number of subnets sampled in a mini-batch and $M<<N$. In our
experiments, we find that $M=1$ works just fine, i.e., we can update
$\mathcal{W}$ using the gradient from any single sampled subnets for each
training batch.

##  3 Methodology

###  3.1 Potential objective: Alleviating Search Instability

By instability, we mean that the same subnet can produce a completely
different performance at different times of the search process. The
instability is caused by the weight-sharing strategy because the weights of
all subnets are coupled, then an update of any subnet’s weights is bound to
affect (usually negatively) other subnets. Therefore, the performance of a
subnet at a specific time does not necessarily represent its real performance
but instead misleads the direction of evolutionary search (described in Sec.
3.2). To mitigate the search instability caused by weight-sharing, a natural
idea is to reduce the number of models involved in weights training (i.e., Eq.
2). For this reason, some works [13, 2] directly reduce the number of models
by progressively shrinking the search space based on the model performance,
but this may eliminate promising models in the early stage of the search. To
avoid this problem, we take an indirect approach in which we keep exploring
various models in the early stage of the search and then spend more effort on
training those promising models in the later stage of the search. In this way,
we can indirectly reduce the number of models involved in weights training
without deliberately reducing the search space. However, how do we determine
whether a model is promising or not?

Here, we propose a new objective, namely potential, to help find promising
models. Specifically, for each sampled model, we maintain and update its
historical performances $Z=(E,F)$, where $E=[e_{1},...,e_{m}]^{T}$ is a column
vector recording the epochs when the model is sampled,
$F=[f_{1},...,f_{m}]^{T}$ is a column vector recording the corresponding
validation accuracy. Note that, $Z$ is dynamically updated with the search
process, so the size of $Z$ (i.e., $m$) varies for models. The potential
$\mathcal{P}$ of a model is calculated by ordinary least squares (OLS):

$\mathcal{P}=(E^{T}E)^{-1}E^{T}F$ (3)

To some extent, $E$ can also reflect how promising a model is, e.g., if $E$ is
densely distributed, it means this model outperforms other models in multiple
rounds of search and hence wins more chances to be sampled. However,
considering only $E$ will exacerbate the Matthew effect, and the search may
get trapped in a local optimum. Our proposed potential solves this problem by
considering the coupling relation between sampling frequency $E$ and
validation accuracy $F$, i.e., the growing trend of accuracy rather than the
accuracy at a specific time. The larger the $\mathcal{P}$ value, the more
promising the model is.

###  3.2 Evolutionary Search

The search algorithm (see Supplement Alg. 1) starts with a warm-up stage,
followed by the evolutionary search stage. In the warm-up, the SuperNet is
trained by uniformly sampling subnets, thus all candidate operations are
trained equally. After the warm-up, top-$P$ best-performing subnets form the
initial population, i.e., $\mathcal{A}^{(0)}$, and will be evolved for
multiple generations. Each generation comprises two sequential processes: 1)
weights training, where each individual (i.e., subnet) is selected from the
population and trained based on Eq 2; and 2) architecture search, comprising
selection, crossover, and mutation (see Fig. 1).

Figure 1:  Overview of search space and search method. Upper-right:
MBConv$3\\_3$, where C, D, H, W indicate channels, depth, height, and width.
Lower-right: An example of exploitation and exploration under different
objectives. (best viewed in color)

Selection. After weights training, we record multiple objectives for all
individuals in the population. We adopt NSGA-II [4] method to select Pareto-
front individuals under the recorded objectives from the population. We
compare different combinations of these objectives in Sec. 4.2 and find that
searching with potential and accuracy can discover better models with less
cost.

Crossover&Mutation. The selection produces $K$ Pareto-front individuals, based
on which we further generate $P-K$ new individuals. Each new individual is
generated by randomly sampling from the SuperNet or performing crossover and
mutation (CM) with certain probabilities. The basic unit of CM is the one-hot
sequence, representing the candidate operation (see Fig. 1).

Exploitation&Exploration. Fig. 1 (lower-right) shows an example of two
important issues in the evolutionary algorithm (EA) based search: exploration
and exploitation. Exploitation prefers the current optimal solution, which
reduces search cost but may lead to a local optimum; exploration is more
likely to find the optimal solution but consumes more resources. The common
opinion about EA is that the steps of crossover and mutation determine the
exploration, and exploitation is done by selection. However, our experiments
in Sec. 4.2 show that setting different objectives in the selection step can
also control the evolution direction. Specifically, accuracy and potential
will make the evolution process towards exploration and exploitation,
respectively, while combining accuracy and potential can balance exploration
and exploitation.

###  3.3 Search Space

SuperNet. The search space is represented by a SuperNet $\mathcal{N}$,
containing all possible subnets. SuperNet comprises two parts: 1) the
searchable part, i.e., $N=6$ layers; 2) the fixed part, i.e., stem block,
global average pooling [17], and a fully connected layer. The stem block is a
standard $3\times 3\times 3$ 3D convolution followed by a 3D batch
normalization and a ReLu6 activation function [12].

Layer. The $i$-th layer comprises a calibration block and $B_{i}$ searchable
blocks. The calibration block is a 3D $1\times 1\times 1$ point-wise
convolution to solve the problem of feature dimension mismatch; thus, all
subsequent blocks have a stride of 1. The number of searchable blocks and the
stride of calibration block in six layers are [4,4,4,4,4,1] and [2,2,2,1,2,1],
respectively. The output channels of the stem block and six layers are 32 and
[24,40,80,96,192,320], respectively.

Block. Each searchable block is a candidate operation, encoded by a one-hot
sequence. We adopt eight candidate operations, including a skip-connection
operation and seven mobile inverted bottleneck convolutions [24], denoted by
MBConv$k\\_e$, where
$k\\_e\in\\{3\\_3,3\\_4,3\\_6,5\\_3,5\\_4,7\\_3,7\\_4\\}$, $k$ is the kernel
size of the intermediate depth-wise convolution (DWConv), and $e$ is the
expansion ratio between the input channel and inner channel of MBConv.

##  4 Experiments

###  4.1 Implementation Details

Datasets. For a fair comparison, we apply the same three datasets as prior
works [9, 8]. CC-CCII [31] has 3,993 CT scans of three classes: novel
coronavirus pneumonia (NCP), common pneumonia (CP), and normal case; MosMed
[20] has 1,110 scans of NCP and normal classes; Covid-CTset [22] has 526 scans
of NCP and normal classes. More details of datasets can be referred to
supplement.

Search stage. We use four Nvidia V100 GPUs to search for 100 epochs, where the
warm-up stage has 10 epochs. During each search epoch, a population of models
are equally trained on the training set and evaluated on the validation set.
The population size is 20, where 10 Pareto-front models are selected from the
population using NSGA-II [4] under multiple objectives (e.g., validation
accuracy, potential, and model size), and 10 new models are generated by
crossover and mutation with the probabilities of 0.3 and 0.2. To improve
search efficiency, we set the input size ($width\times height\times depth$) to
$64\times 64\times 16$. We use Adam optimizer [14] with a weight decay of 3e-4
and an initial learning rate of 0.001.

Retraining stage. After the search stage, we combine the training and
validation set and retrain the Pareto-front models on the combined set for 200
epochs. We use the same Adam settings as the search stage. The 3D input sizes
of CC-CCII, MosMed, and Covid-CTset datasets are $128\times 128\times 32$,
$256\times 256\times 40$, and $512\times 512\times 32$, respectively. Our
framework is based on NNI [19] and available at:
https://github.com/marsggbo/MICCAI2022-EMARS.

###  4.2 Results and Analysis

Figure 2:  The model size-aware search results. X and Y axes indicate model
size and validation accuracy (Acc). The purple and yellow points indicate the
sampled models in the first and last half of the search stage, respectively.
(best viewed in color)  Figure 3:  The potential (P) aware search results.
Different colored points indicate the models sampled in different epoch
periods. The solid and dashed lines in each period indicate the average and
25/75 percentile accuracy, respectively. (best viewed in color)

Model Size-aware Search. Fig. 2 presents model size-aware search results on
CC-CCII dataset. Fig. 2 (a) shows that searching under only validation
accuracy (Acc) will explore both extremes of model size, but with no
performance gain, while Fig. 2 (b)&(c) show that additional consideration of
model size on top of Acc helps find better models in the later stage,
indicating multi-objective can facilitate the search process. Besides,
compared to Fig. 2 (b), searching under Acc and small model size in Fig. 2 (c)
can not only reduce search time from 9.31 hours to 8.46 hours but also
discover competitive models.

Potential-aware Search. We further build three experiments on the CC-CCII
dataset to validate potential objective. Each sub-figure of Fig. 3 divides the
search process into four periods based on the search epoch. Each period is
presented with different colors and marked with the accuracy of 25/50/75th
percentiles. Fig. 3 (a) shows that searching under Acc tends to explore more
models, regardless of whether the model performance is good or bad, leading to
wasting time on those unpromising models (lower-right points). On the
contrary, in Fig. 3 (b), the difference between the 25th and 75th percentiles
and the number of sampled models are gradually reduced with the search
process, which implies that potential will guide the evolution process in the
later stage to exploit promising models already discovered. Although it
reduces search time, it has lower Acc due to being trapped in local optima in
the early stage. Fig. 3 (c) shows that searching under potential, Acc, and
small size can reduce the search time by 19% on average and balance
exploitation and exploration. Specifically, the first two periods are
dominated by exploration, as a wide accuracy range of models is explored, and
we can find models with an accuracy of more than 0.7 faster in the second
period. On the other hand, the last two periods focus more on exploitation, as
the number of unpromising models is significantly reduced, and the accuracy of
25/50/75th percentiles is improved steadily.

Comparison with Prior Works. Table. 1 compares our searched models with prior
works based on four widely used metrics: accuracy, precision, sensitivity, and
f1 score. Precision and sensitivity are a pair of negatively correlated
metrics, so they cannot fully describe model performance. F1 score is the
harmonic mean of the precision and sensitivity; thus, it is a better metric.
As can be seen, our models searched under APS (accuracy, potential, and small
model size) objectives have small sizes and outperform all prior hand-crafted
and NAS-based models on three datasets in terms of accuracy, precision, and f1
score. Besides, MosMed is an imbalanced dataset, and we can find that the
models (e.g., CovidNet3D-S/L and EMARS-A) searched without potential are
overfitted on positive class (i.e., NCP), as they have extremely high
sensitivity but low precision. On the contrary, EMARS-P and EMARS-APS are
searched with potential objective, balancing precision and sensitivity well
and achieving higher accuracy and f1 scores. More results can be referred to
the Supplement.

Table 1: Results on CC-CCII [31], MosMed [20], and Covid-CTset [22] datasets.
A, P, and S in our model name indicate accuracy, potential, and small model
size, e.g., EMARS-A indicates the model searched under the accuracy objective.

Dataset | Model | Size (MB) | Type |  Accuracy |  Precision |  Sensitivity |  F1
---|---|---|---|---|---|---|---
CC- CCII [China] [31] | ResNet3D101 [26] | 325.21 | Manual | 85.54 | 89.62 | 77.15 | 82.92
DenseNet3D121 [5] | 43.06 | 87.02 | 88.97 | 82.78 | 85.76
MC3_18 [26] | 43.84 | 86.16 | 87.11 | 82.78 | 84.89
COVID-AL [28] | - | 86.60 | - | - | -
VGG16-Ensemble[16] | - | 88.12 | 84.04 | 89.19 | 86.54
CovidNet3D-S [8] | 11.48 | Auto | 88.55 | 88.78 | 91.72 | 90.23
CovidNet3D-L [8] | 53.26 | 88.69 | 90.48 | 88.08 | 89.26
MNas3DNet [9] | 22.91 | 87.14 | 88.44 | 86.09 | 87.25
EMARS-A | 5.93 | 89.67 | 89.26 | 89.22 | 89.23
EMARS-P | 5.63 | 88.78 | 88.81 | 88.22 | 88.51
| EMARS-APS | 3.38 | 89.61 | 91.48 | 89.97 | 90.72
Mos- Med [Russia] [20] | ResNet3D101 [26] | 325.21 | Manual | 81.82 | 81.31 | 97.25 | 88.57
DenseNet3D121 [5] | 43.06 | 79.55 | 84.23 | 92.16 | 88.01
MC3_18 [26] | 43.84 | 80.4 | 79.43 | 98.43 | 87.92
DeCoVNet [27] | - | 82.43 | - | - | -
CovidNet3D-S [8] | 12.48 | Auto | 81.17 | 78.82 | 99.22 | 87.85
CovidNet3D-L [8] | 60.39 | 82.29 | 79.50 | 98.82 | 88.11
EMARS-A | 2.89 | 80.98 | 77.91 | 99.61 | 87.44
EMARS-P | 18.22 | 84.34 | 93.56 | 85.49 | 89.34
EMARS-APS | 10.69 | 88.09 | 93.52 | 90.59 | 92.03
Covid- CTset [Iran] [22] | ResNet3D101 [26] | 325.21 | Manual | 93.87 | 92.34 | 95.54 | 93.92
DenseNet3D121 [5] | 43.06 | 91.91 | 92.57 | 92.57 | 92.57
MC3_18 [26] | 43.84 | 92.57 | 90.95 | 94.55 | 92.72
CovCTx[3] | - | 96.37 | - | 97.00 | -
Vit-32$\times$32 [25] | - | 95.36 | - | 83.00 | -
CovidNet3D-S [8] | 8.36 | Auto | 94.27 | 92.68 | 90.48 | 91.57
CovidNet3D-L [8] | 62.82 | 96.88 | 97.50 | 92.86 | 95.12
AutoGluon model [1] | 93.00 | 89.00 | 90.00 | 88.00 | 88.00
EMARS-A | 8.36 | 95.16 | 95.77 | 95.16 | 95.46
EMARS-P | 14.41 | 92.87 | 92.73 | 92.74 | 92.74
| EMARS-APS | 9.95 | 97.66 | 97.61 | 97.58 | 97.59

##  5 Conclusion and Future Work

In this work, we introduce an EA-based neural architecture search (EMARS)
framework, which can efficiently discover superior 3D models under multiple
objectives for COVID-19 3D CT classification. We demonstrate that our proposed
objective, i.e., potential, can effectively alleviate the search instability
and help exploit promising models. The models searched by EMARS under accuracy
and potential objectives have small sizes and outperform the previous work on
three public datasets. We believe our framework can also be extended to other
types of datasets and tasks (e.g., segmentation), which is also our future
work.

##  6 Acknowledge

This work was supported in part by Hong Kong Research Matching Grant
RMGS2019_1_23, the Zhejiang Province Nature Science Foundation of China under
Grant LZ22F020003, and the HDU-CECDATA Joint Research Center of Big Data
Technologies under Grant KYH063120009.

## References

  * [1] Anwar, T.: Covid19 diagnosis using automl from 3d ct scans. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 503–507 (2021)
  * [2] Chen, M., Fu, J., Ling, H.: One-shot neural ensemble architecture search by diversity-guided search space shrinking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 16530–16539 (2021)
  * [3] Chetoui, M., Akhloufi, M.A.: Efficient deep neural network for an automated detection of covid-19 using ct images. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). pp. 1769–1774. IEEE (2021)
  * [4] Deb, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multiobjective genetic algorithm: Nsga-ii. IEEE transactions on evolutionary computation 6(2), 182–197 (2002)
  * [5] Diba, A., Fayyaz, M., Sharma, V., Karami, A.H., Arzani, M.M., Yousefzadeh, R., Van Gool, L.: Temporal 3d convnets: New architecture and transfer learning for video classification. arXiv preprint arXiv:1711.08200 (2017)
  * [6] Elsken, T., Metzen, J.H., Hutter, F.: Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018)
  * [7] Fu, Z., Tang, N., Chen, Y., Ma, L., Wei, Y., Lu, Y., Ye, K., Liu, H., Tang, F., Huang, G., et al.: Ct features of covid-19 patients with two consecutive negative rt-pcr tests after treatment. Scientific Reports 10(1), 1–6 (2020)
  * [8] He, X., Wang, S., Chu, X., Shi, S., Tang, J., Liu, X., Yan, C., Zhang, J., Ding, G.: Automated model design and benchmarking of deep learning models for covid-19 detection with chest ct scans. Proceedings of the AAAI Conference on Artificial Intelligence pp. 4821–4829 (May 2021)
  * [9] He, X., Wang, S., Shi, S., Chu, X., Tang, J., Liu, X., Yan, C., Zhang, J., Ding, G.: Benchmarking deep learning models and automated model design for covid-19 detection with chest ct scans. medRxiv (2020)
  * [10] He, X., Zhao, K., Chu, X.: Automl: A survey of the state-of-the-art. Knowledge-Based Systems 212, 106622 (2021)
  * [11] Horry, M.J., Chakraborty, S., Pradhan, B., Fallahpoor, M., Chegeni, H., Paul, M.: Factors determining generalization in deep learning models for scoring covid-ct images. Mathematical Biosciences and Engineering 18(6), 9264–9293 (2021)
  * [12] Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Mobilenets, H.A.: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
  * [13] Hu, Y., Liang, Y., Guo, Z., Wan, R., Zhang, X., Wei, Y., Gu, Q., Sun, J.: Angle-based search space shrinking for neural architecture search. In: European Conference on Computer Vision. pp. 119–134. Springer (2020)
  * [14] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR (2015)
  * [15] Li, L., Qin, L., Xu, Z., Yin, Y., Wang, X., Kong, B., Bai, J., Lu, Y., Fang, Z., Song, Q., et al.: Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct. Radiology p. 200905 (2020)
  * [16] Li, X., Tan, W., Liu, P., Zhou, Q., Yang, J.: Classification of covid-19 chest ct images based on ensemble deep learning. Journal of Healthcare Engineering 2021 (2021)
  * [17] Lin, M., Chen, Q., Yan, S.: Network in network. In: Bengio, Y., LeCun, Y. (eds.) 2nd International Conference on Learning Representations, ICLR (2014)
  * [18] Liu, H., Simonyan, K., Yang, Y.: DARTS: differentiable architecture search. In: 7th International Conference on Learning Representations, ICLR (2019)
  * [19] Microsoftware: Neural network intelligence (nni). https://github.com/microsoft/nni/tree/v1.4 (2019)
  * [20] Morozov, S., Andreychenko, A., Pavlov, N., Vladzymyrskyy, A., Ledikhova, N., Gombolevskiy, V., Blokhin, I., Gelezhe, P., Gonchar, A., Chernina, V., Babkin, V.: Mosmeddata: Chest ct scans with covid-19 related findings. medRxiv (2020)
  * [21] Pham, H., Guan, M.Y., Zoph, B., Le, Q.V., Dean, J.: Efficient neural architecture search via parameter sharing. In: Dy, J.G., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018. Proceedings of Machine Learning Research, vol. 80, pp. 4092–4101. PMLR (2018)
  * [22] Rahimzadeh, M., Attar, A., Sakhaei, S.M.: A fully automated deep learning-based network for detecting covid-19 from a new and large lung ct scan dataset. medRxiv (2020)
  * [23] Real, E., Aggarwal, A., Huang, Y., Le, Q.V.: Regularized evolution for image classifier architecture search. In: The Thirty-Third AAAI Conference on Artificial Intelligence. pp. 4780–4789. AAAI Press (2019)
  * [24] Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., Chen, L.: Mobilenetv2: Inverted residuals and linear bottlenecks. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR. pp. 4510–4520. IEEE Computer Society (2018)
  * [25] Than, J.C., Thon, P.L., Rijal, O.M., Kassim, R.M., Yunus, A., Noor, N.M., Then, P.: Preliminary study on patch sizes in vision transformers (vit) for covid-19 and diseased lungs classification. In: 2021 IEEE National Biomedical Engineering Conference (NBEC). pp. 146–150. IEEE (2021)
  * [26] Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., Paluri, M.: A closer look at spatiotemporal convolutions for action recognition. In: 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR. pp. 6450–6459. IEEE Computer Society (2018)
  * [27] Wang, X., Deng, X., Fu, Q., Zhou, Q., Feng, J., Ma, H., Liu, W., Zheng, C.: A weakly-supervised framework for covid-19 classification and lesion localization from chest ct. IEEE transactions on medical imaging 39(8), 2615–2625 (2020)
  * [28] Wu, X., Chen, C., Zhong, M., Wang, J., Shi, J.: Covid-al: The diagnosis of covid-19 with deep active learning. Medical Image Analysis 68, 101913 (2021)
  * [29] Xie, L., Chen, X., Bi, K., Wei, L., Xu, Y., Wang, L., Chen, Z., Xiao, A., Chang, J., Zhang, X., et al.: Weight-sharing neural architecture search: A battle to shrink the optimization gap. ACM Computing Surveys (CSUR) 54(9), 1–37 (2021)
  * [30] Yang, Z., Wang, Y., Chen, X., Shi, B., Xu, C., Xu, C., Tian, Q., Xu, C.: CARS: continuous evolution for efficient neural architecture search. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR. pp. 1826–1835. IEEE (2020)
  * [31] Zhang, K., Liu, X., Shen, J., Li, Z., Sang, Y., Wu, X., Zha, Y., Liang, W., Wang, C., Wang, K., et al.: Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of covid-19 pneumonia using computed tomography. Cell (2020)
  * [32] Zheng, C., Deng, X., Fu, Q., Zhou, Q., Feng, J., Ma, H., Liu, W., Wang, X.: Deep learning-based detection for covid-19 from chest ct using weak label. MedRxiv (2020)
  * [33] Zoph, B., Le, Q.V.: Neural architecture search with reinforcement learning. In: 5th International Conference on Learning Representations, ICLR 2017 (2017)