Nikhef-2016-059

Statistical biases in measurements with multiple candidates

P. Koppenburg1.

1Nikhef, Amsterdam, The Netherlands.

The presence of multiple candidates per event can cause raw biases which are
large compared to statistical uncertainties. Selecting a single candidate is
common practice but only helps if the likelihood of selecting the true
candidate is very high. Otherwise, the precision of the measurement can be
affected, and additional biases can be generated, even if none are present in
the data sample prior to this operation.

This paper is an attempt at describing the problem in a systematic way. It
sets definitions, provides examples of potential biases using
pseudoexperiments and gives recommendations.

© P. Koppenburg, licence CC-BY-4.0.

##  1 Introduction

In absence of clear signals of new particles produced in high-energy
colliders, precision measurements provide the best way of finding signs of New
Physics. These measurements require large data samples as well as a very
detailed understanding of potential biases. Large data samples and higher
luminosities are anticipated, notably at the Large Hadron Collider (LHC) or
the KEKB $e^{+}e^{-}$ collider. Many decay rate or asymmetry measurements with
a precision of order $10^{-4}$ are or will soon be available in flavour [1, 2,
3, 4] physics, while electroweak and Higgs physics also enter the precision
regime. The presence and the handling of multiple candidates per event can
cause raw biases which may be large compared to anticipated accuracies. These
biases can be corrected for, provided a proper strategy is employed, with some
loss in precision.

In the following, event stands for all signals originating from a collision
process, or a set of simultaneous processes, as in a single beam-bunch
crossing. A candidate is the reconstructed particle of interest, including its
decay chain, on which the measurement is being performed — typically a Higgs
or $Z$ boson, a top quark, or a $B$ meson. The latter example is used below.

The processes most commonly studied in modern high-energy physics have a small
probability, either because the production rate is small (as for the Higgs
boson [5, 6]) or because the decay rate is low (as for the decay
${{B}^{0}_{s}}\\!\rightarrow{\mu^{+}}{\mu^{-}}$ [7, 8]), or both. (Studies of
processes that are not rare, and where several candidates per event are to be
expected, are not in the scope of this paper.) The same applies to background,
as analysts invest considerable effort to achieve a background rate of the
same order as that of the signal, or less. Yet many high energy physics
analyses are affected by a non-negligible fraction of events containing
several candidates, typically in the range $0.1$ to $20\%$. The probabilities
of selecting one or a second candidate in an event are not uncorrelated. If
they were — given the low probability involved — the rate of events containing
two or more would be vanishingly small. The presence of multiple candidates in
an event is therefore an indication that the event or the candidates are
special. An investigation is required prior to any action being taken.

Multiple candidate events are a nuisance for the analysis. They contribute to
the background level and thus degrade the sensitivity of the measurement. But
most importantly, additional candidates can cause biases if their rate is
correlated with the observable to be measured. These biases may be corrected
using simulation or control samples, provided these are good representations
of the data. Otherwise, additional corrections are required. Such situations
are discussed in Sec. 3.

In some experimental particle physics publications the rate of multiple
candidates and the procedure to handle them is described, but less frequently
are their origin discussed and potential biases addressed. It is usually not
possible for an external reader to assess their nature and their effect on the
measurement, which poses the question of the reproducibility of the analysis.
No standard procedure to address multiple candidates is publicly documented
[9], which forces every collaboration or analysis group to address the problem
(or to decide to ignore it), and often to re-invent the wheel.

The present paper is an attempt at describing the problem in a systematic way.
It organised as follows: In Sec. 2 gives definitions of sources of multiple
candidates and techniques to address them. A set of pseudoexperiments
assessing the size of potential biases is described in Sec. 3. Recommendations
are given in Sec. 4.

##  2 Definitions

Multiple candidates can be of several kinds.

Overlaps

share some of their reconstructed objects (tracks, clusters, jets…). At most
one of the overlapping candidates can be signal.

Reflections

are candidates which share all reconstructed objects but with different
particle identification assignments. Depending on the analysis they can be
peaking in mass (when applicable) or not. Their treatment is analysis-
dependent and thus not discussed further in this paper.

Genuine multiples

do not share any of their properties. They could in principle be both signal,
depending on the signal rate and selection efficiency [10], but are more
likely to be both background, caused by a high multiplicity event, or one of
each.

Reconstruction features

may make physics objects appear multiple times. They are experiment-specific
and not discussed further. However, if applicable, they need to be considered
when dealing with multiple candidates.

###  2.1 Techniques

Several techniques of handling multiple candidates are reported in the
literature.

No action:

The simplest and most frequent approach is to take no action, i.e. keep all
candidates for further analysis. The reader has to assume that this was the
chosen approach when multiple candidates are not mentioned in a publication
[12]. In some occasions this choice is explicitly mentioned [13, 14, 15, 16,
17, 1]. It is the most common approach in production cross-section
measurements [18, 19, 20, 21, 22, 23, 24, 25, 26, 27]. In rare occasions, the
data sample is split by candidate multiplicity [28]. A different background
parametrisation depending on the presence of multiple candidates is then used.

All other techniques aim at reducing the level of multiple candidates.

Arbitration

consists in attempting to identify the signal candidate by using a
discriminating variable, thus applying a tighter selection in events with
multiple candidates compared to the remaining events. Typical choices of
variables are the (transverse [29, 30]) momentum [31] of the candidate or a
decay product, the $\chi^{2}$ of a vertex [32, 33, 34] or a decay tree fit
[35, 36], an impact parameter [37, 38], a flight length [39], a mass of a
decay product [40, 41, 5, 42, 43, 44, 45], the expected rate of the final
state [45], or, the beam-constrained energy at $e^{+}e^{-}$ colliders [46, 47,
48, 49, 50, 51, 52]. Arbitration may also be part of the event reconstruction
[53] and the selection of the collision point [54].

Random picking

is a special case of arbitration where the chosen variable is a random number
and thus not discriminating [55, 56, 57, 58, 59]. A variant of random picking
is weighting, in which candidates are weighted by the inverse of their
multiplicity [60].

Event removal,

where all events with multiple candidates are discarded, is also occasionally
used [61, 62, 63].

The above-mentioned techniques have an associated signal efficiency, which
contributes to the total selection efficiency. The latter is often determined
using simulated data or a control channel. This is discussed further in Sec
3.5.

##  3 Pseudoexperiment

The effect of the presence of multiple candidates is tested with a set of
pseudoexperiments. The model employed here is typical of a $B$ physics
experiment, but is kept intentionally simple, so that it can be translated to
any scenario. The only needed features are that the signal is separated from
the background by a fit to a discriminating variable (here a mass). The
properties of the signal are then investigated.

The problem is formulated as follows. A process resulting in a Gaussian-shaped
signal peak (a ${B}^{0}$ meson mass in this example) is studied. The rate
(which could result in the measurement of a cross-section or a branching
fraction), an asymmetry, and a property (here the lifetime) are to be
measured. Ratios and asymmetries are of particular interest as many of the
detector- and selection-related biases cancel at first order. They allow for
high-precision measurements which can be compared with precise predictions.
Here a $C\\!P$ or production asymmetry between the ${B}^{0}$ and ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ yields is taken as an example,
but the problem can be generalised to any fraction, as for instance a
polarisation.

In the present example ${N_{{B}^{0}}^{\text{gen}}}=50\>000$ signal ${B}^{0}$
mesons are generated (including Poisson fluctuations) on top of as many
combinatorial background candidates. The ${B}^{0}$ signal is represented by a
Gaussian shape and the background has a uniform mass distribution [64] in a
range of 25 times the resolution and is centred on the signal. The same is
done for ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ mesons. In the
following the resulting mass distributions are referred to as the clean case.

Next, additional candidates (called companions) are generated as follows.

  1. 1.

Each signal ${B}^{0}$ meson has a probability ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$ to have a companion candidate in the same event.
The mass distribution of these candidates is background-like [65]. This
candidate can have a swapped flavour[66], with probability ${{\cal
P}^{\text{sig }{{B}^{0}}}_{\text{swap}}}\\!$, or have the same flavour, with
probability $(1-{{\cal P}^{\text{sig }{{B}^{0}}}_{\text{swap}}}\\!)$.

  2. 2.

The same is done for ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$
mesons, introducing the probabilities ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$ and ${{\cal
P}^{\text{sig }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$, which do not
need to be identical to those for ${B}^{0}$ mesons.

  3. 3.

Each background ${B}^{0}$ candidate has a probability ${{\cal
P}^{{{B}^{0}}}_{\text{bkg}}}$ to have a companion candidate with probability
of swapped flavour ${{\cal P}^{\text{bkg }{{B}^{0}}}_{\text{swap}}}$,
generated with uniform mass.

  4. 4.

The same is done for ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$
background candidates with probabilities ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{bkg}}}$ and ${{\cal
P}^{\text{bkg }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$, which do not
need to be identical to those for ${B}^{0}$ background candidates.

All symbols defined above are listed in Table 1.

Table 1: Symbols used in this document. Symbol | Definition
---|---
${N_{{B}^{0}}^{\text{gen}}}$ | Generated number of signal ${B}^{0}$ mesons ($=50\>000$)
${N_{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{gen}}}$ | Generated number of signal ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ mesons ($=50\>000$)
${N^{\text{bkg}}_{{B}^{0}}}$ | Total number of background ${B}^{0}$ candidates
${N^{\text{bkg}}_{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}$ | Total number of background ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ candidates
${\tau_{{B}^{0}}}$ |  ${B}^{0}$ meson lifetime ($=1.52{\mathrm{\,ps}}$)
$\alpha$ | Inverse effective signal-to-background ratio
${N_{{B}^{0}}^{\text{fit}}}$ | Measured ${B}^{0}$ signal yield
${N_{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{fit}}}$ | Measured ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ signal yield
${A_{\text{raw}}}$ | Asymmetry between measured ${B}^{0}$ and ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ signal yields
${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$ | Probability of a ${B}^{0}$ meson to have a companion in the event
${{\cal P}^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$ | Probability of a ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ meson to have a companion in the event
${{\cal P}^{{{B}^{0}}}_{\text{bkg}}}$ | Probability of a background ${B}^{0}$ candidate to have a companion in the event
${{\cal P}^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{bkg}}}$ | Probability of a background ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ candidate to have a companion in the event
${R_{\text{all}}}$ | Fraction of events with multiple candidates
${R_{{B}^{0}}}$ | Fraction of signal events with multiple candidates
${{\cal P}^{\text{sig }{{B}^{0}}}_{\text{swap}}}$ | Probability of a signal ${B}^{0}$ companion to have a ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ flavour
${{\cal P}^{\text{sig }{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$ | Probability of a signal ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ companion to have a ${B}^{0}$ flavour
${{\cal P}^{\text{bkg }{{B}^{0}}}_{\text{swap}}}$ | Probability of a background ${B}^{0}$ companion to have a ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ flavour
${{\cal P}^{\text{bkg }{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$ | Probability of a background ${B}^{0}$ companion to have a ${B}^{0}$ flavour
${\eta^{{{B}^{0}}}_{\text{best}}}$ | Efficiency of picking the signal ${B}^{0}$ when the companion has the same flavour
${\eta^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}$ | Efficiency of picking the signal ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ when the companion has the same flavour
${\eta^{{{B}^{0}}/{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}$ | Efficiency of picking the signal ${B}^{0}$ when the companion has a different flavour
${\eta^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}/{{B}^{0}}}_{\text{best}}}$ | Efficiency of picking the signal ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ when the companion has a different flavour

The fraction of events with multiple candidates is given by

${R_{\text{all}}}=\frac{{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}{N_{{B}^{0}}^{\text{gen}}}+{{\cal
P}^{{{B}^{0}}}_{\text{bkg}}}{N_{\text{bkg
{${B}^{0}$}}}^{\text{gen}}}}{{N_{{B}^{0}}^{\text{gen}}}+{N_{\text{bkg
{${B}^{0}$}}}^{\text{gen}}}},$ (1)

assuming the flavour swapping probabilities to be zero or irrelevant. The
analogous quantity can be defined mutatis mutandis for ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ mesons but is not used in the
following. This emulation is a simplified model of reality as there are no
events with three candidates, which could require an iterative procedure. As
all candidates have passed all selection requirements, they are all equally
signal-like except for their mass.

After the addition of companions, the ${B}^{0}$ and ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ background levels are increased
by amounts depending on the values of the above-defined probabilities.

Figure 1: Typical mass distributions after candidate addition for (left)
${B}^{0}$ and (right) ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$
candidates with ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}=0.4$, ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}=0.1$. All other
probabilities of Table 1 are set to zero. The number of candidates and the
fitted signal yield are indicated.

Typical mass distributions are shown in Figure 1 for ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}=0.4$ and ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}=0.1$. Here and
in the following, large probabilities are shown to make the effects easily
visible in graphics. In a real experiment, the probabilities would be smaller,
but the total yields may be much larger, making the described effects
significant.

No further action results in taking all candidates, and is referred to as the
all scenario. Alternatively, the analyst can choose to pick a random candidate
or perform an arbitration. Weighting and event removal are not tested. Their
effect can be inferred from those of the random selection [67]. After this
operation the ${B}^{0}$ and ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ yields are determined
separately from unbinned maximum-likelihood fits to the candidates mass
distributions. The probability distribution function used in the fit is the
sum of a Gaussian and a uniform background, the same as used in the generation
of the sample. All parameters are left free in the fit.

The uncertainty on the signal yield ${N_{{B}^{0}}^{\text{fit}}}$ depends on
the background yield ${N^{\text{bkg}}_{{B}^{0}}}$ as

$\sigma({N_{{B}^{0}}^{\text{fit}}})=\sqrt{{N_{{B}^{0}}^{\text{fit}}}+\alpha{N^{\text{bkg}}_{{B}^{0}}}},$
(2)

where to a good level of approximation $\alpha$ only depends on the mass
window and the signal peak resolution. In this case
$\sigma({N_{{B}^{0}}^{\text{fit}}})=248$ and $\alpha=0.23$. This example shows
that while the probabilities of companion candidates are different, resulting
in a higher background level for ${B}^{0}$ candidates, the fits return signal
yields consistent with the generated numbers. As expected, the uncertainty is
larger in the case of a higher multiple candidate rate.

Multiple candidate handling techniques are studied as function of the
probabilities defined above with thousand pseudoexperiments per set of
probabilities. The results shown below are averages on these
pseudoexperiments. Many of the subsequent results can be determined
analytically using the probabilities listed in Table 1 and behave almost
linearly with the input probabilities [68].

###  3.1 Varying ${B}^{0}$ companion probabilities

Figure 2: Measured ${B}^{0}$ (top left) yield, (top right) uncertainty and
(bottom) their ratio versus ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$. All other
probabilities of Table 1 are set to zero.

First the ${B}^{0}$ yield is measured as function of the signal probabilities
${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$ and ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$ between $0$ and
$0.5$ with ${{\cal P}^{{{B}^{0}}}_{\text{bkg}}}={{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{bkg}}}=0$. The
resulting measured ${B}^{0}$ yields and uncertainties are reported in Figure
2. The ideal clean case, in which always the correct candidate is picked, is
the benchmark scenario with which all other approaches are compared. When
using all candidates the correct yield is obtained, although with an increased
uncertainty. With the technique consisting of randomly picking candidates, the
signal candidate is discarded in half of the events with multiple candidates.
The fit returns a biased ${B}^{0}$ yield, trivially depending on ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$ as

$\left({N_{{B}^{0}}^{\text{fit}}}\right)^{\text{random}}={N_{{B}^{0}}^{\text{gen}}}\left(1-\frac{1}{2}{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}\right).$ (3)

The biases can be corrected (see Section 3.5), but not the loss of statistical
precision. The relative uncertainty on the ${B}^{0}$ yield [68] — which is
often the quantity for which the selection is optimised — is shown in Fig. 2
(bottom). The random selection technique performs worse than taking all with
respect to this figure of merit.

Next, an asymmetry between the ${B}^{0}$ and ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ yields is determined, defined
by

${A_{\text{raw}}}=\frac{{N_{{B}^{0}}^{\text{fit}}}-{N_{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{fit}}}}{{N_{{B}^{0}}^{\text{fit}}}+{N_{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{fit}}}}.$ (4)

Figure 3: (left) Measured ${A_{\text{raw}}}$ (in percent) versus ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$ and ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$ for random
picking. (right) Uncertainty on this quantity for the three techniques versus
${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$, with ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}=0$. All other
probabilities of Table 1 are set to zero.

In the following pseudoexperiments no asymmetry is generated and any
measurement of this quantity should be consistent with zero. Figure 3 show
this asymmetry versus the ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$ and ${{\cal
P}^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$
probabilities in the range $0$ to $0.5$. Asymmetries of up to $14\%$ can be
generated when performing random picking,

$\left(A_{\text{raw}}\right)^{\text{random}}=\frac{{N_{{B}^{0}}^{\text{gen}}}-{N_{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{gen}}}+\frac{1}{2}\left({N_{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{gen}}}{{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}-\frac{1}{2}{N_{{B}^{0}}^{\text{gen}}}{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}\right)}{{N_{{B}^{0}}^{\text{gen}}}+{N_{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{gen}}}-{\frac{1}{2}\left({N_{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}^{\text{gen}}}{{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}-\frac{1}{2}{N_{{B}^{0}}^{\text{gen}}}{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}\right)}}\stackrel{{\scriptstyle
A_{C\\!P}=0}}{{=}}\frac{\frac{1}{2}{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}-\frac{1}{2}{{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}}{\frac{1}{2}{{\cal
P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}+\frac{1}{2}{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}-2},$ (5)

where the expression considerably simplifies when no true $C\\!P$ asymmetry is
present. On the other hand, using all candidates leaves the asymmetry
unbiased.

The uncertainties on this quantity vary from $0.362\%$, at ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}={{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}=0$, to values of
$0.366\%$ for random picking and $0.382\%$ when keeping all candidates. These
changes in statistical uncertainties are very small ($+0.02\%$) in comparison
with the potential bias introduced by the random picking technique (up to
$14\%$). Similarly, varying the companion probabilities in background, ${{\cal
P}^{{{B}^{0}}}_{\text{bkg}}}$ and ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{bkg}}}$, only slightly
affects the statistical uncertainties [69].

###  3.2 Arbitration

The drawbacks of random picking may be cured by arbitration. The selection of
a “best” candidate is done using an observable $O$ which is not correlated to
that which is used to determine the signal yield (here the candidate mass).
For the technique to be effective, the true signal and the other candidates
must have different distributions of this quantity. This feature is exploited
to pick the candidate that is more signal-like. The arbitration procedure has
an efficiency ${\eta^{{{B}^{0}}}_{\text{best}}}$ of picking the true signal,
which should be as high as possible. For the present discussion, only this
efficiency matters, and not the actual $O$ distributions.

Figure 4: (left) ${B}^{0}$ yield and (right) relative uncertainty versus
efficiency of picking the signal candidate ${\eta^{{{B}^{0}}}_{\text{best}}}$.
The value at ${\eta^{{{B}^{0}}}_{\text{best}}}=0.5$ corresponds to random
picking. The other parameters are set to ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}={{\cal P}^{{{B}^{0}}}_{\text{bkg}}}=0.2$, ${{\cal
P}^{\text{sig }{{B}^{0}}}_{\text{swap}}}\\!={{\cal P}^{\text{bkg
}{{B}^{0}}}_{\text{swap}}}\\!=0$.

The aim is illustrated in Figure 4 (left). As the efficiency of picking the
right candidate is improved, the measured ${B}^{0}$ yield increases linearly
from the value obtained with random picking to that of a perfect selection.
The biases caused by random picking (Eq. 3) are mitigated by arbitration as
${\eta^{{{B}^{0}}}_{\text{best}}}$ approaches unity,

$\left({N_{{B}^{0}}^{\text{fit}}}\right)^{\text{arbitration}}={N_{{B}^{0}}^{\text{gen}}}\left(1-{\eta^{{{B}^{0}}}_{\text{best}}}{{\cal
P}^{{{B}^{0}}}_{\text{sig}}}\right).$ (6)

Similarly, the effect of the arbitration procedure on the biases of
${A_{\text{raw}}}$ can be obtained by multiplying the values of Figure 3
(left) by $2(1-{\eta^{{{B}^{0}}}_{\text{best}}})$.

With increasing values of ${\eta^{{{B}^{0}}}_{\text{best}}}$, the relative
uncertainty on the ${B}^{0}$ yield decreases and becomes better than that
obtained using all candidates. The value of the crossing point in Fig. 4
(right) depends on ${\eta^{{{B}^{0}}}_{\text{best}}}$, ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$ and ${{\cal P}^{{{B}^{0}}}_{\text{bkg}}}$ [68].
It is to be noted again that the relative uncertainty varies very little. As
for random picking, the gain in statistical sensitivity is small compared to
to the potential biases due to different values of ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$ and ${{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$.

A best candidate selection only helps if the efficiency for selecting the
correct candidate is very large. Otherwise, additional asymmetries can be
generated, even if none are present in the data prior to this operation. The
best candidate selection efficiency can be different for ${B}^{0}$ and ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ mesons, ${\eta^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}\neq{\eta^{{{B}^{0}}}_{\text{best}}}$,
if the distributions of $O$ are different for ${B}^{0}$ and ${\kern
1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ signal, background, or both.
Such an asymmetry of selection efficiencies can induce biases of the ${B}^{0}$
and ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ yields, generating
raw asymmetries, as shown in Figure 5. In the (unlikely) extreme cases, raw
asymmetries of $5\%$ can be reached.

In a real measurement, analysts would not pick an observable which can
generate large asymmetries. However, even if checked, this fact is hardly ever
reported in publications. Also, the symmetry of the observable is only known
to a given precision, which in principle should be determined and assigned as
a systematic uncertainty [70].

Figure 5: ${A_{\text{raw}}}$ (in percent) versus (left) efficiencies of
picking the signal candidate ${\eta^{{{B}^{0}}}_{\text{best}}}$ and
${\eta^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}$, and (right)
versus flavour swapping probability ${{\cal P}^{\text{sig
}{{B}^{0}}}_{\text{swap}}}$. Other parameters are set to ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}={{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}={{\cal
P}^{{{B}^{0}}}_{\text{bkg}}}={{\cal P}^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{bkg}}}=0.2$.
Additionally in the right plot,
${\eta^{{{B}^{0}}}_{\text{best}}}={\eta^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}/{{B}^{0}}}_{\text{best}}}=0.6$,
${\eta^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}={\eta^{{{B}^{0}}/{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}=0.8$ and
${{\cal P}^{\text{sig }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}\\!={{\cal
P}^{\text{bkg }{{B}^{0}}}_{\text{swap}}}\\!=0$. ${{\cal P}^{\text{sig
}{{B}^{0}}}_{\text{swap}}}\\!={{\cal P}^{\text{bkg
}{{B}^{0}}}_{\text{swap}}}\\!$ is imposed. The vertical error bars represent
the average uncertainty on the fitted asymmetry.

###  3.3 Flavour swaps

In the above, it is assumed that all $B$ candidates in any given event have
the same flavour, or that it does not matter. This is not necessarily the
case. For overlapping candidates, the flavour of the signal $B$ meson and that
of the other candidate may be correlated, or anti-correlated [71]. The
consequence are non-identical probabilities of being accompanied by a
candidate of different flavour ${{\cal P}^{\text{sig
}{{B}^{0}}}_{\text{swap}}}$, ${{\cal P}^{\text{sig }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$ for ${B}^{0}$
and ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ signal, and ${{\cal
P}^{\text{bkg }{{B}^{0}}}_{\text{swap}}}$, ${{\cal P}^{\text{bkg }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$ for ${B}^{0}$
and ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ background.

In Figure 5 (right) the swapping probabilities ${{\cal P}^{\text{sig
}{{B}^{0}}}_{\text{swap}}}$ and ${{\cal P}^{\text{sig }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$ are varied
(synchronously with ${{\cal P}^{\text{bkg }{{B}^{0}}}_{\text{swap}}}$ and
${{\cal P}^{\text{bkg }{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}$, which have
no effect), for ${\eta^{{{B}^{0}}}_{\text{best}}}=0.6$ and ${\eta^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}=0.8$. Values of
$-2.1\%$ for ${A_{\text{raw}}}$ are obtained when the swapping probabilities
are identical. This corresponds to the point at
${\eta^{{{B}^{0}}}_{\text{best}}}=0.6,{\eta^{{{\kern
1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{best}}}=0.8$ in Fig. 5
(left). Different swapping probabilities can mitigate the problem up to hiding
it completely, as for ${{\cal P}^{\text{sig }{{B}^{0}}}_{\text{swap}}}\\!=1$.
Such a situation is not necessarily desirable, as the situation may not be
reproduced well in the simulation or in a control sample, leading to incorrect
efficiency corrections.

###  3.4 Lifetime and mass measurements

Next, the data described above is used to measure the lifetime of ${B}^{0}$
mesons. It is assumed that the decay time distributions of the signal and
background follow a falling exponential of constants $-{\tau_{{B}^{0}}}$ and
$-{\tau_{{B}^{0}}}/3$, respectively. For companion candidates overlapping with
the signal this constant is $-{\tau_{{B}^{0}}}/2$. It is common for $B$
background candidates to be formed from a combination of final-state particles
from different $b$ hadrons, resulting in a visible lifetime that is lower than
that of a true $b$ hadron. In the case of overlaps, some components of the
candidate are from the true signal, resulting in a visible lifetime which is
larger than that of background but lower than ${\tau_{{B}^{0}}}$ [72].

The three exponential distributions are convolved with a Gaussian-shaped
resolution function [73]. No time-depended selection effects are considered.
This situation leads to a distribution as shown in the left panel of Fig. 6,
where the three contributions are shown.

Figure 6: Typical decay time distribution after companion addition for (left)
all candidates and (right) background-subtracted signal. As in Fig. 1 (left),
the companion probability is ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}=0.4$. All
other probabilities of Table 1 are set to zero.

If the mass and decay time are not correlated for the three above-defined
species, the sPlot technique [74, 75] can be used to statistically subtract
the background and companion candidates using their distinct mass
distributions. The decay time distribution for signal is thus obtained with an
unbinned maximum-likelihood fit of a single exponential to the background-
subtracted data, as shown in Fig. 6 (right). As companion candidates do not
peak in mass they do not affect the signal decay time distribution and no
biases can be caused [76]. The additional background causes an increased
relative uncertainty on the fitted lifetime value, as shown in Fig. 7.

Figure 7: Relative uncertainty on lifetime versus (left) ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$ and (right) ${\eta^{{{B}^{0}}}_{\text{best}}}$.
All other probabilities are set to zero in the former case, while ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}={{\cal P}^{{{B}^{0}}}_{\text{bkg}}}=0.2$ in the
latter case.

Taking all candidates does not bias the measurement, and has a negligible
effect on the uncertainty. Random picking and arbitration degrade the
resolution due to the loss of signal.

The lifetime can also be determined by fitting all data with a signal and one
or several background components [77]. If the specific features of overlapping
companions are ignored, only one component for the background is used.

Figure 8: (left) Two-component fit to the lifetime data which was generated
with three exponential functions. The companion probability is ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}=0.4$. All other probabilities of Table 1 are set
to zero. (right) Measured lifetime versus ${{\cal
P}^{{{B}^{0}}}_{\text{sig}}}$. The vertical error bars represent the average
uncertainty on the fitted lifetime.

Such a fit is shown in Fig. 8 (left). The measured lifetime is biased, as seen
in Fig. 8 (right). For large data samples, the missing component would lead to
poor fits, as in this example. However, it could easily be missed in case of
smaller samples. Sums of similar exponential distributions can be well fitted
by a single exponential [78], resulting in biased lifetime results.

This issue is cured by the use of the sPlot technique, as shown above. But
this technique does not apply to quantities which are correlated with the
discriminating variable. For instance the measurement of the mass of a
resonance requires a good understanding of the background shape. If
overlapping companions tend to accumulate at masses close to but somewhat
lower than the signal, the mass fit could return a biased value. This risk has
been recognised by the CMS collaboration in the analysis of Higgs bosons
decaying to four leptons [43] used in the Higgs boson mass measurement [79].
It is reported that the used arbitration procedure based on dilepton masses
does not sculpt the background shape. The arbitration procedure applied by the
ATLAS collaboration is different and there is no statement on potential biases
[45].

###  3.5 Efficiency corrections

Biases can be caused by the presence of companion candidates, or by the
technique used to remove them. These biases can in principle be corrected for,
but this correction requires a good knowledge of their sizes. The efficiency
of the arbitration technique is
$1-(1-{\eta^{{{B}^{0}}}_{\text{best}}}){R_{\text{all}}}$, where the fraction
of events with multiple candidates ${R_{\text{all}}}$ is given in Eq. 1. For a
random selection ${\eta^{{{B}^{0}}}_{\text{best}}}=\nicefrac{{1}}{{2}}$. The
fraction ${R_{\text{all}}}$ is often reported in publications, giving the
reader an estimate of the scale of the problem. For a quantitative estimate of
the effect of the multiple candidates handling procedure,
${\eta^{{{B}^{0}}}_{\text{best}}}$ is needed, as well as the fraction of
signal events with multiple candidates, ${R_{{B}^{0}}}$ (defined in analogy
with ${R_{\text{all}}}$, setting ${N_{{B}^{0}}^{\text{gen}}}=0$ in Eq. 1).
Only rarely are both numbers given.

Simulated signal events can be used to determine these numbers, provided that
the simulation properly describes the companion candidate rate and their
distributions. At $B$ or charm factories, where the underlying event is the
other heavy meson [51, 52], the simulation is often reported to correctly
model multiple candidates. The description of the underlying event at hadron
colliders is less reliable [80, 81]. The analyst may have to deal with a
different fraction and composition of companion candidates in the simulation
and the data. Different candidate removal efficiencies in data and simulation
lead to a potentially large systematic uncertainty, which is rarely reported
in publications. This uncertainty depends on the properties of the underlying
event, while prior to candidate removal only the signal efficiency is
relevant.

The efficiency can also be assessed using data, for instance applying the
sPlot technique used in Sec. 3.4, or similar background-subtraction
techniques. In the present set of pseudoexperiments, the efficiency of random
picking is determined without bias [82] from the candidate multiplicity in
events with signal. The uncertainty on this efficiency is in the per-mille
range, as shown in Fig. 9 (left), which is not negligible compared to the
relative statistical uncertainty on the signal yield (shown in Fig. 2,
bottom).

Figure 9: (left) Uncertainty on the efficiency of randomly picking the true
signal $B$ versus ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$. (right) Ratio of this
uncertainty to the relative uncertainty on the signal yield (Fig. 2, bottom).
All other probabilities of Table 1 are set to zero.

These two uncertainties are compared in Fig. 9 (right). The systematic
uncertainty related to the random picking efficiency reaches up to 20% of the
statistical uncertainty in these pseudoexperiments. It was already shown in
Fig. 2 that the statistical uncertainty of the randomly picked sample is
larger than that of the sample with all candidates. The systematic uncertainty
is an additional penalty.

Absolute efficiencies are difficult to determine at hadron colliders and thus
potentially involve large systematic uncertainties. It is therefore common
practice to measure relative branching fractions or cross-sections by
normalising the signal to a well-known high-yield normalisation mode [7, 55,
57, 83, 84]. In this case, the ratio of candidate removal efficiencies for the
signal and normalisation modes matters. These efficiencies do not only depend
on the signal and calibration modes, but also on their respective backgrounds.
If the rate of companion candidates, as well as their properties, are not
identical for both modes (which is likely), a systematic uncertainty must be
determined. In order to profit from the statistical power of the larger
calibration sample, analysts should demonstrate that the behaviour of
companion candidates is the same for the signal and the calibration mode.
Otherwise, the systematic uncertainty described above applies.

##  4 Recommendations

In the following recommendations are given.

  1. 1.

Analysts should know how many multiple candidates appear in their selection
and to which category they belong (see in Section 2)

  2. 2.

In all cases the handling of multiple candidates should be performed after the
full selection has been applied.

  3. 3.

Analysts should study the various techniques and select that which minimises
the biases and the total uncertainty.

  4. 4.

Not doing anything usually avoids biases at the price of a slightly higher
background level. The effect on the statistical uncertainty is usually
negligible. In presence of many events with overlapping candidates, the
uncertainty coverage should be checked using pseudoexperiments.

  5. 5.

If overlaps with signal are present, it should be checked whether they behave
signal- or background-like. If neither, a dedicated fit component is required
to avoid biases.

  6. 6.

Any candidate removal technique can cause biases which need to be studied and
may require a correction. Differences between the signal and simulation or a
control samples should be assessed and a systematic uncertainty assigned.

  7. 7.

In case of arbitration using a best candidate selection, additional biases can
be due to the choice of the discriminating variable. This technique is
discouraged. A systematic uncertainty should be assigned.

  8. 8.

If arbitration is nevertheless chosen, analysts should demonstrate that it
improves the precision of the measurement.

  9. 9.

The most important prescription is that analysts should have a strategy and
describe it in the publication. If there is a possibility for any bias a
systematic uncertainty should be assessed.

##  5 Conclusion

This paper shows that the presence of multiple candidates can cause biases in
measurements of rates, asymmetries or properties which are much larger than
the statistical precision. These biases may be difficult to estimate using
simulation if the underlying event is not properly described. Similarly,
corrections using data not only require control samples to be a good
representation of the data, but also of the background affecting the control
sample. Selecting a single candidate per event may generate additional biases
if the rate of multiple candidates is correlated with the observable used to
perform the arbitration. It is recommended to refrain from using the latter
technique. A systematic uncertainty covering the potential biases caused by
multiple candidates should be determined and, if found not be negligible,
reported in the publication.

## Acknowledgements

The author thanks I. Belyaev, U. Egede, V. Gligorov, N. de Groot, T. du Pree,
T. Gershon, D. Martínez Santos, G. Raven, T. Ruf, P. Seyfert, S. Stahl, S.
Stone, N. Tuning and I. van Vulpen for useful discussions. In particular M.
Merk, L. Dufour, and W. Hulsbergen are thanked for carefully reading the
manuscript.

This work is part of the NWO Institute Organisation (NWO-I), which is financed
by the Netherlands Organisation for Scientific Research (NWO).

## References

  * [1] LHCb collaboration, R. Aaij et al., _Measurement of the difference of time-integrated ${C\\!P}$ asymmetries in ${{D}^{0}}\rightarrow{{K}^{-}}{{K}^{+}}$ and ${{D}^{0}}\rightarrow{{\pi}^{-}}{{\pi}^{+}}$ decays_, Phys. Rev. Lett. 116 (2016) 191601, arXiv:1602.03160
  * [2] LHCb collaboration, R. Aaij et al., _Measurement of $C\\!P$ asymmetry in ${{D}^{0}}\rightarrow{{K}^{+}}{{K}^{-}}$ decays_, Phys. Lett. B767 (2017) 177, arXiv:1610.09467
  * [3] LHCb collaboration, _Framework TDR for the LHCb Upgrade: Technical Design Report_ , CERN-LHCC-2012-007.  LHCb-TDR-012
  * [4] T. Aushev et al., _Physics at Super B Factory_ , arXiv:1002.5012
  * [5] ATLAS collaboration, G. Aad et al., _Observation of a new particle in the search for the Standard Model Higgs boson with the ATLAS detector at the LHC_ , Phys. Lett. B716 (2012) 1, arXiv:1207.7214
  * [6] CMS collaboration, S. Chatrchyan et al., _Observation of a new boson at a mass of 125 GeV with the CMS experiment at the LHC_ , Phys. Lett. B716 (2012) 30, arXiv:1207.7235
  * [7] LHCb collaboration, R. Aaij et al., _Measurement of the ${{B}^{0}_{s}}\rightarrow{\mu^{+}}{\mu^{-}}$ branching fraction and effective lifetime and search for ${{B}^{0}}\rightarrow{\mu^{+}}{\mu^{-}}$ decays_, Phys. Rev. Lett. 118 (2017) 191801, arXiv:1703.05747
  * [8] ATLAS collaboration, M. Aaboud et al., _Study of the rare decays of $B^{0}_{s}$ and $B^{0}$ into muon pairs from data collected during the LHC Run 1 with the ATLAS detector_, Eur. Phys. J. C76 (2016) 513, arXiv:1604.04263
  * [9] Internal recommendations exist in LHCb and probably other collaborations. But these are not available for external readers. The present document is based on the LHCb recommendations.
  * [10] As an example, the likelihood of multiple signal candidate events can be determined in the case of ${{D}^{*+}}\\!\rightarrow{{D}^{0}}{{\pi}^{+}}$ with ${{D}^{0}}\\!\rightarrow{{K}^{+}}{{K}^{-}}$ from Ref. [1]. The probability of seeing another signal ${D}^{*+}$ in the same event (which thus contains a charm quark pair) is given by the ${D}^{*+}$ hadronisation fraction, multiplied by the branching fractions and the experimental efficiency. Using the $c$$\overline{c}$ cross-section and hadronisation fractions from Ref. [11], one can infer this probability to be less than $10^{-6}$, while the multiple candidate rate is reported as $1.2$ to $2.4\%$ depending on the considered samples.
  * [11] LHCb collaboration, R. Aaij et al., _Prompt charm production in ${p}{p}$ collisions at $\sqrt{s}=7$$~{}T\kern-0.50003pteV$_, Nucl. Phys. B871 (2013) 1, arXiv:1302.2864
  * [12] This assumption may occasionally be wrong, as shown in conference proceedings which sometimes give more information than the corresponding publication.
  * [13] ATLAS collaboration, G. Aad et al., _Search for a standard model Higgs boson in the mass range 200-600 GeV in the $H\rightarrow ZZ\rightarrow\ell^{+}\ell^{-}q\bar{q}$ decay channel with the ATLAS detector_, Phys. Lett. B717 (2012) 70, arXiv:1206.2443
  * [14] BaBar collaboration, Lees, J. P. et al., _Search for a light Higgs resonance in radiative decays of the $\mathchar 28935\relax(1S)$ with a charm tag_, Phys. Rev. D91 (2015) 071102, arXiv:1502.06019
  * [15] LHCb collaboration, R. Aaij et al., _Observation of two new ${{\mathchar 28932\relax}^{-}_{b}}$ baryon resonances_, Phys. Rev. Lett. 114 (2015) 062004, arXiv:1411.4849
  * [16] LHCb collaboration, R. Aaij et al., _Measurement of the ${b}$-quark production cross-section in 7 and 13 TeV ${p}{p}$ collisions_, Phys. Rev. Lett. 118 (2016) 052002, arXiv:1612.05140, Erratum in preparation
  * [17] LHCb collaboration, R. Aaij et al., _Evidence for ${C\\!P}$ violation in time-integrated ${{D}^{0}}\rightarrow h^{-}h^{+}$ decay rates_, Phys. Rev. Lett. 108 (2012) 111602, arXiv:1112.0938
  * [18] LHCb collaboration, R. Aaij et al., _Observation of ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$-pair production in ${p}{p}$ collisions at $\sqrt{s}=7$$~{}T\kern-0.50003pteV$_, Phys. Lett. B707 (2012) 52, arXiv:1109.0963
  * [19] CMS collaboration, V. Khachatryan et al., _Measurement of prompt $J/\psi$ pair production in pp collisions at $\sqrt{s}$ = 7 TeV_, JHEP 09 (2014) 094, arXiv:1406.0484
  * [20] D0 collaboration, B. Abbott et al., _Small angle $J/\psi$ production in $p\bar{p}$ collisions at $\sqrt{s}=1.8$ TeV_, Phys. Rev. Lett. 82 (1999) 35, arXiv:hep-ex/9807029
  * [21] CDF collaboration, D. Acosta et al., _Measurement of the $J/\psi$ meson and $b-$hadron production cross sections in $p\bar{p}$ collisions at $\sqrt{s}=1960$ GeV_, Phys. Rev. D71 (2005) 032001, arXiv:hep-ex/0412071
  * [22] PHENIX collaboration, A. Adare et al., _$J/\psi$ production versus transverse momentum and rapidity in $p^{+}p$ collisions at $\sqrt{s}$ = 200 GeV_, Phys. Rev. Lett. 98 (2007) 232002, arXiv:hep-ex/0611020
  * [23] CMS collaboration, V. Khachatryan et al., _Prompt and non-prompt $J/\psi$ production in $pp$ collisions at $\sqrt{s}=7$ TeV_, Eur. Phys. J. C71 (2011) 1575, arXiv:1011.4193
  * [24] LHCb collaboration, R. Aaij et al., _Measurement of ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ production in ${p}{p}$ collisions at $\sqrt{s}=7$ TeV_, Eur. Phys. J. C71 (2011) 1645, arXiv:1103.0423
  * [25] ALICE collaboration, K. Aamodt et al., _Rapidity and transverse momentum dependence of inclusive J $/\psi$ production in $pp$ collisions at $\sqrt{s}=7$ TeV_, Phys. Lett. B704 (2011) 442, arXiv:1105.0380, Erratum: Phys. Lett. B718 (2012) 692
  * [26] ATLAS collaboration, G. Aad et al., _Measurement of the differential cross-sections of inclusive, prompt and non-prompt $J/\psi$ production in proton-proton collisions at $\sqrt{s}=7$ TeV_, Nucl. Phys. B850 (2011) 387, arXiv:1104.3038
  * [27] CMS collaboration, S. Chatrchyan et al., _$J/\psi$ and $\psi(2S)$ production in $pp$ collisions at $\sqrt{s}=7$ TeV_, JHEP 02 (2012) 011, arXiv:1111.1557
  * [28] CDF collaboration, T. A. Aaltonen et al., _Study of orbitally excited $B$ mesons and evidence for a new $B\pi$ resonance_, Phys. Rev. D90 (2014) 012013, arXiv:1309.5961
  * [29] CMS collaboration, V. Khachatryan et al., _Search for Resonant Production of High-Mass Photon Pairs in Proton-Proton Collisions at $\sqrt{s}=8$ and $13$ TeV_, Phys. Rev. Lett. 117 (2016) 051802, arXiv:1606.04093
  * [30] CMS collaboration, V. Khachatryan et al., _Observation of the diphoton decay of the Higgs boson and measurement of its properties_ , Eur. Phys. J. C74 (2014) 3076, arXiv:1407.0558
  * [31] BaBar collaboration, J. P. Lees et al., _Measurement of the inclusive electron spectrum from B meson decays and determination of $|V_{ub}|$_, arXiv:1611.05624
  * [32] Belle collaboration, V. Bhardwaj et al., _Inclusive and exclusive measurements of $B$ decays to ${\chi}_{c1}$ and ${\chi}_{c2}$ at Belle_, Phys. Rev. D 93 (2016) 052016, arXiv:1512.02672
  * [33] LHCb collaboration, R. Aaij et al., _${B}$ flavour tagging using charm decays at the LHCb experiment_, JINST 10 (2015) P10005, arXiv:1507.07892
  * [34] LHCb collaboration, R. Aaij et al., _Observation of ${C\\!P}$ violation in ${{B}^{\pm}}\rightarrow{D}{{K}^{\pm}}$ decays_, Phys. Lett. B712 (2012) 203, Erratum ibid. B713 (2012) 351, arXiv:1203.3662
  * [35] BESIII collaboration, M. Ablikim et al., _Confirmation of a charged charmoniumlike state $Z_{c}(3885)^{\mp}$ in $e^{+}e^{-}\rightarrow\pi^{\pm}(D\bar{D}^{*})^{\mp}$ with double $D$ tag_, Phys. Rev. D92 (2015) 092006, arXiv:1509.01398
  * [36] LHCb collaboration, R. Aaij et al., _Measurements of the ${{{B}^{+}}}$, ${{B}^{0}}$, ${{B}^{0}_{s}}$ meson and ${{\mathchar 28931\relax}^{0}_{b}}$ baryon lifetimes_, JHEP 04 (2014) 114, arXiv:1402.2554
  * [37] DELPHI collaboration, J. Abdallah et al., _Determination of heavy quark non-perturbative parameters from spectral moments in semileptonic $B$ decays_, Eur. Phys. J. C45 (2006) 35, arXiv:hep-ex/0510024
  * [38] DELPHI collaboration, J. Abdallah et al., _Measurement of $|V_{cb}|$ using the semileptonic decay ${{B}^{0}}\\!\rightarrow{{D}^{*+}}{\ell^{-}}{\overline{\nu}}_{\ell}$_, Eur. Phys. J. C33 (2004) 213, arXiv:hep-ex/0401023
  * [39] ALEPH collaboration, R. Barate et al., _The Forward - backward asymmetry for charm quarks at the Z_ , Phys. Lett. B434 (1998) 415, arXiv:hep-ex/9811015
  * [40] Belle collaboration, J. Schumann et al., _Observation of ${{\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}}\\!\rightarrow{{D}^{0}}\eta^{\prime}$ and ${{\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}}\\!\rightarrow{{D}^{*0}}\eta^{\prime}$_, Phys. Rev. D72 (2005) 011103, arXiv:hep-ex/0501013
  * [41] Belle collaboration, S. Nishida et al., _Observation of ${{{B}^{+}}}\rightarrow{{K}^{+}}\eta\gamma$_, Phys. Lett. B610 (2005) 23, arXiv:hep-ex/0411065
  * [42] ATLAS collaboration, M. Aaboud et al., _Search for squarks and gluinos in events with hadronically decaying tau leptons, jets and missing transverse momentum in proton-proton collisions at $\sqrt{s}=13$ TeV recorded with the ATLAS detector_, Eur. Phys. J. C76 (2016) 683, arXiv:1607.05979
  * [43] CMS collaboration, S. Chatrchyan et al., _Measurement of the properties of a Higgs boson in the four-lepton final state_ , Phys. Rev. D89 (2014) 092007, arXiv:1312.5353
  * [44] CMS collaboration, V. Khachatryan et al., _Angular analysis of the decay $B^{0}\rightarrow K^{*0}\mu^{+}\mu^{-}$ from pp collisions at $\sqrt{s}=8$ TeV_, arXiv:1507.08126
  * [45] ATLAS collaboration, G. Aad et al., _Measurement of the Higgs boson mass from the $H\rightarrow\gamma\gamma$ and $H\rightarrow ZZ^{*}\rightarrow 4\ell$ channels with the ATLAS detector using 25 fb-1 of $pp$ collision data_, Phys. Rev. D90 (2014) 052004, arXiv:1406.3827
  * [46] Belle and BaBar collaborations, A. J. Bevan et al., _The Physics of the $B$ Factories_, Eur. Phys. J. C74 (2014) 3026, arXiv:1406.6311
  * [47] BaBar collaboration, B. Aubert et al., _Measurement of the branching fractions for the exclusive decays of $B^{0}$ and $B^{+}$ to $\bar{D}^{(*)}D^{(*)}K$_, Phys. Rev. D68 (2003) 092001, arXiv:hep-ex/0305003
  * [48] BaBar collaboration, J. P. Lees et al., _Study of $B^{\pm,0}\rightarrow J/\psi K^{+}K^{-}K^{\pm,0}$ and search for $B^{0}\rightarrow J/\psi\phi$ at BABAR_, Phys. Rev. D91 (2015) 012003, arXiv:1407.7244
  * [49] Belle collaboration, A. Ishikawa et al., _Measurement of Forward-Backward Asymmetry and Wilson Coefficients in $B\rightarrow K^{*}{\ell^{+}}{\ell^{-}}$_, Phys. Rev. Lett. 96 (2006) 251801, arXiv:hep-ex/0603018
  * [50] BESIII collaboration, M. Ablikim et al., _Measurement of $y_{CP}$ in $D^{0}-\overline{D}^{0}$ oscillation using quantum correlations in $e^{+}e^{-}\rightarrow D^{0}\overline{D}^{0}$ at $\sqrt{s}$ = 3.773 GeV_, Phys. Lett. B744 (2015) 339, arXiv:1501.01378
  * [51] CLEO collaboration, N. E. Adam et al., _A Study of Exclusive Charmless Semileptonic B Decay and $|V_{ub}|$_, Phys. Rev. Lett. 99 (2007) 041802, arXiv:hep-ex/0703041
  * [52] Belle collaboration, S.-K. Choi et al., _Measurements of $B\rightarrow\bar{D}D_{s0}^{*+}(2317)$ decay rates and a search for isospin partners of the $D_{s0}^{*+}(2317)$_, Phys. Rev. D91 (2015) 092011, arXiv:1504.02637, [Addendum: Phys. Rev. D92 (2015) 039905]
  * [53] A candidate arbitration is sometimes part of the design of the selection. For instance the highest $p_{\mathrm{T}}$ lepton, or the most energetic jet may be used as seed to further reconstruction. between data and simulation, or signal and control channels, such cases. More generally, the particle trajectory reconstruction software — known as pattern recognition — applies arbitration by design [85]. The pattern recognition decides which detector signal (”hit”) to add to the trajectory of a particle based its the extrapolation to the detector. The closest hit is usually selected. The tracking efficiency is calibrated using data [86, 87, 88], thus removing potential biases.
  * [54] In many analyses the correct determination of the creation point of the particle of interest is equally important as identifying the signal. Examples are measurements of decay-time dependent observables, or analyses of candidates which do not allow the determination of a decay vertex as diphotons. At colliders the origin is usually the primary vertex, for which ambiguities remaining after the selection are addressed by arbitration [89, 90, 91, 92, 26, 93, 94, 95, 96, 8, 97, 98, 99] or random picking [55, 56, 57, 58, 100]. Wrong assignments should be accounted for by a resolution function or by an additional background contribution.
  * [55] LHCb collaboration, R. Aaij et al., _Measurement of the $B^{0}_{s}\rightarrow J/\psi K^{0}_{S}$ branching fraction_, Phys. Lett. B713 (2012) 172, arXiv:1205.0934
  * [56] LHCb collaboration, R. Aaij et al., _Measurement of the time-dependent ${C\\!P}$ asymmetries in ${{B}^{0}_{s}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{0}_{\mathrm{\scriptscriptstyle S}}}$_, JHEP 06 (2015) 131, arXiv:1503.07055
  * [57] LHCb collaboration, R. Aaij et al., _Observation of the ${{\mathchar 28931\relax}^{0}_{b}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{p}{{\pi}^{-}}$ decay_, JHEP 07 (2014) 103, arXiv:1406.0755
  * [58] LHCb collaboration, R. Aaij et al., _Measurement of the effective ${{B}^{0}_{s}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{0}_{\mathrm{\scriptscriptstyle S}}}$ lifetime_, Nucl. Phys. B873 (2013) 275, arXiv:1304.4500
  * [59] ATLAS collaboration, G. Aad et al., _Searches for heavy long-lived charged particles with the ATLAS detector in proton-proton collisions at $\sqrt{s}=8$ TeV_, JHEP 01 (2015) 068, arXiv:1411.6795
  * [60] CLEO collaboration, M. Artuso et al., _Search for the decay ${{\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}}\\!\rightarrow{{D}^{*0}}\gamma$_, Phys. Rev. Lett. 84 (2000) 4292, arXiv:hep-ex/0001002
  * [61] CLEO collaboration, G. S. Adams et al., _Amplitude analyses of the decays $\chi_{c1}\rightarrow\eta\pi^{+}\pi^{-}$ and $\chi_{c1}\rightarrow\eta^{\prime}\pi^{+}\pi^{-}$_, Phys. Rev. D84 (2011) 112009, arXiv:1109.5843
  * [62] LHCb collaboration, R. Aaij et al., _Measurement of forward ${W}$ and ${Z}$ boson production in ${p}{p}$ collisions at $\sqrt{s}=8$ TeV _, JHEP 01 (2016) 155, arXiv:1511.08039
  * [63] ATLAS collaboration, G. Aad et al., _Measurement of the inclusive $W^{\pm}$ and $Z/\gamma^{\ast}$ cross sections in the electron and muon decay channels in $pp$ collisions at $\sqrt{s}=7$ TeV with the ATLAS detector_, Phys. Rev. D85 (2012) 072004, arXiv:1109.5141
  * [64] An exponential shape would be more physical, but would be an unnecessary complication of the model.
  * [65] In the general case one would expect a different mass distribution for overlapping candidates. This different distribution could then be taken into account in the fit. The author is not aware of any publication where this was done. Therefore it is assumed here that overlaps behave background-like.
  * [66] For example a ${{B}^{0}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{*0}}$ signal decay with ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}\\!\rightarrow{\mu^{+}\mu^{-}}$ and ${{K}^{*0}}\\!\rightarrow{{K}^{+}}{{\pi}^{-}}$ could have a companion ${{\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{\kern 1.99997pt\overline{\kern-1.99997ptK}{}}{}^{*0}}$ candidate which shares the same ${J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}$ but adds an unrelated ${{\kern 1.99997pt\overline{\kern-1.99997ptK}{}}{}^{*0}}\\!\rightarrow{{K}^{-}}{{\pi}^{+}}$ candidate. Such a situation produces a correlation between the ${B}^{0}$ signal yield and the ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$ background yield.
  * [67] Weighting leads to the same results as the random selection provided the weights are properly taken into account in the fit. Event removal is equivalent to random picking with probability $0$ instead of $\nicefrac{{1}}{{2}}$. The resulting biases have twice the size of those obtained with this method.
  * [68] Mathematical expressions can be found in the ancillary material.
  * [69] Variation of the probability of a background candidate to have a companion is tested with ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}={{\cal P}^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}=0$. When taking all candidates, the effect of larger ${{\cal P}^{{{B}^{0}}}_{\text{bkg}}}$ and ${{\cal P}^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{bkg}}}$ values is to increase the background level, which does not bias any measurement but degrades the uncertainty in the same way as ${{\cal P}^{{{B}^{0}}}_{\text{sig}}}$ and ${{\cal P}^{{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{sig}}}$ do. However, when a candidate selection technique is applied, only background candidates get removed resulting in a situation equivalent to the clean case.
  * [70] This systematic uncertainty may be negligible and thus not be reported.
  * [71] In the decay ${{{B}^{+}}}\\!\rightarrow{{D}^{+}}{{\pi}^{0}}$ companion candidates can be generated starting from the signal ${D}^{+}$ and adding a fake ${\pi}^{0}$. By construction, the two candidates have the same flavour and the swapping probabilities are zero. In the decay ${{{B}^{+}}}\\!\rightarrow{D}_{C\\!P}{\mu^{+}}{\nu}$ where ${D}_{C\\!P}$ is a ${D}^{0}$ meson decaying to a $C\\!P$ eigenstate, if a true ${D}_{C\\!P}$ is combined with a muon from the other $B$, the latter is likely to be of the opposite flavour and hence the flavours of signal and overlapping candidate are anti-correlated. The swapping probabilities are close to unity. For the decay ${{B}^{0}}\\!\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{*0}}$ [66], the flavour of the fake ${B}^{0}$ candidate depends on the ${K}^{*0}$ to ${\kern 1.99997pt\overline{\kern-1.99997ptK}{}}{}^{*0}$ production ratio, which may be slightly (anti-)correlated with that of ${B}^{0}$ versus ${\kern 1.79993pt\overline{\kern-1.79993ptB}{}}{}^{0}$, ${{\cal P}^{\text{sig }{{B}^{0}}}_{\text{swap}}}\\!\simeq{{\cal P}^{\text{sig }{{\kern 1.25995pt\overline{\kern-1.25995ptB}{}}{}^{0}}}_{\text{swap}}}\\!\simeq 0.5$.
  * [72] In reality the actual factors may differ considerably from the values $\nicefrac{{1}}{{2}}$ and $\nicefrac{{1}}{{3}}$ used here and the distributions would not be pure exponential functions. However, the conclusions are not affected as long as the distributions are smooth curves.
  * [73] This resolution function is irrelevant for this study, but helps making the fit more stable.
  * [74] M. Pivk and F. R. Le Diberder, _sPlot: A statistical tool to unfold data distributions_ , Nucl. Instrum. Meth. A555 (2005) 356, arXiv:physics/0402083
  * [75] Y. Xie, _sFit: a method for background subtraction in maximum likelihood fit_ , arXiv:0905.0724
  * [76] The situation may be very different if the overlaps are peaking in mass.
  * [77] LHCb collaboration, R. Aaij et al., _Measurement of ${C\\!P}$ asymmetry in ${{B}^{0}_{s}}\rightarrow{{D}^{\mp}_{s}}{{K}^{\pm}}$ decays_, JHEP 11 (2014) 060, arXiv:1407.6127
  * [78] K. De Bruyn et al., _Branching Ratio Measurements of $B_{s}$ Decays_, Phys. Rev. D86 (2012) 014027, arXiv:1204.1735
  * [79] ATLAS and CMS collaborations, G. Aad et al., _Combined Measurement of the Higgs Boson Mass in $pp$ Collisions at $\sqrt{s}=7$ and $8$ TeV with the ATLAS and CMS Experiments_, Phys. Rev. Lett. 114 (2015) 191803, arXiv:1503.07589
  * [80] CMS collaboration, V. Khachatryan et al., _Charged particle multiplicities in $pp$ interactions at $\sqrt{s}=0.9$, 2.36, and 7 TeV_, JHEP 01 (2011) 079, arXiv:1011.5531
  * [81] LHCb collaboration, R. Aaij et al., _Measurement of charged particle multiplicities and densities in ${p}{p}$ collisions at $\sqrt{s}=7$$~{}T\kern-0.50003pteV$ in the forward region_, Eur. Phys. J. C74 (2014) 2888, arXiv:1402.4430
  * [82] This may not be true in the general case. If the mass distribution of companion candidates and of the background are different, the sPlot technique [74] may not be able to disentangle all species correctly, unless all these distributions are known.
  * [83] CMS collaboration, S. Chatrchyan et al., _Angular analysis and branching fraction measurement of the decay $B^{0}\rightarrow K^{*0}\mu^{+}\mu^{-}$_, Phys. Lett. B727 (2013) 77, arXiv:1308.3409
  * [84] LHCb collaboration, R. Aaij et al., _Measurement of the S-wave fraction in ${{B}^{0}}\rightarrow{{K}^{+}}{{\pi}^{-}}{\mu^{+}}{\mu^{-}}$ decays and the ${{B}^{0}}\rightarrow{{K}^{*}}(892)^{0}{\mu^{+}}{\mu^{-}}$ differential branching fraction_, JHEP 11 (2016) 047, Erratum ibid. 04 (2017) 142, arXiv:1606.04731
  * [85] R. Frühwirth, _Application of Kalman filtering to track and vertex fitting_ , Nucl. Instrum. Meth. A262 (1987) 444
  * [86] ATLAS collaboration, _Performance of the ATLAS Inner Detector Track and Vertex Reconstruction in the High Pile-Up LHC Environment_ ,
  * [87] CMS collaboration, S. Chatrchyan et al., _Description and performance of track and primary-vertex reconstruction with the CMS tracker_ , JINST 9 (2014) P10009, arXiv:1405.6569
  * [88] LHCb collaboration, R. Aaij et al., _Measurement of the track reconstruction efficiency at LHCb_ , JINST 10 (2015) P02007, arXiv:1408.1251
  * [89] ATLAS collaboration, G. Aad et al., _Search for direct production of charginos, neutralinos and sleptons in final states with two leptons and missing transverse momentum in $pp$ collisions at $\sqrt{s}=$ 8 TeV with the ATLAS detector_, JHEP 05 (2014) 071, arXiv:1403.5294
  * [90] ATLAS collaboration, G. Aad et al., _Search for strong gravity signatures in same-sign dimuon final states using the ATLAS detector at the LHC_ , Phys. Lett. B709 (2012) 322, arXiv:1111.0080
  * [91] ATLAS collaboration, G. Aad et al., _Search for the direct production of charginos, neutralinos and staus in final states with at least two hadronically decaying taus and missing transverse momentum in $pp$ collisions at $\sqrt{s}$ = 8 TeV with the ATLAS detector_, JHEP 10 (2014) 096, arXiv:1407.0350
  * [92] ATLAS collaboration, G. Aad et al., _Search for displaced vertices arising from decays of new heavy particles in 7 TeV pp collisions at ATLAS_ , Phys. Lett. B707 (2012) 478, arXiv:1109.2242
  * [93] R. Aaij et al., _The LHCb trigger and its performance in 2011_, JINST 8 (2013) P04022, arXiv:1211.3055
  * [94] LHCb collaboration, R. Aaij et al., _Study of the kinematic dependences of ${{\mathchar 28931\relax}^{0}_{b}}$ production in ${p}{p}$ collisions and a measurement of the ${{\mathchar 28931\relax}^{0}_{b}}\rightarrow{{\mathchar 28931\relax}^{+}_{c}}{{\pi}^{-}}$ branching fraction_, JHEP 08 (2014) 143, arXiv:1405.6842
  * [95] LHCb collaboration, R. Aaij et al., _Measurement of forward ${{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}$ production cross-sections in ${p}{p}$ collisions at $\sqrt{s}=13$$~{}T\kern-0.50003pteV$_, JHEP 10 (2015) 172, Erratum ibid. 05 (2017) 063, arXiv:1509.00771
  * [96] M. Kucharczyk, P. Morawski, and M. Witek, _Primary Vertex Reconstruction at LHCb_ , tech. rep., CERN, Geneva, Sep, 2014
  * [97] LHCb collaboration, R. Aaij et al., _Precision measurement of ${C\\!P}$ violation in ${{B}^{0}_{s}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{+}}{{K}^{-}}$ decays_, Phys. Rev. Lett. 114 (2015) 041801, arXiv:1411.3104
  * [98] LHCb collaboration, R. Aaij et al., _Measurement of ${C\\!P}$ violation in ${{B}^{0}_{s}}\rightarrow\phi\phi$ decays_, Phys. Rev. D90 (2014) 052011, arXiv:1407.2222
  * [99] LHCb collaboration, R. Aaij et al., _First measurement of time-dependent ${C\\!P}$ violation in ${{B}^{0}_{s}}\rightarrow{{K}^{+}}{{K}^{-}}$ decays_, JHEP 10 (2013) 183, arXiv:1308.1428
  * [100] LHCb collaboration, R. Aaij et al., _Measurement of ${C\\!P}$ violation in ${{B}^{0}}\rightarrow{{J\mskip-3.0mu/\mskip-2.0mu\psi\mskip 2.0mu}}{{K}^{0}_{\mathrm{\scriptscriptstyle S}}}$ decays_, Phys. Rev. Lett. 115 (2015) 031601, arXiv:1503.07089