# New simulation software technologies
at the LHCb Experiment at CERN

Michał Mazurek1,2 Gloria Corti1 Dominik Müller1 1CERN, European Organization
for Nuclear Reasearch
Esplanade des Particules 1, 1211 Meyrin, Switzerland
2NCBJ, National Centre for Nuclear Research
Andrzeja Sołtana 7, 05-400 Otwock, Poland<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The LHCb experiment at the Large Hadron Collider (LHC) at CERN has
successfully performed a large number of physics measurements during Runs 1
and 2 of the LHC. Monte Carlo simulation is key to the interpretation of these
and future measurements. The LHCb experiment is currently undergoing a major
detector upgrade for Run 3 of the LHC to process events with five times higher
luminosity. New simulation software technologies have to be introduced to
produce simulated data samples of sufficient size within the computing
resources allocated for the next few years. Therefore, the LHCb collaboration
is currently preparing an upgraded version of its Gauss simulation framework.
The new version provides the LHCb specific functionality while its generic
simulation infrastructure has been encapsulated in an experiment independent
framework, Gaussino. The latter combines the Gaudi core software framework and
the Geant4 simulation toolkit and fully exploits their multi-threading
capabilities. A prototype of a fast simulation interface to the simulation
toolkit is being developed as the latest addition to Gaussino to provide an
extensive palette of fast simulation models, including new deep learning-based
options.

###### keywords:

LHCb, CERN, Monte Carlo simulation, fast simulations,
multi-threading

68-04

Submitted to Computing and Informatics

##  1 Introduction

The LHCb detector is a single-arm spectrometer with a forward angular
coverage, designed for the study of particles containing $b$ or $c$ quarks. It
is located at the Large Hadron Collider (LHC) at the European Organization for
Nuclear Research (CERN). During the LHC Run 1 and Run 2 the LHCb experiment
has successfully performed a large number of measurements in heavy flavour
physics, however, the precision of its measurements is becoming more and more
limited by the statistics of the simulated samples. In addition, the LHCb
experiment is currently installing a major upgrade for the data taking of Run
3 in order to be able to process events with a 5 times higher luminosity. The
LHCb experiment is also planning a further upgrade with another increase in
luminosity by a factor of 5 to 10. Therefore, higher capacity in data storage
and computing power is needed in order to prepare the experiment for the
changes in the future Runs. Moreover, the whole software is being adapted to
work in a multi-threaded environment in order to efficiently use the available
computing resources. As an example, the experiment will rely on a real-time
reconstruction trigger implemented on GPUs. In the last years, the LHCb Gauss
simulation framework has been evolving to provide a palette of fast
simulations options and to use the new functional Gaudi framework and Geant4
multi-threading technologies. In this paper, recent developments in the
simulation software are discussed. A brief description of the LHCb detector
and its upgrade are presented in Section 2. Details about the LHCb simulation
software and the new multi-threading approach can be found in Section 3.
Finally, a description of the fast simulation interface in Gauss-on-Gaussino
with an example on how to use the libraries needed to prepare a training
dataset is presented in Section 4.

##  2 LHCb Detector

The LHCb detector [1, 2] covers the pseudorapidity range $2<\eta<5$. It
includes a high-precision tracking system consisting of a silicon-strip vertex
detector (VErtex LOcator, VELO) surrounding the $pp$ interaction region, a
large-area silicon-strip detector (TT) located upstream of a dipole magnet
with a bending power of about $4{\mathrm{\,Tm}}$, and three stations of
silicon-strip detectors and straw drift tubes placed downstream of the magnet.
The tracking system (T1-T3) provides the momentum measurement of charged
particles. Different types of charged hadrons are distinguished using
information from two ring-imaging Cherenkov detectors, located upstream and
downstream of the magnet (RICH1 and RICH2). Photons, electrons and hadrons are
identified by the LHCb calorimeter system [3]. An Electromagnetic CALorimeter
(ECAL) and a Hadronic CALorimeter (HCAL) determine the amount of energy
deposited by the particles of electromagnetic or hadronic nature,
respectively. Some minimum ionizing particles may also be registered by the
calorimeters. Muons are identified by a system composed of alternating layers
of iron and multiwire proportional chambers (M1-M5). In order to reduce the
number of events collected and maximizing those of interest at the same time,
an online event selection is performed via a combination of hardware and
software triggers. The L0 trigger (hardware stage) operates on the information
coming from the calorimeter and muon systems at a rate of 40  MHz, followed by
a High Level Trigger (HLT, software stage), which applies a full event
reconstruction.

The LHCb collaboration is currently installing a major upgrade that will allow
the experiment to significantly increase the amount of data collected, by
operating with a luminosity of up to $2\times
10^{33}\;\mathrm{cm}^{-2}\mathrm{s}^{-1}$, 5 times higher than that of Run 1
and Run 2 and writing data at a 30  MHz rate. In order to cope with the higher
readout rates, the front-end electronics of all the sub-detectors will be
replaced. Moreover, most of the LHCb sub-detectors will be replaced and only
the heaviest ones, namely the calorimeters and muon stations, will remain
mostly in place. In Run 1 and Run 2 the event output rate was limited to 1
MHz and through a hardware-based L0 trigger. This has been removed for Run 3
and LHCb is introducing a software-only trigger working at the 30  MHz bunch
crossing rate. Processing and the selection of events at this very high rate
have a major impact on software and computing systems.

##  3 Simulation software at LHCb

In LHCb, data collected by the detector is processed using a set of custom
applications based on the Gaudi [4, 5] core software framework. A simplified
view on the LHCb data processing applications, as used in Run 1 and Run 2, is
shown in Figure 1. LHCb applications identically process events either
collected by the detector itself or events produced by the simulation
software. Simulated events are first handled by Gauss that performs the event
generation and tracking of particles through the detector material. The Boole
application mimics the specific sub-detector technologies and electronics
response, providing the same digital output of the data acquisition system.
The Moore application then emulates the L0 trigger response for the simulated
data and provides the software-based HLT trigger. More complex objects are
produced by the offline reconstruction in Brunel. Each application reads and
writes processed data to storage.

Figure 1: Simplified view on the LHCb data flow and data processing
applications as used in Run 1 and Run 2.

###  3.1 Gauss simulation software

The LHCb simulation framework Gauss [6] is built, as all the other LHCb
software applications, using the general Gaudi data-processing framework.
Gaudi helps in the configuration of algorithms and tools in the application.
It also controls the data flow and the event loop. Dependencies in the
simulation framework are illustrated in Figure 2(a). One of the two main tasks
of the Gauss framework is to control the generation of collisions, in most
cases $pp$ with Pythia [7, 8], where a specific LHCb configuration [9] is
used. Gauss then makes use of EvtGen [10] in order to model the decays of
unstable particles. The other main task of Gauss is the propagation of the
generated particles through the experimental apparatus and the simulation of
physics processes occurring within the sub-detectors using the Geant4 toolkit
[11, 12, 13]. The information to be able to mimic the response of given sub-
detectors when a particle intersects them (MC hits) and to understand trigger
and reconstruction performance (MC truth) are written to a file for further
processing.

###  3.2 LHCb software changes for Run 3

As mentioned in the first section, providing a high-throughput software
relying on a better utilization of modern computing architectures is crucial
in order to cope with the higher event output rate. A typical LHCb application
needs around 2 to 4  GB RAM on average [14]. Therefore, a multi-process
approach, where each instance of the application is run on a separate core, is
not optimal from the memory consumption point of view. As a result, the LHCb
core software framework, Gaudi, has been re-engineered to work in a multi-
threading mode in order to benefit from the inter-event parallelism as much as
possible. A lot of effort has been put in order to guarantee thread-safety, as
many objects in the application are now shared between the events. This has
been achieved mostly by introducing a functional paradigm, in which a set of
algorithms operate on data by either producing, transforming or consuming it,
and ensuring that the internal state is unchanged. The implementation of a new
scheduler [15] and the optimization of the software by utilizing vector
processing units, introducing new C++ features and rethinking the data models,
were the other key components in the new Gaudi framework.

Geant4 Pythia8 EvtGen GaudiLHCbGauss 111Additional LHCb-specific configuration
for Geant4, Pythia and EvtGen is not shown in the dependency graph. (a) Gauss
Run 1 and Run 2 framework.

Geant4Pythia8GaudiLHCbGaussinoGauss ††footnotemark:  (b) Gauss-on-Gaussino
framework

Figure 2: Dependencies [16] in the simulation software stack before and after
upgrade.

Gauss, the LHCb simulation software, like all the other LHCb applications,
depends on the changes introduced in Gaudi. Moreover, Gauss has to adapt to
the changes introduced by the new approach in the detector description. DD4hep
[17] is a detector description toolkit that provides experiment-independent
libraries for conditions management and visualization of the geometry. On the
other hand, Gauss still has to provide support for the older geometries when
producing simulated samples for Run 1 and Run 2 physics analyses. Additional
challenge arises from the fact that Gauss delegates tasks to external
libraries used in the framework (e.g. Geant4) that are already working in
their own multi-threading realms.

###  3.3 Gauss-on-Gaussino

In order to address the challenges mentioned above, the LHCb simulation team
decided to move all the LHCb-independent components from the simulation
software and place it in a separate project, called Gaussino [18, 19, 20], as
a core simulation framework, on which all the new versions of Gauss will be
built. It is also possible to run Gaussino as a standalone application.
Dependencies in the new LHCb version of the framework, called here Gauss-on-
Gaussino for clarity, are illustrated in Figure 2(b). Gaussino follows the
Gaudi’s inter-event-based parallelism of the event loop, in which algorithms
are scheduled in a way that guarantees thread-safety. Gaussino communicates
with Geant4 objects by creating corresponding factories that act as Gaudi
tools. In the event loop, Gaudi places the generated event on the top of a
FIFO queue. Geant4 worker threads then take the event from the queue and
perform the simulation following their own multi-threading scheme. The multi-
threaded approach in Gauss-on-Gaussino has already made it possible to
simulate more events in time by limiting the memory consumption of each event
as shown in an earlier paper [20]. Nevertheless, this will not be enough to
meet the requirements imposed by the upgrade of the detector, which will be
discussed in the following section.

##  4 Fast simulation interface in Gauss-on-Gaussino

Producing necessary simulated samples for physics analyses with Gauss consumed
around 80% [21, 22, 23] of all the distributed computing resources available
to LHCb during Run 2. An additional few percent were used to process the
simulated samples through the subsequent applications. The increase in the
number of events, caused by the upcoming upgrade, will require to simulate
even more events. If all samples would be produced with a detailed simulation,
the foreseen computing resources allocated to LHCb will be quickly exceeded
(Figure 3). Scenarios with different fractions of the samples produced with
detailed simulation, fast, and ultra-fast models would allow to fulfill the
needs within the resource forecast.

Figure 3: Projection of the computing resources available to LHCb [21]. Monte
Carlo simulations will quickly exceed in the upcoming years the resources
allocated to LHCb by the distributed computing system provided that no action
is taken in order to fit in the requirements (e.g. fast simulations). The CPU
needs for producing 100% of the required samples with the detailed simulation
is shown by the green bars. The CPU needs in two scenarios with different
fractions of events produced with detailed, fast and ultra-fast simulations
are shown by the blue and dashed purple lines respectively.

A lot of efforts have been made by the Gauss developers in order to reduce the
CPU time spent in the simulation. In Gauss, propagating particles through
matter dominates the time spent by the application. A finer analysis of the
relative time spent in each sub-detector of the upgrade geometry is presented
in Figure 4(a) for the current version of the framework. As seen in the
results, simulation of the particle showers in the calorimeters is especially
demanding. Consistent results are obtained for Gauss-on-Gaussino as expected
and shown in Figure 4(b). Additional information about the time spent by each
particle is given in Figure 10 and Figure 11 in the Appendix.

A campaign with the goal to introduce a palette of fast simulation models to
complement the detailed Geant4-based simulation has been launched. ReDecay
[24] is a technique, in which the underlying $pp$ interaction is reused in the
simulation of the detector multiple times, with an independently generated
signal decay for each event. Lamarr [25] is an in-house, ultra-fast
parametrization framework that extends up to the reconstruction level and
provides high-level reconstruction objects in the output. When it comes to the
calorimeters, a fast simulation model of ECAL based on a point library [26] is
currently being implemented. There are already a few proposals of fast
simulation models based on generative adversarial networks [27] for the
calorimeters and RICH detectors, however, a special interface is needed in
Gaussino to exploit them via the fast simulation mechanisms available in
Geant4. Recent work to support the implementation of these last fast
simulations in Gaussino is described in the rest of this section.

(a) Current Gauss framework with Geant4 v10.6.

††∗ Simulation of the optical photons in RICH1 and RICH2 was turned off.

(b) New multi-threaded (1 thread) Gauss-on-Gaussino framework with Geant4
v10.7.

Figure 4: Relative time [16] spent in each sub-detector when simulating 100
minimum bias events using different versions of the framework with the beam
conditions as foreseen in the Run 3 data-taking period and the upgrade
geometry. The time spent in the calorimeters is very similar in the current
and new versions of the framework as expected. Simulation of the optical
photons in RICH1 and RICH2 was turned off in the current version of Gauss as
it is not yet available in Gauss-on-Gaussino, for easier comparison.

###  4.1 Implementation of the interface

Gaussino provides a generic FastSimulation interface to Geant4 objects in
order to minimize the work spent in the future on implementing fast simulation
models, and also to guarantee the integrity of the simulated data. Following
the convention already present in Gaussino, the FastSimulation interface
consists of factories ensuring Geant4 objects are configured properly and at
the right moment when running the application. A set of the most important
factories and their Geant4 counterparts are presented in Figure 5. Most of the
work necessary to implement a fast simulation model itself can be pushed to
the configuration in python files. The most optimistic scenario is that the
developer will only have to implement a G4VFastSimulationModel::DoIt()
callback method in C++ that is the key component of the whole interface and
actually describes the whole process of generating fast hits.

GaussinoFast
SimulationPhysicsFactoryRegionFactoryModelFactoryDetectorFactoryGeant4G4FastSimulationPhysicsG4RegionG4VFastSimulationModelG4VSensitiveDetector
construct() construct() construct() construct() Sensitive Detector
Construction Hit Extraction Figure 5: A simplified model [16] of the
FastSimulation interface with a set of dedicated factories that construct the
corresponding Geant4 objects.

Two, purely abstract, fast simulation models are introduced in Gaussino in
order to measure the performance of the interface, and to mark off a lower
bound for all further fast simulation models. An ImmediateDeposit model
generates one hit per particle that intercepts the region where the
ImmediateDeposit model is active and deposits all of its energy in that hit.
ImmediateDeposit gives useful information about the timing needed for the
infrastructure itself to call the fast simulation methods. The ShowerDeposit
model works in a similar manner, but it splits the energy of a particle into a
selected number of hits, and generates them randomly around the position where
the particle intercepted the region. ShowerDeposit provides the minimum amount
of time needed to generate a specific number of hits with no additional
calculations.

Figure 6: Comparison of the time [16] spent by different fast simulation
models (ImmediateDeposit and ShowerDeposit) and a detailed simulation with
Geant4 in the electromagnetic calorimeter. In each of the models tested, a
particle gun generates a grid of evenly-spaced photons of a particular energy.
For the detailed simulation the time of the current version of Gauss is also
given as reference.

A comparison between different fast simulation models tested with a particle
gun that creates a grid of 3328 evenly-spaced photons originating at the LHCb
interaction point, is presented in Figure 6. Time spent in the
ImmediateDeposit model is comparable across different photon energies and
works as expected. On the other hand, the time spent in the ShowerDeposit
model increases with the energy of photons. Naturally, this is caused by a
larger number of hits generated in each shower. In the worst case, a 100  GeV
photon generates 21558 hits on average in the calorimeter. Only around 25
seconds are needed for the fast simulation infrastructure in Gaussino with 3
threads to simulate 3328 x 21558 hits. In a detailed simulation, the time
needed to simulate the same number of hits rises to 2145 seconds. The results
prove that the infrastructure, used by the FastSimulation interface, provides
the possibility to significantly improve the time spent by the simulation
software in the detector, provided the fast simulation model gives a similar
level of precision in physics.

###  4.2 Fast simulation training datasets

Many of the advanced, fast simulation models require prior tuning or training
on some input data in order to provide valid results. When implementing a fast
simulation model, a developer specifies a region that does not necessarily
have to coincide with the LHCb sub-detectors boundaries. The information
required to train these models is not always available in an output file, as
Gauss stores only the minimum amount of information required for physics
studies. Therefore, the developer should be given the possibility to turn off
any unneeded optimization features and to gather information at any given
place in the detector.

Information about the simulated objects can be easily obtained by introducing
a new sensitive sub-detector that would register hits of an abstract type with
all the information needed to train the fast simulation model. A few
difficulties may be encountered with this approach though. Since these
detectors will only be used in just a few, specific studies, setting them up
should be configurable on the fly without having to introduce them in the
existing detector description, hosted in a database in an xml format. This
functionality is provided by a new package in Gaussino called
ExternalDetector, that allows for sub-detectors of any shape to be inserted.
An example of the external plane-like detector, as seen by Geant4, is
illustrated in Figure 7. A side view of the same setup is presented in Figure
8, together with simulated particles information.

Figure 7: LHCb upgrade geometry [16] as seen by the Geant4 toolkit with an
example of a plane-like detector (red, thin plane), introduced by the
ExternalDetector package. When used as a collector of particle information, it
may provide the source of training information about incident particles for
all the sub-detectors placed downstream from it along the beamline: ECAL (cyan
box), HCAL (orange box), or muon system (green box).  Figure 8: Particles
generated [16] using the current Gauss framework in the simulation of a
minimum bias event with the beam conditions as foreseen in the Run 3 data-
taking period and the upgrade geometry. An external plane-like detector that
collects information about traversing particles is depicted with the red
color.

The user can also choose what kind of factories should be attached to an
external detector. In principle, the external detector can become a sensitive
detector (i.e. it will be able to register hits) or the user can add a
monitoring tool that will be launched at the end of each event in order to
verify the integrity of the collected data. MCCollector provides a set of
abstract sensitive detector factories that are easier to configure than those
used in the standard simulation. Finally, it might be the case that the
external detector will overlap with other existing volumes in the geometry.
ParallelGeometry exploits an abstract concept introduced by Geant4 that allows
for having multiple geometries in parallel, each of them performing the
particle transport without interference from objects defined in other
geometries.

The packages provide a generic way of producing the training datasets for fast
simulation models. Two simple examples are presented in this paper in order to
show how they can be used for the models under development for ECAL (e.g.
point library [26] and GANs [27]). Visual representations of the training
datasets, produced by placing a collector plane in front of ECAL, are
illustrated in Figure 9. In Figure 9(a), a grid of 3328 evenly-spaced photons,
similar to that used as an example when testing the performance of the
interface, is shown. In Figure 9(b), an input for the fast simulation studies
requiring minimum bias events is presented.

This new interface will be exploited to provide training samples and integrate
fast simulation models for given sub-detectors in Gaussino.

(a) Particle gun with a grid of 3328 evenly-spaced 100  MeV photons.

(b) Minimum bias event with the beam conditions as foreseen in the Run 3 data-
taking period and the upgrade geometry.

Figure 9: Visualization [16] of the training dataset produced by placing a
collector plane in front of the electromagnetic calorimeter. Each of the
images represent the ECAL energy deposits (hits) projected onto an $xy$-plane.
The main role of the collector plane is to gather the positions of all
particles intercepting the front face of ECAL. The particle positions are then
linked (red lines) with the energetic centers of the showers generated by
these particles.

##  5 Conclusions

Monte Carlo simulation is a key component to model the effects of detector
acceptance and imposed selection requirements. In order to address the
computing constraints of the future upgrades of the LHCb detector, the LHCb
simulation software Gauss has been redesigned to provide support for inter-
event parallelism and integrate fast simulation models in Geant4. In this
paper, a FastSimulation interface in Gaussino, the new experiment independent
core simulation framework developed in LHCb, was presented alongside with
performance tests that show how the multi-threaded infrastructure of the fast
simulation interface has a negligible contribution to the execution time. A
general way of producing the training datasets for the fast simulation models
using the LHCb setup was also discussed. This new functionality will make it
possible to integrate in the Gaussino framework the fast simulation models
that are currently being developed in the LHCb experiment as well as future
ones.

## References

  * [1] LHCb Collaboration: The LHCb detector at the LHC, In: JINST, Vol. 3, No. LHCb-DP-2008-001, 2008 p. S08005, 10.1088/1748-0221/3/08/S08005.
  * [2] LHCb Collaboration: LHCb detector performance, In: Int. J. Mod. Phys., Vol. A30, 2015 p. 1530022, 10.1142/S0217751X15300227.
  * [3] Abellan Beteta, C. et al.: Calibration and performance of the LHCb calorimeters in Run 1 and 2 at the LHC, https://cds.cern.ch/record/2729028, 2020\.
  * [4] Barrand, G.—Belyaev, I.—Binko, P.—Cattaneo, M.—Chytracek, R.—Corti, G.—Frank, M.—Gracia, G.—Harvey, J.—Herwijnen, E.—Maley, P.—Mato, P.—Probst, S.— and Ranjard, F.: GAUDI — A software architecture and framework for building HEP data processing applications, In: Computer Physics Communications, Vol. 140, No. 1, 2001 pp. 45–55, https://doi.org/10.1016/S0010-4655(01)00254-5, cHEP2000.
  * [5] Clemencic, M.—Degaudenzi, H.—Mato, P.—Binet, S.—Lavrijsen, W.—Leggett, C.— and Belyaev, I.: Recent developments in the LHCb software framework gaudi, In: Journal of Physics: Conference Series, Vol. 219, No. 4, 2010 p. 042006, 10.1088/1742-6596/219/4/042006.
  * [6] Clemencic, M.—Corti, G.—Easo, S.—Jones, C. R.—Miglioranzi, S.—Pappagallo, M.— and and, P. R.: The LHCb Simulation Application, Gauss: Design, Evolution and Experience, In: Journal of Physics: Conference Series, Vol. 331, No. 3, 2011 p. 032023, 10.1088/1742-6596/331/3/032023.
  * [7] Sjöstrand, T.—Mrenna, S.— and Skands, P.: PYTHIA 6.4 physics and manual, In: JHEP, Vol. 05, 2006 p. 026, 10.1088/1126-6708/2006/05/026.
  * [8] Sjöstrand, T.—Mrenna, S.— and Skands, P.: A brief introduction to PYTHIA 8.1, In: Comput. Phys. Commun., Vol. 178, 2008 pp. 852–867, 10.1016/j.cpc.2008.01.036.
  * [9] Belyaev, I. et al.: Handling of the generation of primary events in Gauss, the LHCb simulation framework, In: J. Phys. Conf. Ser., Vol. 331, 2011 p. 032047, 10.1088/1742-6596/331/3/032047.
  * [10] Lange, D. J.: The EvtGen particle decay simulation package, In: Nucl. Instrum. Meth., Vol. A462, 2001 pp. 152–155, 10.1016/S0168-9002(01)00089-4.
  * [11] Allison, J.—Amako, K.—Apostolakis, J.—Araujo, H.—Dubois, P.— et al.: Geant4 developments and applications, In: IEEE Trans.Nucl.Sci., Vol. 53, 2006 p. 270, 10.1109/TNS.2006.869826.
  * [12] Agostinelli, S. et al.: Geant4: A simulation toolkit, In: Nucl. Instrum. Meth., Vol. A506, 2003 p. 250, 10.1016/S0168-9002(03)01368-8.
  * [13] Clemencic, M. et al.: The LHCb simulation application, Gauss: Design, evolution and experience, In: J. Phys. Conf. Ser., Vol. 331, 2011 p. 032023, 10.1088/1742-6596/331/3/032023.
  * [14] Bozzi, C.—Ponce, S.— and Roiser, S.: The core software framework for the LHCb Upgrade, In: EPJ Web Conf., Vol. 214, 2019 p. 05040, 10.1051/epjconf/201921405040.
  * [15] Matev, R.: A 30 MHz software trigger for the LHCb upgrade, https://cds.cern.ch/record/2631781, Jul 2018.
  * [16] LHCb Collaboration: Performance of the fast simulation interface in Gauss-on-Gaussino, https://cds.cern.ch/record/2781378, Sep 2021.
  * [17] Frank, M.—Gaede, F.—Grefe, C.— and Mato, P.: DD4hep: A Detector Description Toolkit for High Energy Physics Experiments, In: Journal of Physics: Conference Series, Vol. 513, No. 2, 2014 p. 022010, 10.1088/1742-6596/513/2/022010.
  * [18] Siddi, B. G. and Müller, D.: Gaussino - a Gaudi-Based Core Simulation Framework, In: 2019 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC), 2019, pp. 1–4, 10.1109/NSS/MIC42101.2019.9060074.
  * [19] Müller, D.: Adopting new technologies in the LHCb Gauss simulation framework, In: EPJ Web Conf., Vol. 214, 2019 p. 02004, 10.1051/epjconf/201921402004.
  * [20] LHCb Collaboration: Performance of a multithreaded prototype for the future LHCb simulation framework (Gauss-on-Gaussino), https://cds.cern.ch/record/2694003, Oct 2019.
  * [21] LHCb Collaboration: LHCb CPU Usage Forecast, https://cds.cern.ch/record/2696552h, 2019.
  * [22] Whitehead, M. P.: A Palette of Fast Simulations in LHCb, https://cds.cern.ch/record/2630475, Jul 2018.
  * [23] LHCb collaboration: Computing Model of the Upgrade LHCb experiment, http://cds.cern.ch/record/2319756, 2018, CERN-LHCC-2018-014, LHCb-TDR-018.
  * [24] Müller, D.—Clemencic, M.—Corti, G.— and Gersabeck, M.: ReDecay: A novel approach to speed up the simulation at LHCb, In: Eur. Phys. J., Vol. C78, 2018 p. 1009, 10.1140/epjc/s10052-018-6469-6.
  * [25] LHCb Collaboration: Performance of the Lamarr Prototype: the ultra-fast simulation option integrated in the LHCb simulation framework, https://cds.cern.ch/record/2696310, Oct 2019.
  * [26] Rama, M. and Vitali, G.: Calorimeter fast simulation based on hit libraries LHCb Gauss framework, In: EPJ Web Conf., Vol. 214, 2019 p. 02040, 10.1051/epjconf/201921402040.
  * [27] Ratnikov, F.: Generative Adversarial Networks for LHCb Fast Simulation, In: EPJ Web Conf., Vol. 245, 2020 p. 02026, 10.1051/epjconf/202024502026.

##  Appendix A Appendix

(a) Relative time spent by each particle in a sub-detector measured with
respect to the total time spent on the simulation.

(b) Relative time by each particle in a sub-detector measured with respect to
the time spent in that sub-detector.

Figure 10: Performance [16] of the current Gauss framework when simulating 100
minimum bias events with the beam conditions as foreseen in the Run 3 data-
taking period and the upgrade geometry.

(a) Relative time by each particle in a sub-detector measured with respect to
the total time spent on the simulation.

(b) Relative time by each particle in a sub-detector measured with respect to
the time spent in that sub-detector.

Figure 11: Performance [16] of the new multi-threaded (1 thread) Gaussino-
based framework when simulating 100 minimum bias events with the beam
conditions as foreseen in the Run 3 data-taking period and the upgrade
geometry.