Design and performance of the LHCb trigger and full real-time reconstruction in Run 2 of the LHC
R. Aaij S. Akar J. Albrecht M. Alexander A. Alfonso Albero S. Amerio L. Anderlini P. d'Argent A. Baranov W. Barter S. Benson D. Bobulska T. Boettcher S. Borghi E. E. Bowen L. Brarda C. Burr J. -P. Cachemiche M. Calvo Gomez M. Cattaneo H. Chanal M. Chapman M. Chebbi M. Chefdeville P. Ciambrone J. Cogan S. -G. Chitic M. Clemencic J. Closier B. Couturier M. Daoudi K. De Bruyn M. De Cian O. Deschamps F. Dettori F. Dordei L. Douglas K. Dreimanis L. Dufour G. Dujany P. Durante P. -Y. Duval A. Dziurda S. Esen C. Fitzpatrick M. Fontanna M. Frank M. Van Veghel C. Gaspar D. Gerstel Ph. Ghez K. Gizdov V. V. Gligorov E. Govorkova L. A. Granado Cardoso L. Grillo I. Guz F. Hachon J. He D. Hill W. Hu W. Hulsbergen P. Ilten Y. Li C. P. Linn O. Lupton D. Johnson C. R. Jones B. Jost M. Kenzie R. Kopecna P. Koppenburg M. Kreps R. Le Gac R. Lefèvre O. Leroy F. Machefert G. Mancinelli S. Maddrell-Mander J. F. Marchand U. Marconi C. Marin Benito M. Martinelli D. Martinez Santos R. Matev E. Michielin S. Monteil A. Morris M. -N. Minard H. Mohamed M. J. Morello P. Naik S. Neubert N. Neufeld E. Niel A. Pearce P. Perret F. Polci J. Prisciandaro C. Prouve A. Puig Navarro M. Ramos Pernas G. Raven F. Rethore V. Rives Molina P. Robbe G. Sarpis F. Sborzacchi M. Schiller R. Schwemmer B. Sciascia J. Serrano P. Seyfert M. -H. Schune M. Smith A. Solomin M. Sokoloff P. Spradlin M. Stahl S. Stahl B. Storaci S. Stracka M. Szymanski M. Traill A. Usachov S. Valat R. Vazquez Gomez M. Vesterinen B. Voneki M. Wang C. Weisser M. Whitehead M. Williams M. Winn M. Witek Z. Xiang A. Xu Z. Xu H. Yin Y. Zhang Y. Zhou



























roman




EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH (CERN)



pdflatex

        

        


         CERN-LHCb-DP-2019-001 
  
         December 30, 2023
 
         










 Design and performance of the LHCb trigger and full real-time reconstruction in Run 2 of the LHC









Author list on the following page








  The LHCb collaboration has redesigned its trigger to enable the full offline detector reconstruction
  to be performed in real time. Together with the real-time alignment and calibration of the detector, and a
  software infrastructure to make persistent the high-level physics objects produced during real-time processing,
  this redesign enabled the widespread deployment of real-time analysis during Run 2.
  We describe the design of the Run 2 trigger and real-time reconstruction, and present
  data-driven performance measurements for a representative sample of LHCb's physics programme.
  
  








 Submitted to JINST 






 CERN on behalf of the collaboration, licence http://creativecommons.org/licenses/by/4.0/CC-BY-4.0.


















R. Aaij^19,
S. Akar^40,7†,
J. Albrecht^11,
M. Alexander^34,
A. Alfonso Albero^24,
S. Amerio^17,
L. Anderlini^15,
P. d'Argent^12,
A. Baranov^22,
W. Barter^26†,36,
S. Benson^19,
D. Bobulska^34,
T. Boettcher^39,
S. Borghi^37,26,
E. E. Bowen^28†, b,
L. Brarda^26,
C. Burr^37,
J.-P. Cachemiche^7,
M. Calvo Gomez^24,c,
M. Cattaneo^26,
H. Chanal^6,
M. Chapman^30,
M. Chebbi^26†,
M. Chefdeville^5,
P. Ciambrone^16,
J. Cogan^7,
S.-G. Chitic^26,
M. Clemencic^26,
J. Closier^26,
B. Couturier^26,
M. Daoudi^26,
K. De Bruyn^7†,26,
M. De Cian^27,
O. Deschamps^6, 
F. Dettori^35,
F. Dordei^26†,14,
L. Douglas^34,
K. Dreimanis^35,
L. Dufour^19†,26,
G. Dujany^37†,9,
P. Durante^26,
P.-Y. Duval^7,
A. Dziurda^21,
S. Esen^19,
C. Fitzpatrick^27,
M. Fontanna^26,
M. Frank^26,
M. Van Veghel^19,
C. Gaspar^26,
D. Gerstel^7,
Ph. Ghez^5,
K. Gizdov^33,
V.V. Gligorov^9,
E. Govorkova^19,
L.A. Granado Cardoso^26,
L. Grillo^12†, 26†, 37,
I. Guz^26,23,
F. Hachon^7,
J. He^3,
D. Hill^38,
W. Hu^4,
W. Hulsbergen^19,
P. Ilten^29,
Y. Li^8,
C.P. Linn^26†,
O. Lupton^38†,26,
D. Johnson^26,
C.R. Jones^31,
B. Jost^26,
M. Kenzie^26†,31,
R. Kopecna^12,
P. Koppenburg^19,
M. Kreps^32,
R. Le Gac^7,
R. Lefèvre^6,
O. Leroy^7,
F. Machefert^8,
G. Mancinelli^7,
S. Maddrell-Mander^30,
J.F. Marchand^5,
U. Marconi^13,
C. Marin Benito^24†,8,
M. Martinelli^27†,26,
D. Martinez Santos^25,
R. Matev^26,
E. Michielin^17,
S. Monteil^6,
A. Morris^7,
M.-N. Minard^5,
H. Mohamed^26,
M.J. Morello^18,a,
P. Naik^30,
S. Neubert^12,
N. Neufeld^26,
E. Niel^8,
A. Pearce^26,
P. Perret^6,
F. Polci^9,
J. Prisciandaro^25†, 1,
C. Prouve^30†, 25,
A. Puig Navarro^28,
M. Ramos Pernas^25,
G. Raven^20,
F. Rethore^7,
V. Rives Molina^24†,
P. Robbe^8,
G. Sarpis^37,
F. Sborzacchi^16,
M. Schiller^34,
R. Schwemmer^26,
B. Sciascia^16,
J. Serrano^7,
P. Seyfert^26,
M.-H. Schune^8,
M. Smith^36,
A. Solomin^30,d,
M. Sokoloff^40,
P. Spradlin^34,
M. Stahl^12,
S. Stahl^26,
B. Storaci^28†,
S. Stracka^18,
M. Szymanski^3,
M. Traill^34,
A. Usachov^8,
S. Valat^26,
R. Vazquez Gomez^16†,26,
M. Vesterinen^32,
B. Voneki^26†,
M. Wang^2,
C. Weisser^39,
M. Whitehead^26†,10,
M. Williams^39,
M. Winn^8,
M. Witek^21,
Z. Xiang^3,
A. Xu^2,
Z. Xu^27†,5,
H. Yin^4,
Y. Zhang^8,
Y. Zhou^3.



^1Université catholique de Louvain, Louvain, Belgium

^2Center for High Energy Physics, Tsinghua University, Beijing, China

^3University of Chinese Academy of Sciences, Beijing, China

^4Institute of Particle Physics, Central China Normal University, Wuhan, Hubei, China

^5Univ. Grenoble Alpes, Univ. Savoie Mont Blanc, CNRS, IN2P3-LAPP, Annecy, France

^6Clermont Université, Université Blaise Pascal, CNRS/IN2P3, LPC, Clermont-Ferrand, France

^7Aix Marseille Univ, CNRS/IN2P3, CPPM, Marseille, France

^8LAL, Univ. Paris-Sud, CNRS/IN2P3, Université Paris-Saclay, Orsay, France

^9LPNHE, Sorbonne Université, Paris Diderot Sorbonne Paris Cité, CNRS/IN2P3, Paris, France

^10I. Physikalisches Institut, RWTH Aachen University, Aachen, Germany

^11Fakultät Physik, Technische Universität Dortmund, Dortmund, Germany

^12Physikalisches Institut, Ruprecht-Karls-Universität Heidelberg, Heidelberg, Germany

^13INFN Sezione di Bologna, Bologna, Italy

^14INFN Sezione di Cagliari, Monserrato, Italy

^15INFN Sezione di Firenze, Firenze, Italy

^16INFN Laboratori Nazionali di Frascati, Frascati, Italy

^17INFN Sezione di Padova, Padova, Italy

^18INFN Sezione di Pisa, Pisa, Italy

^19Nikhef National Institute for Subatomic Physics, Amsterdam, Netherlands

^20Nikhef National Institute for Subatomic Physics and VU University Amsterdam, Amsterdam, Netherlands

^21Henryk Niewodniczanski Institute of Nuclear Physics  Polish Academy of Sciences, Kraków, Poland

^22Yandex School of Data Analysis, Moscow, Russia

^23Institute for High Energy Physics (IHEP), Protvino, Russia

^24ICCUB, Universitat de Barcelona, Barcelona, Spain

^25Instituto Galego de Física de Altas Enerxías (IGFAE), Universidade de Santiago de Compostela, Santiago de Compostela, Spain

^26European Organization for Nuclear Research (CERN), Geneva, Switzerland

^27Institute of Physics, Ecole Polytechnique  Fédérale de Lausanne (EPFL), Lausanne, Switzerland

^28Physik-Institut, Universität Zürich, Zürich, Switzerland

^29University of Birmingham, Birmingham, United Kingdom

^30H.H. Wills Physics Laboratory, University of Bristol, Bristol, United Kingdom

^31Cavendish Laboratory, University of Cambridge, Cambridge, United Kingdom

^32Department of Physics, University of Warwick, Coventry, United Kingdom

^33School of Physics and Astronomy, University of Edinburgh, Edinburgh, United Kingdom

^34School of Physics and Astronomy, University of Glasgow, Glasgow, United Kingdom

^35Oliver Lodge Laboratory, University of Liverpool, Liverpool, United Kingdom

^36Imperial College London, London, United Kingdom

^37School of Physics and Astronomy, University of Manchester, Manchester, United Kingdom

^38Department of Physics, University of Oxford, Oxford, United Kingdom

^39Massachusetts Institute of Technology, Cambridge, MA, United States

^40University of Cincinnati, Cincinnati, OH, United States

^aScuola Normale Superiore, Pisa, Italy

^bDunnhumby Ltd., Hammersmith, United Kingdom

^cLa Salle, Universitat Ramon Llull, Barcelona, Spain

^dInstitute of Nuclear Physics, Moscow State University (SINP MSU), Moscow, Russia

^† Author was at institute at time work was performed.



 






































































































































plain 

arabic















§ INTRODUCTION

The experiment is a dedicated heavy-flavour physics experiment at the LHC, focused on the
reconstruction of particles containing  and  quarks. During Run 1, the LHCb physics programme
was extended to electroweak, soft QCD and even heavy-ion physics.
This was made possible in large part due to a versatile real-time reconstruction and trigger system,
which is responsible for reducing the rate of collisions saved for
off­line analysis by three orders of magnitude. The trigger used by LHCb in Run 1 <cit.> executed a simplified two-stage
version of the full off­line reconstruction. In the first stage, only charged particles with at least ∼ 1 of transverse
momentum () and displaced from the primary vertex (PV) were available; the threshold was somewhat lower for muons,
which in addition were not required to be displaced. This first stage reconstruction enabled the bunch crossing rate to be reduced efficiently
by roughly one order of magnitude. In the following second stage, most charged particles with ≳ 300
were available to classify the bunch crossings (hereafter “events”). Particle-identification information
and neutral particles such as photons or mesons were available on-demand to specific
classification algorithms. Although this trigger enabled the majority of the LHCb physics
programme, the lack of low-momentum charged particles at the first stage and full particle identification
at the second stage limited the performance for -hadron physics in particular. In addition,
resolution differences between the online and off­line reconstructions made it difficult to precisely understand
absolute trigger efficiencies.


For these reasons, the LHCb trigger system was redesigned during 2013–2015 to
perform the full off­line event reconstruction. The entire data processing framework was
redesigned to enable a single coherent real-time detector alignment and calibration, as well
as real-time analyses using information directly from the trigger system. The key objectives
of this redesign were twofold: firstly, to enable the full off­line reconstruction to run in the trigger, greatly
increasing the efficiency with which charm- and strange-hadron decays could be selected; and
secondly, to achieve the same quality of alignment and calibration within the trigger as was achieved
off­line in Run 1, enabling the final signal selection to be performed at the trigger level. 











arrows.meta,automata, decorations.pathreplacing



	lhcbnode/.style=draw, thick, fill=white,
		align=center, font=,
		rectangle, rounded corners,
		inner sep=3pt,
		minimum width=4.5cm, minimum height=.6cm,





	calibnode/.style=draw, thick, fill=white,
		align=center, font=,
		rectangle, rounded corners,
		inner sep=3pt,
		minimum width=1.8cm, minimum height=.6cm,
	tracknode/.style=draw, thick, fill=white, text=white, 
		align=center, font=,
		rectangle, rounded corners,
		
		
		
		minimum width=6.0cm, minimum height=1.0cm,
	track2node/.style=draw, thick, fill=white, text=white, 
		align=center, font=,
		rectangle, rounded corners,
		
		
		
		minimum width=6.0cm, minimum height=1.5cm,
	startnode/.style=fill=red!10,
	endnode/.style=fill=blue!10,
	greennode/.style=fill=green!10,
	bluenode/.style=fill=rgb,255:red,68; green,114; blue,196, text=white, font=,
	pvnode/.style=fill=rgb,255:red,132; green,180; blue,223, text=white, font=,
	fitnode/.style=fill=rgb,255:red,112; green,173; blue,71, text=white, font=,
	hlt1node/.style=fill=rgb,255:red,237; green,125; blue,49, text=white, font=,
	outputnode/.style=fill=rgb,255:red,255; green,223; blue,128, 
	                   text=rgb,255:red,1; green,25; blue,147, 
			   double,
			   font=,
	outputpvnode/.style=fill=rgb,255:red,255; green,255; blue,215, 
	                     text=rgb,255:red,1; green,25; blue,147, 
			     double,
			     font=,
	orange/.style=fill=orange!60!yellow!70!white, text=brown!60!black,
	rate/.style=right=.1cm, midway,
	mbrate/.style=left=.1cm, midway,
	widearr/.style=ultra thick, -latex,
	bluearr/.style=fill=rgb,255:red,68; green,114; blue,196,
	                draw=rgb,255:red,68; green,114; blue,196,
			line width=1.5mm,
	pvarr/.style=fill=rgb,255:red,132; green,180; blue,223,
	              draw=rgb,255:red,132; green,180; blue,223,
		      line width=1.5mm,
	greenarr/.style=fill=rgb,255:red,112; green,173; blue,71,
	                 draw=rgb,255:red,112; green,173; blue,71,
			 line width=1.5mm,
	hlt1arr/.style=fill=rgb,255:red,237; green,125; blue,49,
	                 draw=rgb,255:red,237; green,125; blue,49,
			 line width=1.5mm,
	grarr/.style=fill=rgb,255:red,192; green,192; blue,192,
	                 draw=rgb,255:red,192; green,192; blue,192,
			 line width=1.5mm,
	darkgreen/.style=fill=green!40!black, text=white, font=, 
	myviolet/.style=fill=violet!60!black, text=white, font=,
	lightblue/.style=fill=blue!30!white




A schematic diagram showing the trigger data flow in Run 2 is depicted in Fig. <ref>.
The LHCb trigger is designed to allow datataking with minimal deadtime at the full LHC bunch crossing rate of 40 MHz.
The maximum rate at which all LHCb subdetectors can be read out is imposed by the bandwidth and frequency of the front-end
electronics, and corresponds to around 1.1 MHz when running at the designed rate of visible interactions per bunch crossing in LHCb of μ=0.4.

During Run 2 LHCb operated at μ=1.1 in order to collect a greater integrated luminosity, which limited
the actual readout rate to about 1 MHz.


A system of field-programmable gate arrays with a
fixed latency of 4 (the trigger) determines which events are kept. Information from the electromagnetic calorimeter,
hadronic calorimeter, and muon stations
is used in separate trigger lines.






The High Level trigger () is divided into two stages, and .
The first level of the software trigger performs an inclusive selection of
events based on one- or two-track signatures, on the presence of muon tracks
displaced from the PVs, or on dimuon combinations in the event.
Events selected by the trigger are buffered to disk storage in the online system.
This is done for two purposes: events can be processed further during inter-fill periods, and
the detector can be calibrated and aligned run-by-run before the stage.
Once the detector is aligned and calibrated, events are passed to , where
a full event reconstruction is performed. This allows for a wide range of inclusive and exclusive
final states to trigger the event and obviates the need for further offline processing. 

This paper describes the design and performance of the Run 2 LHCb trigger system, including the
real-time reconstruction which runs in the . 
The software framework enabling real-time analysis (“TURBO”) has been described in detail elsewhere. The initial 
proof-of-concept deployed in 2015 <cit.> allowed off­line-quality signal candidates selected in the 
trigger to be written to permanent storage. It also allowed physics analysts to use the off­line analysis tools when 
working with these candidates, which was crucial in enabling LHCb to rapidly produce a number of publications 
proving that real-time analysis was possible without losing precision or introducing additional systematics. 
Subsequent developments  <cit.> generalized this approach to allow not only the 
signal candidate but also information about other, related, particles in the event to be saved. 
These developments also transformed the proof-of-concept implementation into a scalable solution which will 
now form the basis of LHCb's upgrade computing model <cit.>.










§ THE LHCB DETECTOR


The detector <cit.> is a single-arm forward
spectrometer covering the  range 2<η <5.
The detector coordinate system is such that z is along the beam line and
x is the direction in which charged particle trajectories are deflected by the magnetic field.

The detector includes a high-precision tracking system
consisting of a silicon-strip vertex detector () surrounding the pp
interaction region <cit.>, a large-area silicon-strip detector (TT) located
upstream of a dipole magnet with a bending power of about
4 Tm, and three stations of silicon-strip detectors (IT) 
 and straw
drift tubes <cit.> () placed downstream of the magnet. These are collectively referred to as the T-stations.
The tracking system provides a measurement of momentum, , of charged particles with
a relative uncertainty that varies from 0.5% at low momentum to 1.0% at 200.
A sketch of the various track types relevant in LHCb is shown in Fig. <ref>.


The minimum distance of a track to a PV,
the impact parameter, is measured with a resolution of (15+[29]/).

Different types of charged hadrons are distinguished using information
from two ring-imaging Cherenkov detectors <cit.>.
Photons, electrons and hadrons are identified by a calorimeter system consisting of
scintillating-pad () and preshower detectors (), an electromagnetic
calorimeter () and a hadronic calorimeter (). Muons are identified by a
system composed of alternating layers of iron and multiwire
proportional chambers () <cit.>.





The detector data taking is divided into fills and runs. A fill is a single period
of collisions delimited by the announcement of stable beam conditions and
the dumping of the beam by the , and typically lasts around twelve hours.
A fill is subdivided into runs, each of which lasts a maximum of one hour. The downtime
associated with run changes is negligible compared to other sources of downtime.

Detector simulation has been used in the tuning of most reconstruction
and selection algorithms discussed in this paper. In simulated LHCb events,
pp collisions are generated using  <cit.>
with a specific configuration <cit.>.  Decays of hadronic particles
are described by  <cit.>, in which final-state
radiation is generated using  <cit.>. The
interaction of the generated particles with the detector, and its response,
are implemented using the toolkit <cit.> as described in
Ref. <cit.>.








§ DATA ACQUISITION AND THE TRIGGER


All trigger systems consist of a set of algorithms that classify events (or parts thereof) as either interesting or uninteresting for further
analysis, so that the data rate can be reduced to a manageable level by keeping only interesting events or interesting parts of them.
It is conventional to refer to a single trigger classification algorithm as a “line”, so that a trigger consists of a set of
trigger lines.



 §.§ Hardware trigger

The energies deposited in the  , , and are used in the -calorimeter system to trigger the selection of events.
All detector components are segmented transverse to the beam axis into cells of different size.
The decision to trigger an event is based on the transverse energy
deposited in clusters of 2×2 cells in the and .
The transverse energy of a cluster is defined as

    =∑_i=1^4 E_i sinθ_i ,

where E_i is the energy deposited in cell i and θ_i is the angle between the z-axis and 

a line from the cell centre to the average pp interaction point (for more details, see Ref. <cit.>).
Additionally, information from the and systems is used to distinguish between hadron, photon and electron candidates.

The -muon trigger searches for straight-line tracks in the five muon stations.
Each muon station is sub-divided into logical pads in the x-y plane.
The pad size scales with the distance to the beam line.
The track direction is used to estimate the of a muon candidate, assuming that the particle originated from the interaction point and received a single kick from the magnetic field.
The resolution of the -muon trigger is about 25 % averaged over the relevant range.
The trigger decision is based on the two muon candidates with the largest :
either the largest must be above the threshold, or
the product of the largest and second largest values must be above the threshold.
In addition there are special trigger lines that select events with low particle multiplicity 
to study central exclusive production and inclusive jet trigger lines for QCD measurements.

To reduce the complexity of events and, hence, to enable a faster reconstruction in the subsequent software stage,
a requirement is placed on the maximum number of hits in most trigger lines.
The trigger accepts a low rate of events, and therefore, only a loose SPD requirement is applied,
while no SPD requirement is applied in the high trigger
used for electroweak production analyses in order to avoid systematic uncertainties associated with the determination
of the corresponding efficiency.
The thresholds used to take the majority of the data are listed in Table <ref> as a function of the year of data taking.
Note that while the use of requirements selects
simpler and faster-to-reconstruct events, it does not result in a significant loss of absolute signal efficiency compared to
a strategy using only and requirements.
This is because the signal-background discrimination deteriorates rapidly with increasing event complexity
for all but the dimuon and electroweak trigger lines. Note that the 2017 thresholds are looser than the 2016 thresholds because the
maximum number of colliding bunches, and hence, the collision rate of the LHC was significantly lower in 2017, due to difficulties with
part of the injection chain. The optimization of the criteria is described in more detail in Sec. <ref>.







 §.§ High level trigger

Events selected by are transferred to the Event Filter Farm (EFF) for further selection.
The EFF consists of approximately 1700 nodes, 800 of which were added for Run 2, with 27000 physical cores.
The EFF can accommodate ≈50000 single-threaded processes using hyper-threading technology.

The is written in the same framework as the software
used in the off­line reconstruction of events for physics analyses.
This allows for off­line software to be easily incorporated into the trigger.
As detailed later, the increased EFF capacity and
improvements in the software allowed the off­line reconstruction to be performed in the in Run 2.

The total disk buffer of the EFF is 10 PB, distributed such that farm nodes with
faster processors get a larger portion of the disk buffer. At an average event size of 55 kB
passing , this buffer allows for up to two weeks of  consecutive data taking before
has to be executed. Therefore, it is large enough to accommodate both regular running (where,
as we will see, the alignment and calibration is completed in a matter of minutes) and to serve
as a safety mechanism to delay processing in case of problems with the detector or calibration.

Around 40% of the trigger output rate is dedicated to inclusive topological trigger lines, another 40% is
dedicated to exclusive -hadron trigger lines, with the rest divided among dimuon lines, trigger lines
for electroweak physics, searches for exotic new particles, and other exclusive trigger lines
for specific analyses. There are in total around 20 and 500 trigger lines.





 §.§ Real-time alignment and calibration



The computing power available in the Run 2 EFF allows for automated
alignment and calibration tasks, providing off­line quality information to the trigger
reconstruction and selections, as described in Ref. <cit.>. A more detailed description
of this real-time alignment and calibration procedure will be the topic of a separate publication.
 





Dedicated samples selected by are used to align and calibrate the detector in real time.
The alignment and calibrations are performed at regular intervals, and




the resulting alignment and calibration constants are updated only if they differ significantly
from the current  values.

The major detector alignment and calibration tasks consist of:


  * the alignment, followed by the alignment of the tracking stations;

  * the alignment;

  * alignment of the rotations around various local axes in both detectors of the primary and secondary mirrors;

  * global time calibration of the ;

  * gas refractive-index calibration;

  * Hybrid Photon Detectors calibration;

  * LED (relative) and π^0 (absolute) calibrations.

Each of these tasks has a dedicated trigger line which supplies it with the types of events required.
When the required sample sizes have been collected,
the selected events are saved to the disk buffer of the EFF, and calibration and alignment tasks are
performed in parallel within the EFF. A schematic view of the alignment and calibration procedure
is shown in Fig. <ref>, together with the time when the tasks are launched and the typical time taken to complete them.

















§ PARTIAL EVENT RECONSTRUCTION



reconstructs the trajectories of charged particles traversing the full tracking system, called long tracks,
with a larger than 500.
In addition, a precise reconstruction of the PV is performed.
 The details of both steps are presented in Sec. <ref>.

Tight timing constraints in mean that most 
particle-identification algorithms cannot be executed.

The exception is muon identification, which due to
a clean signature produced by muons in the detector can be performed
already in ,
as described in Sec. <ref>.
As discussed in Sec. <ref>, a subset of specially selected events 
serves as input to the alignment and calibrations tasks.




 §.§ Track and vertex reconstruction in 



The sequence of algorithms which reconstruct vertices and long tracks is shown in Fig. <ref>.
The pattern recognition deployed in consists of three main steps: reconstructing the tracks,
extrapolating them to the TT stations to form upstream tracks, and finally extending them further to the T stations
to produce long tracks.
Next, the long tracks are fitted using a Kalman Filtering and the fake trajectories are rejected.
The set of fitted tracks is re-used to determine the positions of the PVs.



  §.§.§ Pattern recognition of high-momentum tracks



The hits in the are combined to form straight lines loosely pointing towards the beam line <cit.>.
Next, at least three hits in the are required in a small region around a straight-line extrapolation from the  <cit.> to form so-called upstream tracks.



The is located in the fringe field of the dipole magnet,
which allows the momentum to be determined with a relative resolution of about 20%.
This momentum estimate is used to reject low tracks. Matching
long tracks with hits additionally reduces the number of fake tracks.

Due to the limited acceptance of the ,
tracks pointing to the region around the beampipe do not deposit charge in the ; therefore, they are passed on without requiring hits.







	
















The search window in the and is defined by the maximum possible deflection of charged particles with
larger than 500. The search is also restricted to one side of the straight-line extrapolation
by the charge estimate of the upstream track. The use of the charge estimate is new in Run 2
and enabled the threshold to be lowered from 1.2to 500with respect to Run 1.
For a given slope and position upstream of the magnet and a single hit in the tracking detectors downstream of the magnet, and ,
the momentum is fixed and hits are projected along this trajectory into a common plane.
A search is made for clusters of hits in this plane which are then used to define the final long track <cit.>.
In 2016 two artificial neural nets were implemented to increase the purity and the efficiency of the track candidates in the pattern recognition <cit.>. 






  §.§.§ Track fitting and fake-track rejection


Subsequently, all tracks are fitted with a Kalman filter to obtain the optimal parameter estimate. 
The settings of the online and off­line reconstruction are harmonised in Run 2 to obtain
identical results for a given track. Previously, the online Kalman filter performend only a single iteration which
did not allow the ambiguities from the drift-time measurement of hits on a track to be resolved.
In Run 2 the online fit runs until convergence or maximally 10 iterations.
Furthermore the possibility to remove up to two outliers has been added in the online reconstruction.
The off­line reconstruction is changed to use clusters which are faster to decode
but have less information, and to employ a Kalman filter that utilizes
a simplified geometry description of the detector.  
This significantly speeds up the calculation of material corrections in the filtering procedure.
For Run 2 the calculation of material corrections due to multiple scattering has been improved.
The new description is additive for many small scatterers resulting in more standard normal pull distributions.

The changes made to the off­line Kalman filter enable running the same algorithm in both the and off­line.
These changes neither affect the impact parameter resolution nor the momentum resolution as shown in Sec. <ref>.

Since 2016 the fake track rejection, described in details in Sec. <ref>, has been
used in reducing the rate of events passing this stage by 4%.



  §.§.§ Primary vertex reconstruction


Many analyses require a precise knowledge of the PV position
and this information is used early in the selection of displaced particles.
The full set of tracks is available in . Therefore, the PVs in Run 2 are
reconstructed using tracks only, neglecting the additional momentum information on long tracks
which is only available later. This does not result in a degradation in resolution compared to
using a mixture of and long tracks. Furthermore, this approach produces a consistent PV position
from the beginning to the end of the analysis chain which reduces systematic effects.




As there is no magnetic field in the , the Kalman filter for tracks uses a linear
propagation, allowing for a single scattering at each detector plane, tuned using simulation.
This simplification results in no loss of precision compared to a more detailed material description, but significantly reduces
the amount of time spent in the filtering phase, as no expensive computations are necessary.
A byproduct of this simpler track fit is that the PV covariance matrix is more
accurate than that used off­line in Run 1, with pull distributions more compatible with unit widths in all three dimensions.

The PV resolution is obtained

by randomly splitting the input VELO tracks into two subsets.
The PV algorithm is executed independently on each subset, and 
the PVs found in each subset are matched based on the distance between them.
The width of the distribution of the difference of matched PV positions in a given dimension,
corrected by a factor of √(2), gives the corresponding PV position resolution.

The resolution of the PV reconstruction  for  Run 2
is shown in Fig. <ref>
compared to the Run 1 (2012) off­line reconstruction algorithm.  The new algorithm performs equally well
for the x (y) coordinate, while with respect to Run 1 the resolution on the z coordinate is improved by about 10%.




Additionally, the parameters of the PV reconstruction have been retuned to
give a higher efficiency and smaller fake rate <cit.>.




The resulting improvement in efficiency of reconstructing PVs is 0.5% for PVs associated with the production of a b quark pair,  1.3% for those associated with the production of a c quark pair, and 6.6% for light quarks production.
Simultaneously, the fraction of fake PVs, for example due to material interactions or the decay vertices of
long-lived particles, is reduced from 3.5% to 1%.




 §.§ Muon identification






The muon identification starts with fully fitted tracks.
Hits in the stations are searched for in momentum-dependent regions of interest
around the track extrapolation.
Tracks with p<3 cannot be identified as muons, as they would not be able to reach the stations. 
Below a momentum of 6the muon identification algorithm requires hits in the first two stations after the calorimeters.
Between 6 and 10an additional hit is required in one of the last two stations.
Above 10, hits are required in all the four stations. 
This same algorithm is used in , and off­line.

In , the track reconstruction is only performed for tracks with above 500 .
For particles with lower , a complementary muon-identification algorithm has been devised, 
which is more similar to the muon identification performed in Run 1.
Upstream track segments are extrapolated directly to the stations, where hits are searched for around the track extrapolation.
The regions of interest used in this search are larger than those used for otherwise-reconstructed long tracks.
If hits are found in the muon system, the VELO-TT segment is extrapolated through the magnetic field using the momentum estimate
and matched to hits not already used in the long-track reconstruction.
This procedure extends the muon-identification capabilities down to a of 80for a small additional resource cost,
significantly improving the performance for lower-momentum muons which are important in several measurements <cit.>.




The muon identification code has been reoptimized for Run 2, gaining significant efficiency,
in particular at small . 
This is demonstrated using LHCb simulation in Fig. <ref>. 

The performance of the muon identification in is shown in Fig. <ref> (left)
as determined from unbiased  decays using the tag-and-probe method.
This performance is obtained by studying the efficiency of the single-muon trigger,
which includes requirements on the displacement of the muon and on the minimum momentum (6) and
(1.1).

Analogously in Fig. <ref> (right) the misidentification efficiency
with the same criteria is shown for pions from D^0 decays.
The muon identification efficiency of the single-muon trigger is slightly reduced by
the displacement and (transverse) momentum requirements, at the benefit of a lower misidentification
probability.








§ FULL EVENT RECONSTRUCTION



The full event reconstruction consists of three major steps: the track reconstruction of charged particles, 
the reconstruction of neutral particles, and particle identification (PID).
The track reconstruction exploits the full information from the tracking sub-detectors,
performing additional steps of the pattern recognition which are not possible in due to strict time constraints.
As a result high-quality long and downstream tracks are found with the most precise momentum estimate achievable.
Similarly, the most precise neutral cluster reconstruction algorithms are executed in the reconstruction.
Finally, in addition to the muon identification available in , exploits the full particle identification
from the RICH detectors and calorimeter system.
The code of all reconstruction algorithms has been optimized for Run 2 to better exploit the
capabilities of modern CPUs. Together with the algorithmic changes described in the following sections, this results
in a two times faster execution time while delivering the same or in several cases better physics performance than that
achieved off­line in Run 1.




 §.§ The track reconstruction of charged particles














A sketch of the track and vertex reconstruction sequence in is shown in Fig. <ref>.
The goal is to reconstruct  all tracks without a minimal requirement.
This is particularly important for the study of the decays of lighter particles, such as charmed or strange hadrons, whose abundance
means that only some of the fully reconstructed and exclusively selected final states fit into the available trigger bandwidth.
Often, not all of the decay products of a charm- or strange-hadron decay pass the 500requirement of , particularly for decays into three or more final-state particles. Therefore, to efficiently
trigger these decays, it is necessary to also reconstruct the lower-momentum tracks within the acceptance.


In a first step, the track reconstruction of is repeated. A second step is then used to reconstruct
the lower-momentum tracks which had not been found in due to the kinematic thresholds in the reconstruction. Those tracks
and T-station clusters used to reconstruct long tracks with a good fit quality in the first step are disregarded for this second step.
A similar procedure as in is employed: the remaining tracks are extrapolated through the magnet
to the T-stations using the same algorithm, where the search window is
now defined by the maximal possible deflection of a particle with larger than 80 .
No hits are required for the second step to avoid the loss of track efficiency induced by acceptance gaps in the .
The new Run 2 track finding optimization results in 27% fewer fake tracks and a reconstruction
efficiency gain of 0.5% for long tracks. 
In addition, a standalone search for tracks in the T stations is performed <cit.>,
and these standalone tracks are then combined with tracks to form long tracks <cit.>.
The redundancy of the two long-track algorithms increases the efficiency by a few percent.

Tracks produced in the decays of long-lived particles like or that
decay outside the are reconstructed using T-station segments that are extrapolated backwards
through the magnetic field and combined with hits in the . For Run 2, a new algorithm was used to reconstruct these
tracks<cit.>. It uses two multivariate classifiers, one to reject fakes, and another
to select the final set of hits in the in case several sets are compatible with the same T-station segment. In combination with
other improvements, this results in a higher efficiency and a lower fake rate compared to the corresponding Run 1 algorithm.








The next step in the reconstruction chain is the rejection of fake tracks.
These fakes result from random combinations of hits or a mismatch of track segments upstream and
downstream of the magnet. They are reduced using two techniques. First, all tracks are fitted using the same Kalman
filter. In Run 1, the only selection to reject fake tracks was based on a reduced χ^2.
For Run 2, the upper limit on this χ^2 selection was increased to allow
for a better overall track reconstruction efficiency. To offset the corresponding increase in the number of fake tracks, a neural network
was trained using the TMVA <cit.> package to efficiently remove these tracks<cit.>. Its input variables are the
overall χ^2 of the Kalman filter, the χ^2 values of the fits for the different track
segments, the numbers of hits in the different tracking detectors, and the of the track.

A previous version of the neural network was widely and successfully used in Run 1 to discard fake tracks at the analysis level.
The use of a different set of variables, whose computations are less time consuming, together with optimization of the code made it possible
to deploy this classifier in the Run 2 trigger without any significant impact on the execution time.
The evaluation uses only about 0.2% of the total CPU budget. Furthermore, the better performance  of this fake track rejection in both stages of
the HLT leads to 16% less CPU consumption in the entire software trigger.
The neural network was trained on simulated tracks. The working point was chosen
such that it rejects 60% of all fake tracks, while maintaining an efficiency of about 99%.

The performance of the fake track removal was validated on first collision data in 2015 to ensure a uniform response
over a large area of the phase space. As an example, the
performance for D^0 and π decays is shown in Fig. <ref>.



After the removal of fake tracks, the remaining tracks are filtered to remove so-called clones.
Clones can be created inside a single pattern-recognition algorithm or, more commonly, originate
from the redundancy in the pattern-recognition algorithms.
Two tracks are defined as clones of each other if they share enough
hits in each subdetector. Only the subdetectors where both tracks have hits
are considered. The track with more hits in total is kept and the other is discarded.
This final list of tracks is subsequently used to select events as discussed in
Sec. <ref>.





  §.§.§ Tracking efficiency




The track reconstruction efficiency is determined using a tag-and-probe method on  decays that originate from the decays of -hadrons <cit.>. One muon is reconstructed using the full reconstruction, while the other muon is reconstructed using only specific subdetectors, making it possible to probe the others. For Run 2, the track reconstruction efficiency is determined in using  the data collected by specific trigger lines, see Sec. <ref>.
The performance compared to Run 1 is shown in Fig. <ref>.
Given that the OT has a readout window which is larger than 25and therefore is prone to spillover effects when reducing the bunch spacing from 50to 25, a small reduction in the track reconstruction efficiency is observed in 2015.








  §.§.§ Invariant mass resolution




The invariant mass resolution is determined on a sample of  decays, where the originates from a -hadron decay.
The dimuon invariant mass distribution is modelled using a double Crystal Ball function <cit.>,
where the weighted mean of the standard deviations of the two Gaussian components is used to estimate the resolution.
The distributions for subsamples of the 2012 and 2016 data can be seen in Fig. <ref>,
the resolutions are 12.4 for the 2012 data sample and 12.7 for the 2016 data sample. The difference comes from a slightly higher-momentum spectrum in 2016, due to the larger beam energy in Run 2, and a small degradation in the performance due to the use of a simplified description of the
detector geometry throughout Run 2.







  §.§.§ Impact parameter and decay time resolutions



The impact parameter and decay time resolutions are extracted with data-driven methods
which are described in more detail elsewhere<cit.>.












The impact parameter is defined as the distance between a particle trajectory
and a given PV. It is one of the main discriminants between particles produced directly in the primary
interaction and particles originating from the decays of long-lived hadrons.
The impact parameter resolution as a function of 1/ is shown
in Fig <ref>.
Only events with one reconstructed PV are used, and the PV fit is rerun excluding each track in turn.
The resulting PV is required to have at least 25 tracks to minimise the contribution from the PV resolution.
Multiple scattering induces a linear dependence on 1/.
For high particles, the impact parameter resolution is roughly 12in both the x and y directions.
The observed improvement of about 1in 2017 data taking is due to
the use of an updated error parametrisation.




The decay time of a particle is determined from the distance between the PV and the secondary decay vertex.
An excellent decay time resolution is a key ingredient of time-dependent mixing and violation measurements.
The resolution is determined from decays combined with two random tracks which mimic ϕ decays.
In the absence of any impact parameter requirements these combinations come mainly from prompt particles
and, therefore, the expected decay time is zero. The width of the distribution is thus
a measure of the decay time resolution. A comparison of the decay time resolution as a function of momentum for Run 1, 2015, and 2016 data taking is shown in Fig. <ref>. For Run 2  the average resolution is about 45for a 4-track vertex.































 §.§ Muon reconstruction



As mentioned in Sec. <ref>, the same muon identification algorithm is used
in and , apart from the fact that the algorithm takes as its input the full set of fitted tracks
available after the reconstruction.




 §.§ RICH reconstruction


The identification of different particle species
is crucial across LHCb's physics programme.



The detectors provide the main
discrimination between deuterons, kaons, pions, and protons.
Cherenkov light is emitted in a cone around the flight direction
of a charged particle, where the cone width depends on the velocity of the particle.
The photon yields, expected Cherenkov angles, and estimates of the per-track Cherenkov angle resolution
are computed under each of the deuteron, proton, kaon, pion, muon and electron mass hypotheses.
The reconstruction considers simultaneously all reconstructed tracks and
all Cherenkov photons in and in each event. The reconstruction algorithm
provides a likelihood for each mass hypothesis.
As the reconstruction consumes significant computing power, it could not be run for every track
in the Run 1 real-time reconstruction. Improvements in the and in the reconstruction itself made
it possible, however, to run the full algorithm in the Run 2 .
The performance of the particle identification is shown in Fig. <ref> for the 2012 and 2016 data.
A small improvement is obtained in Run 2 for particles below 15of momentum.





 §.§ Calorimeter reconstruction

The reconstruction of electromagnetic particles (photons, electrons, and mesons) is performed by the calorimeters.
A cellular automaton algorithm 
is used to build clusters from the energy deposits in the different calorimeter subsystems,
which are combined to determine the total energy of each particle <cit.>.
Neutral particles are then identified according to their isolation with respect to the reconstructed tracks.
Electron identification is also provided by combining information from the isolation of clusters in the , the presence of clusters in the , the energy deposited in the , and the position of possible Bremsstrahlung photons.

High-E_T mesons and photons are indistinguishable at the trigger level, as they both appear as a single cluster,
while low-E_T mesons are built by combining resolved pairs of well-separated photons.
The neutral-cluster reconstruction algorithm run in is the same as that run off­line.

The identification of these clusters as either neutral objects or electrons uses information from both the PS/SPD detectors,
and a matching between reconstructed tracks and calorimeter clusters. Early in Run 2 this online identification was not identical to the off­line
version because the HLT did not reconstruct T-tracks (see Fig. <ref>), since these are not
directly used by physics analyses. They are, however, relevant for neutral-particle identification. This misalignment
was gradually reduced as Run 2 progressed, first by adding the reconstruction of T-tracks and then by subsequently applying a Kalman filter to them

to align the algorithm to the off­line reconstruction sequence.

A fully automated calibration was introduced in 2018.
The automatic LED calibration is performed for fills longer than 3.5 hours
as indicated in Fig. <ref>, while the absolute π^0 calibration
is processed once per month when sufficient data (amounting to 300M events) is collected. 
The performance of the calorimeter reconstruction is shown in Fig. <ref>
using B^0 → (K^+ π^-) γ  decays.
The invariant mass resolution has been improved with respect to Run 1 from about 91to
87.











§ TRIGGER PERFORMANCE


The LHCb trigger performance is optimized around two key metrics: the kinematic and occupancy thresholds for
each of the main trigger lines (muon, dimuon, electron, photon, and hadron); and the optimization of the timing
budget, which defines the maximum allowed output rate. An automated procedure is used to divide the bandwidth among a set
of representative signal channels. It has been significantly improved with respect to Run 1 and is described here.
The procedure for determining the processing budget is also described, and the and performance
is evaluated using a tag-and-probe approach on Run 2 data.









 §.§ bandwidth division

The relative simplicity of the information available for the trigger decision means that the trigger lines listed in Table <ref> cover the majority of the LHCb physics programme. Out of these, the high-muon trigger consumes a relatively negligible rate and is insensitive to the running conditions. The remaining five trigger lines: hadron, muon, electron, photon and dimuon, must have their and thresholds tuned to maximize signal efficiency under different LHC conditions. In particular, during the luminosity ramp-up of the LHC, signal efficiencies can be improved by reducing the thresholds to maintain an output rate close to the maximal readout rate of 1 MHz.

Once the LHC reaches its nominal number of colliding bunches in a given year, determining the optimal division of rate between the channels is important for achieving the physics goals of the experiment. This
so-called “bandwidth division” is performed using a genetic algorithm to minimise the following pseudo-χ^2 for a broad range of simulated signal samples that are representative of the LHCb physics programme:
	
    χ^2(r) = ∑^N_iw_i× ( 1 - ε(r)_i/ε(r)^max_i )^2  .

The sum is over the N signal samples, ε(r)_i is the efficiency including detector dead time of the i^th data set, and ε^max is the efficiency including detector dead time when all of the bandwidth is allocated to this data set.
The ratio of dead-time-corrected efficiencies is designed to ensure that inefficient signal samples contribute more to the χ^2, i.e. the algorithm prioritizes improving the efficiency of signal samples which start with a low absolute efficiency over making identical absolute efficiency improvements for signals with high efficiencies to begin with. The weight assigned to each data set, w_i, is predetermined by the LHCb collaboration and is designed to grant more bandwidth to higher-priority physics channels.

The dead-time correction to the signal efficiency acts as a rate limiter and is dependent upon the filling scheme: 

    ε(r) = ϵ× [1-δ^phys(r) ] × [1-δ^tech(r) ].

Here ϵ is the overall signal efficiency and r is the retention of collisions collected using random trigger lines
(henceforth “nobias”). The physics dead time δ^phys(r) is zero below r_limit=1.1 MHz, which is the maximum HLT1 throughput, and r/r_limit above this. The technical dead time, δ^tech(r) is determined from a model trained on a filling-scheme dependent emulation of the detector readout dead time.


The results of the bandwidth optimization are shown in Fig. <ref> for the 2016 and 2017 data-taking conditions.

The different optimal points are mostly connected to the different LHC running conditions in the two years,
in particular to problems in 2017 
which limited the maximum possible number of bunches in the
LHC and hence led to lower trigger thresholds.



 §.§ Measuring the processing speed

Trigger configurations are tested for processing speed and memory usage on 13nobias
collected at the same average number of visible interactions per bunch crossing as in regular data taking.
As the trigger conditions affect the complexity of events
processed by the , these tests are repeated for each configuration. Nobias events passing the configuration
are processed by a dedicated “average” EFF node loaded with the same number of total tasks as in the online data-taking configuration.
The timing is measured separately for running on events passing , and running on events passing both and .
Each and task processes an independent sample of around 10,000 events during this test, with the number of
events chosen to balance robustness and turnaround speed.
The processing speed of each of the individual and tasks is then averaged in order to remove fluctuations due to the limited number of test events, and these values are compared to the available budget.
In addition, the memory usage is plotted as a function of the event number to verify that there are no memory leaks.

These tests give confidence that the is running within its budget, and they are particularly important
for spotting problems after any major changes are made to a stable configuration.
They do not, however, give a perfect reflection of the performance expected on the full farm. In
particular, the effect of calibration and alignment tasks which run in parallel with the and tasks is neglected,
as are the I/O issues associated with sending events from to the tasks, the buffering of events
and the action of reading them back into tasks,
and the overhead from sending the events accepted by to the off­line storage. For this reason it makes little sense to quote detailed
performance numbers for the from these tests.









 §.§ Optimization of the timing and disk buffers

The timing budget of the is defined as the average time available for each task to process an event, when
the processing farm is fully loaded.[The processing farm consists of processors with a certain number of physical cores,
but a single task will not generally fully load a single physical core. For this reason the number of logical tasks which
are launched for each physical CPU core is optimized by measuring the overall throughput of events in the farm. For the 2015
farm, this number is typically around 1.6, depending on the node in question.]
With a traditional single-stage , as was the case in Run 1, the timing budget is easily determined because the events must always
be processed as they arrive during the collider runtime.  Therefore, it is simply the number of tasks divided by the input
event rate, which amounted to about 50 ms for a farm with around 50000 logical cores as was available in 2015.
The calculation is more complicated for the two-stage used in Run 2. Since the
second stage is deferred and can occur during LHC downtime there are two timing budgets, one for
the stage and one for the stage. These budgets depend on the assumptions made about the length
and distribution of the LHC runtime and downtime periods.








The LHC downtime is not uniformly distributed throughout the year. Most occurs
during a winter shutdown, lasting several months, and “technical stops” lasting approximately two weeks each and distributed throughout the year.
The runtime is consequently also concentrated, with a peak structure
of repeated 10–15 hour-long collision periods with inter-fill gaps of 2–3 hours between them. The timing budget is determined by simulating
the rate at which the disk buffer fills up and empties, using the processing speed measurements and the
most recent LHC fill structure as a guide.[In 2015 this optimization used the 2012 fill information.] The objective is to ensure
that the disk buffer will never exceed more than 80% capacity at any point throughout the remaining data taking period, and is evaluated every two weeks using the actual disk occupancy at the time as the starting point. 


The output rate of is adjusted to keep the disk buffer usage within the desired limits. This output rate is controlled by switching
between two configurations, where the tighter configuration sacrifices some rate and efficiency
for the inclusive general purpose trigger lines while protecting the trigger lines used for specific areas of the physics
programme.
The buffer usage is monitored throughout the year, and biweekly simulations are made using the present buffer capacity, output rate and throughput to determine the projected disk usage until the end of the year. Should a significant fraction of these simulations exceed the 80% usage threshold, the configuration is tightened. An example simulation and the disk usage throughout 2017 are shown in Fig. <ref>.





 


















 §.§ Efficiency measurement method


All efficiencies are measured on background-subtracted data using the so-called TISTOS method
described in the Run 1 performance paper <cit.> and
briefly recapped here. Off­line-selected signal events are divided into the following categories:


  * TIS: Events which are triggered independently of the presence of the signal decay. These are
unbiased by the trigger selection except for correlations between the signal decay and the rest of the event. (For example when triggering
on the “other” B in the event and subsequently looking at the momentum distribution of the “signal” B,
the correlation in their momenta is caused by the fact that they both originate in the same fragmentation chain.)

  * TOS: Events which are triggered on the signal decay independently of the presence of the rest of the event.

All efficiencies quoted in this paper are TOS efficiencies, given by

    ϵ = N(TOS and TIS)/N(TIS),

where N(TIS) is the number of signal TIS events in the sample, while N(TOS and TIS) is the number of signal events which are both
TOS and TIS. The number of signal events passing and failing the TOS criterion is measured using a histogram sideband subtraction,
as described in Ref. <cit.>.
In order to reduce the correlations between TOS and TIS events, 
the efficiency is plotted as a function of the and, where appropriate, decay time of the signal particle.  




 §.§ Samples used for performance measurements

The performance of the and trigger selections is evaluated using samples of trigger-unbiased signals collected during Run 2,
shown in Figs. <ref> and <ref>. The signal channels are representative of the LHCb physics programme, and
are selected using relatively loose criteria on the kinematics and displacement of the -hadron and the final-state particles.

As the -hadron signals are all fully selected by exclusive TURBO trigger lines, only their and efficiencies
can be measured. The -hadron signals are selected by off­line selections without imposing any trigger requirements,
and therefore, they can be used to measure the , , and efficiencies. The exception is the →γ decay,
where requiring TIS at or results in a signal yield and purity which are too small to be usable.  Therefore, this mode is
only used to study the photon and electron trigger performance.









 §.§ performance






The efficiencies of the trigger lines in Run 2 are shown in Fig. <ref> for -hadrons and Fig. <ref>
for -hadrons, respectively, as a function of the and the data-taking period. 
The is optimized to fill the available ≈ 1 MHz bandwidth for a
given set of LHC running conditions, and so the efficiency evolves as a function of those conditions. In particular,
if the LHC has to run with a reduced number of colliding bunches, the required rejection factor to reach 1 MHz output rate
is smaller and the criteria can be correspondingly loosened, which is the cause of the jumps visible in the bottom plots.
In the case of the →γ decay, the low efficiency of the dedicated photon trigger is due to the limited
information available to separate electrons and photons within the system. This identification relies on the
amount of electromagnetic showering observed in the preshower detector, before the photons or electrons reach the , 
and whether there is a hit or not in the detector.







The chosen working point is such that the photon trigger has a high purity but relatively low efficiency. However, many genuine photons are selected by the electron trigger, which is also efficient for photons.
In addiction, a significant amount of photons convert in the detector material between the magnet and the plane. 
These converted photons are reconstructed as neutral clusters off­line, but leave a hit in the detector and are therefore triggered
as electrons.
In practice the →γ signal is selected using both electron and photon trigger lines to account for these effects.


The efficiency of each trigger is measured with respect to events where the corresponding criterion from Table <ref> has
already been applied. 
The distribution of hits for → signal candidates in Run 2 data is shown in Fig. <ref>,
and is representative of typical
heavy-flavour signals in LHCb. The efficiency of the thresholds is  generally around 90% for the and 50% for
the other heavy-flavour trigger lines.
The advantage of these SPD requirements is that they allow looser kinematic thresholds.




























The trigger efficiencies as functions of the hadron and η are shown in Fig. <ref>,
except for the photon trigger where the signal yields are too small.
The efficiency is relatively flat in η for any given bin, although the calorimeter-based trigger lines do have
a slightly better efficiency at high pseudorapidities.







 §.§ performance

The trigger stage processes approximately 1 MHz of events that pass the trigger, and reduces the event rate to around 110 kHz, which
are further processed by . The reconstruction sequence was described in Sec. <ref>, while this section describes
the performance of the trigger lines.



  §.§.§ Inclusive lines


has two inclusive trigger lines which select events containing a particle whose decay vertex is displaced from the PV:
a line which selects a single displaced track with high , and a line which selects
a displaced two-track vertex with high . The single track line is a reoptimization of the Run 1 inclusive single track
trigger <cit.>, while the displaced two-track vertex trigger is a new development for Run 2.
Both lines start by selecting good quality tracks that are inconsistent with originating from the PV.
The single-track trigger then selects events based on a hyperbolic requirement in the 2D plane of the track displacement
and .[More complicated multivariate selection criteria, for example boosted decision trees using track quality
information in addition to the displacement and , were tried but gave no significant increase in performance.] The
two-track displaced vertex trigger selects events based on a MatrixNet classifier <cit.> whose input variables are the
vertex-fit quality, the vertex displacement, the scalar sum of the of the two tracks, and the displacement of the
tracks making up the vertex.

These trigger lines were primarily optimized for inclusively selecting the decays of  and  hadrons, and were trained using 26 different
- and -hadron decays in order to make them as efficient as possible on the full spectrum of possible decay topologies. Care was
taken, however, to make sure that these trigger lines would also be efficient for more exotic displaced signatures, for example hypothetical
supersymmetric particles.

The performance of these trigger lines is shown in Figs. <ref> and <ref>.


The two-track line is more efficienct at low , whereas the single track line performs best at high ,
such that combined they provide high efficiency over the full range.








Unlike the trigger configurations, which changed frequently in response to varying LHC conditions, the trigger configuration
was kept largely stable, with some updates at the end of each data-taking year.
The variation in the total efficiency as a function of the data-taking
period is shown in Fig. <ref>. The -hadron efficiencies have been stable throughout the Run 2 data taking.
The -hadron efficiencies decreased midway through 2016, when a tighter configuration was used to prevent the disk buffer from
overflowing due to unexpectedly high LHC efficiency and availability. The improvements seen for
some of the -hadron channels in 2017 with respect to 2016 are caused by changes in the corresponding reference
off­line selections leading to a different average and displacement of the -hadron, not any intrinsic variation in the performance or thresholds.








  §.§.§ Muon lines




The muon lines select muonic decays of  and  hadrons, as well muons originating from decays of and bosons.
As muons have an intrinsically cleaner signature than hadrons, the muon lines make use of simple rectangular
selection criteria as opposed to the multivariate inclusive lines.
There are four primary lines: one line that selects a single displaced muon with high for flavour physics;
a second single muon line that selects very high muons, without displacement criteria, for electroweak physics;
a third line that selects a dimuon pair compatible with originating from the decay of a charmonium or bottomonium resonance, or from Drell-Yan production; and a fourth line
that selects displaced dimuons with no requirement on the dimuon mass.
The efficiencies of the lines relevant for -hadron decays are shown in Fig. <ref>
as obtained from data with the TISTOS method.
Note that because these muon trigger lines only run on events selected by and trigger lines, their absolute efficiency is
lower than that of the inclusive single-track trigger, which runs on all -selected events.
In addition to these lines, for Run 2 a new line dedicated to lower-dimuons
has been developed which has tighter criteria on the displacement of the dimuon but runs on all -selected events,
rather than just the muon ones. This line is particularly important for selecting rare decays of strange hadrons, that are not triggered by the muon lines,
increasing their efficiency up to a factor three <cit.>.






  §.§.§ Calibration trigger lines



contains two primary types of calibration trigger lines: a line which selects D^0 → K^- π^+ candidates with significant displacement
from the PV, and a line which selects J/ψ→μ^+ μ^- candidates. The former is used for providing a pure sample of
good tracks (the D^0 decay products) for the alignment of the tracking system, while the latter is used to provide
a pure sample of muons for the alignment
of the muon chambers. In addition, other trigger lines select events enriched in off-axis tracks or tracks which populate the lower-occupancy
regions of the detectors, for use in the and alignment, respectively. The purity and yield of the calibration trigger lines is illustrated
in Fig. <ref>, which shows the D^0 and J/ψ candidates reconstructed online in their respective lines for a specific fill
corresponding to approximately 18.5 pb^-1 of integrated luminosity.































  §.§.§ Low multiplicity event and exclusive trigger lines

Special trigger lines for low-multiplicity events are needed to enable the study of central exclusive production (CEP). This kind of process takes place by
colourless, low-t-channel exchange between protons and can result in particle production in the central rapidity region. The protons remain intact
and are deflected only slightly, so such production is typically accompanied by large ranges of rapidity with little detector activity, known as “rapidity gaps”.
The trigger development initially focussed  on acquiring large samples of exclusively-produced dimuon candidates, but evolved to cover final states
involving hadrons and calorimeter objects.

Since low levels of activity are anticipated for CEP, events with more than 30 hits in the are rejected at the
hardware level. Lower-bounds are also placed on relevant detector activity measurements as appropriate for each final state. 

These criteria indirectly favour the selection of events with exactly one pp interaction, as opposed to either multiple- or zero-interaction events.





At the stage, the low-multiplicity events containing muons or electromagnetic calorimeter objects occur at a low enough rate
that can be selected with no additional requirements. 

but low-multiplicity events containing hadrons are required to have at least two tracks reconstructed in the .



In addition, the low thresholds implemented in the Run 2 tracking allowed several special exclusive trigger selections to
be implemented for the first time, notably trigger lines that select two-body beauty and charm hadron decays without biasing their decay times <cit.>.



In 2018 the HeRSCheL detector <cit.> is employed in the selection of CEP events,
allowing for a reduction of the thresholds.



  §.§.§ bandwidth division

The bandwidth is preferentially allocated to the inclusive and muon trigger lines which, by selecting - and - hadron decays, cover most
of the physics programme. The large disk buffer available in Run 2 also makes it possible to allow generous rates for other
trigger lines, however, with a total output rate of 150 kHz which is around two times the Run 1 average. The rates
and the overlaps in the events selected by the different trigger lines are shown in Fig. <ref>.





 §.§ performance

The trigger stage reduces the event rate to around 12.5 kHz, at which point the remaining events
are saved to permanent storage for further analysis. The reconstruction sequence was described in Sec. <ref>,
while this section describes the performance of a representative set of trigger lines.



  §.§.§ Inclusive -hadron trigger lines

The inclusive -hadron trigger lines look for a two-, three-, or four-track vertex with sizable , significant displacement from
the PV, and a topology compatible with the decay of a  hadron. As in Run 1 <cit.>,
these “topological” trigger lines rely on a multivariate selection of the displaced vertex. This selection is implemented in a
MatrixNet classifier whose inputs have been discretized <cit.> in order to minimize the variation in selection performance
with varying detector conditions and speed up the evaluation time. The efficiency of the topological trigger lines is increased
for decays involving muons by relaxing the requirement on the multivariate discriminant whenever one or more of the tracks associated
with the topological vertex is positively identified as a muon or electron.

The topological trigger lines are trained to separate signal -hadron decays
which can be fully reconstructed inside the detector acceptance from those which cannot, as well as from
displaced vertices formed from the decays of  hadrons originating from the PV. The displaced
vertices from  hadrons are the most numerous background.
Harder to discriminate against, however, are the backgrounds from -hadron decays that are only 
partially contained in the detector acceptance, or -hadron
decays in which much of the energy is taken by neutral particles. 

The selection has been reoptimized <cit.> for Run 2, taking advantage of the full off­line reconstruction
now available in to
loosen the selection criteria when building vertex candidates, and fully relying on the multivariate algorithm to discriminate
between signal and background. The resulting efficiencies

are shown in Fig. <ref> and 
Fig. <ref> for a specific decay mode, 
while the evolution of the efficiency as a function of the data-taking conditions is shown in Fig. <ref>.































  §.§.§ Muon and dimuon trigger lines


The muon and dimuon trigger lines select a wide spectrum of signals: low-mass Drell–Yan dimuons for electroweak physics,
dimuons originating from the PV for production measurements, dimuons with displacement from the PV for the study
of -, -, and -hadron decays and heavy dimuons for exotic particle searches and electroweak physics.
As mentioned in Sec. <ref>, in Run 2 the and off­line muon-identification procedures are identical.

Owing to this improvement and because muons provide a relatively rare and clean event signature,
the dimuon trigger lines generally have a  high efficiency which is only limited in some cases
by the rate of the selected signal, most notably for production measurements.
This is illustrated in Fig. <ref> where the efficiency of the muon trigger lines is shown for B^+→ decays.
Note that the muon topological trigger lines have a lower absolute efficiency compared to the hadron topological trigger lines because they only
process events passing the single-muon selection.
In addition to the standard inclusive muon lines used in Run 1, for Run 2 new lines have been developed in particular for
dimuons with lower for exotic particle searches (dark photons) and for rare strange-hadron decays <cit.>.




























  §.§.§ Exclusive and calibration trigger lines



In addition to the inclusive trigger lines, the full off­line reconstruction performed at the start of means that it is possible to fully reconstruct certain decays of interest and select them using dedicated trigger lines without
any loss in efficiency compared to the off­line analysis. This is especially important for -hadron trigger lines because around 10% of all
13 TeV proton-proton collisions produce a cc̅ pair, and it is not possible to write all -hadron signals to off­line storage.
In order to reduce the necessary disk space, the exclusive -hadron trigger lines make extensive use of the TURBO stream. 
All events selected by those trigger lines, except those containing neutral particles, are sent to the TURBO stream.
The selection criteria of these trigger lines are usually a slightly looser version of those used
in the off­line analysis, enabling the candidates saved in the TURBO stream to be directly used by the analysts.
In total, over 200 different
exclusive trigger lines which select the decays of  hadrons are deployed in Run 2. They are generally tuned to have S/B ratios well in excess of 1 already at
the output of the trigger, with the final selection performed off­line using information reconstructed in the trigger and tuned to minimize systematic uncertainties.
The purity achievable using the trigger-level information has already been illustrated in Fig. <ref> for a representative sample of -hadron decays.

In addition, contains a suite of calibration trigger lines, which are used to measure the performance of
the track-finding and particle-identification algorithms in a data-driven way. These trigger lines select
high-yield charm, charmonium, and  decays using a tag-and-probe
approach, where the probe particle is kept unbiased with respect to either the tracking or particle-identification information. There are around 50 such lines
in total, and they select around 500 Hz of calibration signals.













  §.§.§ Low multiplicity event trigger lines

At the stage there are dedicated selections for each relevant final state with a low track multiplicity.
There are 32 lines: two to select exclusive dimuon production, three to select exclusive production of photons or electrons, and the remainder
to select various hadronic final states, dominated by lines that select low-hadrons.


The trigger efficiencies have been determined in data and are shown in Fig. <ref> for two channels of particular interest: dimuon and dihadron. The dimuon trigger efficiency is determined using a sample of independently triggered candidates reconstructed in events containing exactly two muon tracks inside the detector acceptance. The dimuon candidate is required to have satisfied the relevant low-multiplicity trigger. The efficiency is shown as a function of dimuon mass, where the rise at 800results from the 400requirement for each muon. In the case of exclusive production, where the candidate is expected to be produced with low , this leads to an implicit lower bound on the mass of the exclusively-produced object at m()≈ 800. The non-zero efficiency for candidates with m() ≲ 800 arises from candidates with higher .

The dihadron trigger efficiency, which includes the effect of a 50% prescale, is determined using ϕ(1020)→ candidates reconstructed in low-multiplicity events and triggered independently of the signal candidate. The ϕ(1020) candidate is required to pass the relevant low-multiplicity and trigger lines, and the background from misidentified pions is reduced using information from the RICH sub-detectors. The efficiency is shown as a function of the of the ϕ(1020) meson.






  §.§.§ bandwidth division

The bandwidth is divided into the full stream, containing inclusive trigger lines, and the TURBO stream, which contains exclusive
trigger lines that fully reconstruct relevant decays. Most of the full stream rate is taken up by the topological -hadron, inclusive -hadron, and
dimuon trigger lines, while the TURBO stream rate is divided among several hundred exclusive -hadron trigger lines. As the TURBO stream
trigger lines perform a full selection of high-purity signals, their rates are generally proportional to the signal abundance. The rates
and the overlaps in the events selected by the different trigger lines are shown in Fig. <ref>, where the exclusive trigger lines are
counted as one item for brevity.







§ CONCLUSIONS

The design and performance of the LHCb Run 2 reconstruction and High Level Trigger have been presented.
The use of real-time alignment and calibration and improvements in the reconstruction software allows for events to be fully reconstructed in the High Level Trigger with equivalent quality to the Run 1 off­line performance, and enables signals
to be selected with a purity close to that achievable off­line.
This in turn enables physics analysis to be performed
directly with the output of the reconstruction in the trigger. To this end, a significant fraction of triggered
events is saved in a reduced “real-time analysis” format, 
saving only higher-level reconstructed objects relevant to physics analysis and not the full raw detector data.


The successful deployment of this full real-time reconstruction and analysis during Run 2 is a critical
stepping stone towards the LHCb upgrade, whose software trigger will have to deal with roughly 100 times greater
data rates while maintaining a high acceptance over the same broad range of physics channels.











§ ACKNOWLEDGEMENTS

We express our gratitude to our colleagues in the CERN
accelerator departments for the excellent performance of the LHC. We
thank the technical and administrative staff at the LHCb
institutes.
We acknowledge support from CERN and from the national agencies:
CAPES, CNPq, FAPERJ and FINEP (Brazil); 
MOST and NSFC (China); 
CNRS/IN2P3 (France); 
BMBF, DFG and MPG (Germany); 
INFN (Italy); 
NWO (Netherlands); 
MNiSW and NCN (Poland); 
MEN/IFA (Romania); 
MSHE (Russia); 
MinECo (Spain); 
SNSF and SER (Switzerland); 
NASU (Ukraine); 
STFC (United Kingdom); 
NSF (USA).
We acknowledge the computing resources that are provided by CERN, IN2P3
(France), KIT and DESY (Germany), INFN (Italy), SURF (Netherlands),
PIC (Spain), GridPP (United Kingdom), RRCKI and Yandex
LLC (Russia), CSCS (Switzerland), IFIN-HH (Romania), CBPF (Brazil),
PL-GRID (Poland) and OSC (USA).
We are indebted to the communities behind the multiple open-source
software packages on which we depend.
Individual groups or members have received support from
AvH Foundation (Germany);
EPLANET, Marie Skłodowska-Curie Actions and ERC (European Union);
ANR, Labex P2IO and OCEVU, and Région Auvergne-Rhône-Alpes (France);
Key Research Program of Frontier Sciences of CAS, CAS PIFI, and the Thousand Talents Program (China);
RFBR, RSF and Yandex LLC (Russia);
GVA, XuntaGal and GENCAT (Spain);
the Royal Society
and the Leverhulme Trust (United Kingdom);
Laboratory Directed Research and Development program of LANL (USA).





tocsectionReferences
inbibliographytrue
LHCb



