The LHCb DAQ and Trigger Systems: recent updates

Ricardo GracianiXXXIV International Meeting on Fundamental

Physics.

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

2

Overview

•LHCb•The Original Design•Trigger Rate:

–From 200 Hz to 2000 Hz•Level 1 suppression:

–1 MHz Readout•Outlook

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

3

LHCb

• LHC:

– 40 MHz crossing rate– 30 MHz with bunches from both directions

• Luminosity: 2·1032 cm-2 s-1

– 10 to 50 times lower than @ ATLAS, CMS

• One Arm Forward Spectrometer

• LHCb rates:(for visible events at least 2 tracks in acceptance)

– Total rate (minimum bias): 10 MHz– bb: ~100kHz

• Whole decay of one B in acceptance: 15kHz– cc: ~600kHz

pp interactions/crossing

B (rad) B (rad)

Production angle Of B vs B

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

4

The Spectrometer

p p

250 mrad

10 mrad

Muon system

Vertex Locator

RICH detectors

Tracking systemDipole magnet

Calorimeters

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

5

LHCb Physics Program

•Bs oscillation frequency, phase and ΔΓs

– BsDs, J/ΨΦ, J/Ψη, ηcΦ from Bd0–+

with BdJ/KS as a proof of principle– And from bs penguin

in various channels, differing sensitivity to new physics:– Time-dependent CP asymmetry of BsDs

K and DsK

– Time dependent CP asymmetries of Bd and Bs KK – Comparison of decay rates in the BdD0(K+π-,K-π+,K+K-)K*0 system– Comparison of decay rates in the B-D0(K+π-,K+π-π+π-)K- system– Dalitz analysis of B- D0(KSπ-π+)K- and Bd

D0(KSπ-π+)K*0

•Rare decays– Radiative penguin Bd K* , Bs Φ , Bd – Electroweak penguin Bd K*0

– Gluonic penguin Bs ΦΦ, Bd ΦKs

– Rare box diagram Bs

•Bc , b-baryon physics + unexpected !

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

6

The LHCb TP

HLT

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

7

From 200 to 2000 Hz

• Initial Design:– 200 Hz of partial or fully reconstructed B final States. – Exclusive B Trigger.

• Limitations:– Strong MC dependence:

• Efficiency determination• Detector Performance (vertex, proper time, PID)• Backgrounds

– Huge MC samples are necessary.• Conclusion:

– Review strategy• Use real Data for the above studies

– Re-evaluation of Computing needs• Factor 10 reduction on MC needs• Factor 10 increase on Trigger rate

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

8

2 kHz Trigger Output

• Best guess at present (split between streams still to be determined)

• Exclusive reconstruction is kept• Large inclusive streams to be used to control

calibration and systematics (trigger, tracking, PID, tagging,…)

• 10-15% acceptance for any channel on inclusive samples.

Output rate Trigger Type Physics Use

200 Hz Exclusive B candidates Specific final states

600 Hz High Mass di-muons J/, bJ/X

300 Hz D* Candidates Charm, calibrations

900 Hz Inclusive b (e.g. b) B data mining

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

9

Cost Estimate

200Hz 2000Hz

Hoffman2007

Now2008

CERN

CPU(MSI2k) 2.0 0.9

Disk(PB) 0.3 0.8

Tape(PB) 1.2 1.4

Tier-1’s

CPU(MSI2k) 8.3 4.4

Disk(PB) 1.6 2.4

Tape(PB) 0.75 2.1

Tier-2’sCPU(MSI2k) - 7.6

Disk(PB) - 0.02

Relative Cost 1.0 0.91

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

10

DAQ Architecture

StorageSystem

TIER0

Scalable in depth: more CPUs (<2200)Scalable in width: more detectors in Level-1

Multiplexing Layer

FE FE FE FE FE FE FE FE FE FE FE FE

Switch Switch

Level-1Traffic

HLTTraffic

1000 kHz5.5 GB/s

40 kHz1.6 GB/s

94 SFCs

Front-end Electronics

7.1 GB/s

TRM

Sorter

L1-DecisionReadout Network

Switch Switch Switch

SFC

Switch

CPU

CPU

CPU

SFC

Switch

CPU

CPU

CPU

SFC

Switch

CPU

CPU

CPU

SFC

Switch

CPU

CPU

CPU

SFC

Switch

CPU

CPU

CPU

CPUFarm

~1600 CPUs

~ 250 MB/stotal

TFCSystem

ECS

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

11

The “old” System

• Detector L0 boards:– talk Gb Ethernet– implement buffer to allow for Level 1 latency

• Two Data Streams:– Level-1 trigger with latency limit of ~58 ms @1

MHz• 3.8 kB/event ~4.3 GB/sec

– HLT traffic without latency limitations @ 40 kHz• ~30 kB/event ~1.2 GB/s

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

12

1 MHz DAQ

CPUFarm

Readout Network

Front-end ElectronicsTFC

System

StorageSystem

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU

CPU50 SubFarms~1800 CPUs

FE FE FE FE FEFE

SwitchSwitchSwitch Switch

ECS

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

13

Advantages

• Major simplification of the DAQ system– Only one data flow through the system– Elimination of the SFCs; – Event-Building in each CPU node– Elimination of the Level-1 decision sorter– No latency limitations for event processing– Well suited for multi-core CPU architectures

• For the Trigger software– ALL information available at all times in full

precision and full granularity• Better trigger efficiency

• Required Switching power: ~ 60 GB/s

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

14

The Hardware Candidate

Force10 equipment Port densities

– Biggest switch has 14 slots for line-cards

– Biggest port density is 90 Gb Ethernet ports per line-card (90/48 over-committed)

➔ 14*90 = 1260 GbEthernet ports

Switching Fabric➣ Switching capacity is

raw ~ 1.6 Tb/sec,usable ~ 1.2 Tb/sec (150 GB/s)

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

15

Reviewed HLT Strategy

XXXIV IMFP The LHCb DAQ and Trigger Systems: recent updates

16

Outlook

• LHCb had a very scalable and flexible DAQ schema.

• LHCb Trigger includes inclusive samples– HLT rate from 200 to 2000 Hz.

• High-mass di-muon• D*• Inclusive b

• Full L0 rate (1 MHz) sent to Online Filter Farm for a full software High Level Trigger.

• Review Vertical Trigger strategy for HLT.