M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Architectural Design and Supervisor RoIBuilder
• T/DAQ Organization
• The Pilot Project
• Supervisor & RoI Builder
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Major T/DAQ Milestones
• LVL1 Technical Design Report June 1998
• Technical Progress Report and Work Plan June 1998 for DAQ, EF, LVL2 and DCS
• Technical Proposal on DAQ and HLT Dec. 1999
• Technical Design Report on DAQ and HLT June 2001
• LVL1 Trigger Construction Completed Dec. 2002(excluding on detector electronics)
• Stand-alone DCS System Completed Jan. 2003
• DAQ and HLT Construction Completed Dec. 2003(reduced processing power)
• Integration of detectors with T/DAQ/DCS Dec. 2004completed
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
T/DAQ IB Members and ResponsibilitiesInstitute Representative Email address L1 L2 DAQ/EF DCSArgonne National Lab Robert BLAIR [email protected] 1 1Bern Klaus PRETZL [email protected] NEW SINCE IMOU 1Birmingham John GARVEY [email protected] 1Bucharest Mihai CAPRINI [email protected] 1CERN Livio MAPELLI [email protected] 1 1 1 1Copenhagen, NBI John Renner HANSEN [email protected] 1 1Cracow Zbigniew HAJDUK [email protected] 1Edinburgh Owen Boyle [email protected] NEW SINCE IMOU 1 1Geneva Allan CLARK [email protected] 1Genova Paulo MORETTINI [email protected] 1 1Heidelberg Eike-Erik KLUGE [email protected] 1Innsbruck Dietmar KUHN [email protected] 1 1Istanbul Arif MAILOV [email protected] NEW SINCE IMOU 1Jena Andreas REINSCH [email protected] 0.5 0.5JINR, Dubna Vladislav KOTOV [email protected] 1KEK Hiroyuki IWASAKI [email protected] 1 1Kobe Hiroshi TAKEDA [email protected] NEW SINCE IMOU 1Kyoto University Hiroshi SAKAMOTO [email protected] NEW SINCE IMOU 1Lecce Ornella PALAMARA [email protected] 1Lisbon Amelia TEIXEIRA MAIO [email protected] 1Liverpool Robert DOBINSON [email protected] 1 1London, QMW Eric EISENHANDLER [email protected] 1London, RHBNC John STRONG [email protected] 1 1London, UCL Peter CLARKE [email protected] 1 1Mainz Lutz KOPKE [email protected] 1 1Manchester Richard HUGHES JONES [email protected] 1 1Mannheim Reinhard MAENNER [email protected] 1 1Marseille Francois ETIENNE [email protected] 1Michigan State Maris A. ABOLINS [email protected] 1 1Moscow State Serguei SIVOKLOKOV [email protected] 1Nagasaki Yasushi NAGASAKA [email protected] NEW SINCE IMOU 1Naples Sergio PATRICELLI [email protected] NEW SINCE IMOU 1NIKHEF Joseph VERMEULEN [email protected] 1 1 1Novosibirsk Iouri MERZLIAKOV [email protected] 1Pavia Valerio VERCESI [email protected] 1Prague AS & CU Maria SMIZANSKA [email protected] 1Protvino Serguei KOPIKOV [email protected] 1RAL Charles Norman GEE [email protected] 1 1 1Rome I Emilio PETROLO [email protected] 1 1Rome II Rinaldo SANTONICO [email protected] 1Saclay Jiri BYSTRICKY [email protected] 1 1Sheffield Chris BOOTH [email protected] NEW SINCE IMOU ? 1Shinshu Tohru TAKESHITA [email protected] NEW SINCE IMOU 1St Petersburg Iouri RIABOV [email protected] 1 1Stockholm Sten HELLMAN [email protected] 1Technion Simon ROBINS [email protected] NEW SINCE IMOU 1 1Tel Aviv Halina ABRAMOWICZ [email protected] 1 1Tokyo ICEPP Tomio KOBAYASHI [email protected] NEW SINCE IMOU 1Tokyo MU Ryosuke HAMATSU [email protected] NEW SINCE IMOU 1UCI Andrew James LANKFORD [email protected] 1 1Udine Fabrizio Scuri [email protected] NEW SINCE IMOU 1Weizmann Lorne LEVINSON [email protected] 1 1Wisconsin Sau Lan WU [email protected] 1 1
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
National CommitmentsLVL 1 LVL 2 DAQ / DCS to ta l
tr igger tr igger evt. filter
Armenia 0Australia 0Aust r ia 3 0 0 3 0 0Azerbaijan 0Belarus 0Brazi l 0Canada 0Czech Republic 4 0 4 0Denmark 5 0 0 5 0 0 1 0 0 0Finland 0France IN2P3 0France CEA 1 3 5 0 2 5 5 0 3 9 0 0Georgia 0Germany BMBF 2 9 1 0 1 0 2 5 7 7 0 4 7 0 5Greece 0I s r a e l 3 1 5 6 0 3 7 5I t a l y 2 6 0 0 9 0 0 2 4 0 0 5 9 0 0Japan 3 0 2 5 1 5 0 0 4 5 2 5Morocco 0Netherlands 3 2 5 3 2 5 2 0 0 8 5 0Norway 0Poland 1 5 0 1 5 0Portugal 3 0 0 3 0 0Romania 0Russia + JINR 4 5 4 5Slovak Republic 0Slovenia 0Spain 0Sweden 6 0 0 6 0 0Switzer land 4 0 0 0 4 0 0 0Turkey 1 5 0 1 5 0United Kingdom 3 7 2 0 1 1 5 0 1 0 7 0 5 9 4 0US DoE + NSF 3 2 9 0 6 7 5 3 9 6 5CERN 2 5 0 0 1 5 0 0 4 4 0 0 1 6 0 0 1 0 0 0 0
total 1 5 6 7 0 1 0 2 9 0 1 8 9 8 5 1 8 0 0 4 6 7 4 5
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Level 1 Trigger Participating Institutes
Germany Heidelberg Mainz
Italy Naples Rome I Rome II
Israel Haifa Tel Aviv Weizmann
Japan KEK Kobe Kyoto UE
Shinshu Tokyo ICEPP Tokyo MU
Sweden Stockholm
UK Birmingham London QMW RAL
CERN
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Commitments to LVL1• Calorimeter Trigger Processor (7640 kCHF, Dec. 2002)
Birmingham, London QMW, RAL (49%); Heidelberg, Mainz (38%);Stockholm (8%); not covered (5%)
• Muon Trigger Processor for RPC System (2605 kCHF, Dec. 2002)Naples, Rome I & II (100%)
• Muon Trigger Processor for TGC System (3340 kCHF, Dec. 2002)KEK, Kobe, Kyoto, Shinshu, Tokyo ICEPP & MU (91%); Haifa,Tel Aviv, Weizmann (9%)
• Muon Central Trigger Processor (675 kCHF, Dec. 2002)CERN (100%)
• Central Trigger Processor (535 kCHF, Dec. 2002)CERN (100%)
• TTC Distribution Backbone (1310 kCHF, Dec. 2002)CERN (100%)
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Level 2 Participating Institutes
Austria Innsbruck
Chech RepublicPrague AS & CU
Denmark Copenhagen
France CEA/Saclay
Germany Mannheim
Italy Genova Lecce RomeI
Israel Haifa Tel Aviv Weizmann
Netherlands NIKHEF
Poland Cracow
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
LVL2 Participating Institutes Cont.
Russia Moscow SU
UK EdinburghLondon UCL
LiverpoolManchester
London RHBNCRAL
US ArgonneU of Wisconsin
UC Irvine Michigan State U
CERN
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Commitments to LVL2
• LVL2 Calorimeter Trigger (1870 kCHF, Dec. 2003)Argonne, Michigan State (45%); Mannheim (11%); Saclay (45%)
• LVL2 Muon Trigger (1290 kCHF, Dec. 2003)CERN (16%); Genova, Lecce, Rome I & II (53%); Mannheim (28%);Haifa, Tel Aviv, Weizmann (5%)
• LVL2 tracking trigger (4595 kCHF, Dec. 2003)CERN (22%); Copenhagen (11%); Cracow (3%); Mannheim (10%);London RHBNC & UCL, Manchester, RAL (18%); NIKHEF (7%);Prague AS & CU (1%); UC Irvine, Wisconsin (28%)
• LVL2 global trigger (1590 kCHF, Dec. 2003)Argonne, Michigan State (18%); CERN (19%); Genova, Lecce, Rome I(14%); Liverpool, Manchester, RAL (19%); Saclay (31%)
• LVL2 Supervisor/RoI Builder (845 kCHF, Dec. 2003)Argonne, Michigan State (100%)
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Data Acquisition and Event Filter
Austria Innsbruck
Denmark Copenhagen
France CEA Saclay
Germany Mainz
Italy Pavia Udine
Netherlands NIKHEF
Portugal
Romania Bucharest
Russia NovosibirskJINR
Protvino St. Petersburg NPI
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Data Acquisition and Event Filter cont.
Switzerland Bern Geneva
Turkey Ankara Istanbul
UK EdinburghLondon UCSheffield
LiverpoolManchester
London RHBNCRAL
US ArgonneWisconsin
UC Irvine Michigan State
CERN
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Commitments to DAQ/EF
• DAQ Readout (including readout buffers, local DAQ and interfaces to othersystems) (9275 kCHF excluding Common Project items, Dec 2003)Argonne, Michigan State, UC Irvine, Wisconsin (7%); Bern, Geneva (18%);CERN (22%); Copenhagen (5%); Edinburg, Liverpool, London RHBNC &UCL, Manchester, RAL, Sheffield (12%); Pavia, Udine (12%); Ankara,Istanbul (1%); NIKHEF (3%); Saclay (20%)
• Event Builder (3710 kCHF, Dec 2003)Bern, Geneva (32%); CERN (27%);KEK, Nagasaki (27%); Saclay (18%)
• Event Filter and Back-End DAQ (5675 kCHF excluding Common ProjectItems, Dec. 2003)Bern, Geneva (19%); CERN (25%); Pavia, Udine (23%); Innsbruck (5%);Ankara, Istanbul (1%); JINR (1%); KEK, Nagasaki (9%); Lisbon (5%); Mainz(14%); NIKHEF (1%)
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Detector Control System1720 kCHF, excluding Common Project Items, Jan. 2003
• Netherlands: NIKHEF 12%
• CERN 93%
• Russia St. Petersburg NPI manpowercontributiononly
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Pilot Project Definition
• Guidelines– 12 - 18 month program starting in March ‘98
– Coherent and complementary activities focused on main issues
– Integrate and coordinate present and future work of LVL2 group
• Inputs– URD
– Lessons from Demo Program
• Goals– Demonstrate full LVL2 functionality
– Select technology candidates
– Define architecture and prepare integration with DAQ/EF
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Critical Issues for LVL2
• Data Collection from ROB’s– RoI Mapping
– Preprocessing
• Data flow and control protocols– Data Transfer: via Supervisor or Processor
– Separate or integrated control network
– Processor allocation strategy
• Architectural Considerations– Boundary between LVL2 and EF
– Can LVL2 and EF use same network technology?
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Pilot Project Timeline
Technicalchoices
Demonstrators
Final spec.
TPR
96 97 98 99 00 01
Pilot project
High LevelTriggerPrototype &
Design phase
TP TDR
Architecturechoice
Technology candidates
DAQ-1
•Functional Components•Software testbeds•System integration
HLT
I`NTEGRATION
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Pilot Project Activities Matrix
Menus & strategiesEvaluation of:- SW options
- Performance requirements
Inputs to ROB URDMultiROB
ProcessorsCo-processors
NetworksTechnology watch
SoftwareTestbeds
FunctionalComponents
Processors, i/f’s & Networks
Pilot Project
Training samplesAlgorithm benchmarks
Framework
ROI BuilderROI distributors Paper
Computer
URDCost
New ideas
Reference Application
SystemDesign
Small Full sliceLarger commercial
Network
Modelling IntegrationSupervisor ROB Complex
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
ATM Testbed Workplan• Organizer Maris Abolins, Bob Blair, Patrick Le Du
• Need A large ATM switch is needed to study system performance.
• Full functionality should be demonstrated, including
• the operation and performance of LVL2 trigger algorithms.
• Objectives Establish a 64-port ATM testbed for system performance studies
• Approach Construct a large ATM testbed with commercial components
• (PCs and VME SBCs).
• Install 16 ATM ports in a 64-port FORE switch at Saclay.
• Install a 16-port FORE switch at ANL/MSU.
• Import U.S. material to Saclay to complete a 32-port system.
• Transport to CERN to install a 32-64 port testbed
• combining components from the ATM and Ethernet testbeds.
• Duration June 98 - Dec 99
• Deliverables Saclay testbed with 16 ATM ports Jun 98
• U.S. testbed with 16 ATM ports Jun 98
• Milestones U.S. testbed operational Jun 98
• Combined 32-port testbed at Saclay Dec 98
• Combined 32-64 port testbed at CERN Mar 99
• Dependencies Availability of RD31 ATM modules and components.
• Availability of Ethernet testbed components.
• Agreement with DAQ/EF group for additional components.
• Partners ANL, CERN, MSU, RHBNC, Rome, Saclay
• Risks No risk. Commercial components.
• Status/date 27/4/98 Saclay testbed with 12 ports equipped.
• 27/4/98 U.S. testbed: FORE switch and compnonents ordered.
• 27/4/98 RD31 ATM switch available Jan-June 1999.
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Workplan to Integrate RoI Builder and Multi-supervisor
• Organizer John Dawson
• Objectives Demonstrate event routing of Level 1 information
• and CPU scaling of supervisor
• Approach Construct routing hardware using S-link input
• and implementing hardware event assembly for ROI builder.
• Measure performance versus number of supervisor CPU's
• for scaling.
• Duration June - Dec 1998
• Deliverables Hardware router for ROI builder Nov 98
• Supervisor scaling document Dec 98
• Milestones All orders placed for components Apr 98
• LVL1/LVL2 Interface Fully Specified Apr 98
• Preliminary Design of RoI Builder May 98
• US testbed assembled at ANL Jun 98
• US testbed fully operational Jul 98
• ANL testbed meeting Jul 98
• S-Link Performance Measurements Aug 98
• RoI Builder Design Complete Aug 98
• Scaling Tests of Supervisor Start Sep 98
• RoI Builder Prototype Ready Oct 98
• Supervisor Scaling tests complete Nov 98
• RoI Builder to Saclay Dec 98
• Partners ANL & MSU
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Summary of Pilot Project Milestones
• LVL1/LVL2 Interface Fully Specified
• Preliminary Design of Pilot RoI Builder Complete
• Users Requirement Document (URD) form and content defined
• Supervisor Emulator Design Ready
• Start S-Link Performance Measurements
• U.S. Pilot Testbed Operational
• Final Design of Pilot RoI Builder Complete
• Start Scaling Tests of Pilot Supervisor
• Supervisor Emulator Written
• S-Link Performance Measurements Complete
• System Design Document (SDD) for 1st baseline design ready
• New Cost Estimate Ready
• Generic models for selected test setups ready in Simdaq
• Start Integration of Reference Software
• Pilot RoI Builder Complete
• Draft 0,1 of SDD Ready
• Draft 0.1 of Design Costing Document (DCD) Ready
• Supervisor Scaling Tests Finished
• WEB document on event & control records LVL1 to LVL2 &LVL2 to LVL3 first draft ready
• 15-Apr-98
• 15-May-98
• 15-Jun-98
• 15-Jul-98
• 15-Jul-98
• 15-Jul-98
• 15-Aug-98
• 15-Sep-98
• 15-Sep-98
• 15-Sep-98
• 15-Sep-98
• 15-Sep-98
• 1-Oct-98
• 15-Oct-98
• 15-Oct-98
• 31-Oct-98
• 31-Oct-98
• 1-Nov-98
• 15-Nov-98
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Pilot Project Milestones - continued
• LVL2 TTC record and interface to LVL1 Trigger Throttle Defined
• Pilot RoI Builder & Switch shipped to Saclay
• Reference Software Integration Complete
• Pilot Ethernet SRB ships to CERN
• Details of LVL1 Throttle Specified
• SRB Interface to LVL2 and Event Filter Defined
• URD Updated
• Proposal for Common Supervisor for LVL2 and EF complete
• URD Version 1.0 Review Complete
• Final Baseline Design Complete
• Costing of Final Baseline Design Approved
• Comparison of performance of selected test setups with models complete
• Version 1.0 of SDD Ready
• Version 1.0 of DCD Ready
• Documentation of modeling of selected test setups and comparison withactual performance complete
• 30-Nov-98
• 30-Nov-98
• 15-Dec-99
• 15-Jan-99
• 1-Feb-99
• 1-Mar-99
• 1-Apr-99
• 1-Jun-99
• 15-Jun-99
• 15-Jun-99
• 30-Jun-99
• 30-Jun-99
• 1-Jul-99
• 15-Jul-99
• 1-Oct-99
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Supervisor and RoI Builder
major contributions by
Bob Blair, John Dawson, Yuri Ermoline andJim Schlereth
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
LVL1/LVL2 Layout
CPULVL2
NW i/f
CPULVL2
NW i/f
CPULVL2
NW i/f
CPULVL2
NW i/f
CPULVL2
NW i/f
CPULVL2
NW i/f
NetWork
RO
BIN
RO
BIN
RO
BIN
RO
BIN
NW
i/f
Con
trol
ler
RO
BIN
RO
BIN
RO
BIN
RO
BIN
NW
i/f
Con
trol
ler
CPUROI
NW i/f
Input Router
In FIFO
CPUROI
NW i/f
In FIFO
CPUROI
NW i/f
In FIFO
CPUROI
NW i/f
In FIFO
LVL2
i/f
Muo
nToC
TP
Inte
rfac
e
LVL2
i/f
RoI
Col
lect
or
RoI
Col
lect
or
RoI
Col
lect
or
RoI
Col
lect
or
L1A
e/γ jetsτ ETCT
P
Calorimeter Trigger
LVL1
LVL2
- R
oI B
uild
er/S
uper
viso
r
ROB ComplexROB Complex ROB Complex
LVL2 CPU Farm
LVL1 information:
• RoI generation algorithms• Trigger tower mapping to
modules/crates• RoI read-out organisation
in modules/crates• RoI address encoding
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
LVL1 RoI Readout
• Assumptions
– One Readout Link per LVL1 Crate --> 12 Links
– Same links through whole system (calorimeter, muon, CTC)
– Zero-suppressed data is transferred• Questions
– Link Technology
– Data Formats
LVL1 ROI Read-out
RoI Read-out
RO
C
RoI Read-out
RO
C
RoI Read-out
RO
C
e/γ, τ jets ET,ET, µ, CTP
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
LVL1 Calorimeter Data Formats
• Item: Length Comments: (bits):
• * Start of Event (Header) 32
• * L1ID 32 24 bits
• * BCID 32 12 bits
• * RoI data
• RoI Type 4 e.g. 1 = e/g RoI eta-phiindex 12 0.1 x 0.1 RoI Thresholdset 8
• * Error Word 1 32
• * Error Word 2 32
• * End of Event (Trailer) 32
Word #
1
Content
2
3
4
N+4
N - number of RoIs
Header
Trailer
BCID
RoI data
Error Word 2
L1ID 00
00000
Error Word 1
N+5
N+6
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
LVL1 Muon Data Format
• * Start of Event (Header) 32
• * L1ID 32 24 bits
• * BCID 32 12 bits
• * RoI data (16 candidatesmaximum)
• Sector number 8 224 sectors Sub-sector number 5
• Pt value 3
• * Error Word 1 32
• * Error Word 2 32
• * End of Event (Trailer) 32
Word #
1
Content
2
3
4
N+4
N - number of muon RoIs
Header
Trailer
BCID
RoI data
Error Word 2
L1ID 00
00000
Error Word 1
N+5
N+6
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Supervisor & RoI Builder
12 S-Linksfrom LVL1
RoI
Pro
cess
or
RoI
Pro
cess
or
RoI
Pro
cess
or
RoI
Pro
cess
or
FPGA card
In 9U VME
Add Processorsas needed
S-Link
Event is assembled in cardand fed to RoI Processorsin turn.
VME interface permitsemulation of LVL1 input.
To Network
RoI Builder
M. Abolins, US ATLAS Internal Review, BNL, Aug. 4, 1998.
TLASU.S. ATLAS
Important Questions
• Is S-Link suitable? Tests in progress.
• Are current processors adequate for 100 kHz?
• Is the design robust and fault tolerant?
• Do we need an event size cutoff? Yes - 32 RoI’s.
• Is a “busy” from LVL2 necessary and how will itbe implemented? Yes...
• How will we test the device? Do we need a LVL1emulator? Design in progress.
Top Related