The Muon System Electronics Upgrade
Alessandro Cardini
on behalf of
The LHCb Muon Upgrade GroupCagliari, Ferrara, Firenze, Frascati, PNPI, Roma 1, Roma2
A. Cardini for the Muon Upgrade Group
The Muon System Upgrade in brief
• Phase 1: get ready for the 40 MHz readout– nODE (with nSYNC ASIC) for an efficient detector readout by TELL40– L0muon trigger implemented on TELL40– nPDM and nSB for an efficient chamber pulsing and control using GBT
through GBT-SCA– Ready @ LS2
• Phase 2: new high readout granularity detectors for high-luminosity regions equipped with a new highly-integrated FEE ASIC– Baseline planning: prototypes ready @LS2, then construction, to be
installed @ LS3– I will not discuss this part today
213 June 2013
A. Cardini for the Muon Upgrade Group 3
Converging to a viable solution• A lot of work has been done in the last month - mainly
pushed by the time schedule for the upgrade approval within INFN
• Weekly meeting within the “Muon Electronics Upgrade Group”
• Meeting within the “Tell40 Firmware Working Group” - where many details were discussed
• Meetings with L0muon Marseille colleagues to discuss the impact of the new muon system readout architecture on the LLT
13 June 2013
A. Cardini for the Muon Upgrade Group 4
Design considerations / 1
• HIT and TDC info on same link?
– Pros• Direct correspondence between TDC data and logical channel
– Cons• The number of link to the TRIG40 will increase (need to send TDC
info on every link)
• The latency due to the data generation will increase: HITS are NZS (low latency), but for TDC data a certain form of ZS is needed, and this will require a certain elaboration time might not be compatible with LLT latency
use separate links13 June 2013
A. Cardini for the Muon Upgrade Group 5
Design considerations / 2• Use GBT WideBus (112 bits) or not (80 bits)?
– Use of GBT WideBus seems much more efficient: +40% information can be transmitted per frame better use of the bandwidth and important reduction of the number of links
– Up to know we have been using a system with 8B/10B encoding (no error correction capabilities and only a minimal capabilities of error detection) and we did not see any problem in this we believe we could also use WideBus without problems
– No need to re-cable M4R1 and M5R1 ODE crates
Use WideBus13 June 2013
A. Cardini for the Muon Upgrade Group 6
Design considerations / 3• Modularity
– We want to use a single nODE type
– Taking into account the number of active inputs on our ODEs (120, 168, 192), the number of Trigger Unit (TU) per ODEs and the number of logical channels per TU (10, 14, 16, 24, 28), the usable modularity to be used to group input channels are:
• No re-cabling 96 o 192 channels• By re-cabling M4/M5 R1 64, 96 o 192 channels• By re-cabling M4/M5 R1 and with new TB 32, 64, 96 o 192 channels
– By using standard (80 bits) mode GBT modularity 96 (or 192) is not usable, this would imply to choose a modularity of 64 or 32 than NEEDS re-cabling WideBus preferred
13 June 2013
Station Region Logical
Channels per TU
SYNCs per TU
Active channels per SYNC
TU per ODE (Active Optical links)
Active ODE
Channels
ODE per quadrant
per station
M2 or M3
R1 28 4 7 6 168 2R2 16 2 8 12 192 2R3 28 4 7 6 168` 2R4 28 4 7 6 168 2
M4 or M5
R1 24 3 8 8* 192 2*R2 14 2 7 12 168 1R3 10 2 5 12 120 1R4 10 2 5 12 120 1
A. Cardini for the Muon Upgrade Group 7
System readout architecture• Our working hypothesis (Paolo’s
hypothesis “E”)
– Use GBT WideBus (112 bits) 96 bits for data (hits or TDC info) + 16 bits header (BX counter, ZS information, other info…)
– We keep HITS and TDC info separate (on different links)
• 2 GBT for HITs• 2 GBT for TDC info
– We use a 32 channel (nSYNC) chip on nODE• Max. 32 bits/chip for TDC information
(implication on max. occupancy)
– We remove Intermediate Boards (IB) in M5R4
– Use High granularity pad detectors in M2R1 (384 pads/chamber) and M2R2 (192 pads/chamber)
13 June 2013
VTTx
GBTxECS
VTRxGBTSCA
VTTx
New SYNC
New SYNC
96
In
pu
t ch
Trig
ou
tTD
C o
ut
GBTx #1trig
New SYNC
GBTx #1TDC
New SYNC
New SYNC
96
In
pu
t ch
Trig
ou
tTD
C o
ut
GBTx #2trig
New SYNC
GBTx #2TDC
Paolo
Cia
mbro
ne IN
FN-
LNF
M
uon m
app
ing
Preliminary
• Substitute present ODE with nODE– 2 GBTx per nODE in wide frame mode format (112 data bit)
• 96 bit for trigger hits• 16 bit for header (format to be defined)
• Remove IB in M5R4 • Use high granularity chambers in M2R1 and M2R2
Hit Link summary
8
Station Region Active
Channels per nODE
Logical Channels per TU
TU per nODE
TU per GBT
nSYNC per nODE
GBT per nODE
nODE per quadrant
per station
O.L per quadrant
per station
M2
R1 192 96 2 1 6 2 6 12
R2 192 48 4 2 6 2 6 12
R3 168 28 6 3 6 2 2 4
R4 168 28 6 3 6 2 2 4
M3
R1 168 28 6 3 6 2 2 4
R2 192 16 12 6 6 2 2 4
R3 168 28 6 3 6 2 2 4
R4 168 28 6 3 6 2 2 4
M4
R1 96 /192 24 4/8 4 3/6 1 /2 2 3
R2 168 14 12 6 6 2 1 2
R3 120 10 12 6 6 2 1 2
R4 120 10 12 6 6 2 1 2
M5
R1
96 /192 24 4/8 4 3/6 1 /2 2 3
R2 168 14 12 6 6 2 1 2
R3 120 10 12 6 6 2 1 2
R4 144 24 6 3 6 2 2 4
nODE required: 35/quadrant 140 in total
Paolo
Cia
mbro
ne IN
FN-
LNF
M
uon m
app
ing
Preliminary 9
• The number of O.L. for each region of a quadrant (4 stations) is compatible with 1 AMC40– If the resources of each AMC40 and/or ATCA40 were enough, 1 TELL 40 could elaborate the
data of a quadrant
TRIG40 arrangement
O.L per quadrant
Region M2 M3 M4 M5 tot
R1 12 4 3 3 22
R2 12 4 2 2 20
R3 4 4 2 2 12
R4 4 4 2 4 14
ATCA40
QiR1
AMC40
QiR2
AMC40
QiR3
AMC40
QiR4
AMC40
12 (M2)4 (M3)
3 (M4)3 (M5)
12 (M2)4 (M3)
2 (M4)2 (M5)
4 (M2)4 (M3)
2 (M4)2 (M5)
4 (M2)4 (M3)
2 (M4)4 (M5)
Custom – 10 Gbpsn
GBTxn
4 TELL40 required to readout HIT information
Paolo
Cia
mbro
ne IN
FN-
LNF
M
uon m
app
ing
Preliminary 10
TDC Links info
Station Region Active
Channels per nODE
nSYNC per nODE
Active Channels per nSync
Max. Occupanc
y
GBT per nODE
nODE per quadrant
per station
O.L per quadrant per
station
M2
R1 174 6 32 ~25% 2 6 12R2 192 6 32 ~25% 2 6 12R3 168 6 28 ~28% 2 2 4R4 168 6 28 ~28% 2 2 4
M3
R1 168 6 28 ~28% 2 2 4R2 192 6 32 ~25% 2 2 4R3 168 6 28 ~28% 2 2 4R4 168 6 28 ~28% 2 2 4
M4 or M5
R1 96 o 192 3 o 6 32 ~25% 1 o 2 2 3R2 168 6 28 ~28% 2 1 2R3 120 6 20 ~40% 2 1 2R4 120 6 20 ~40% 2 1 2
M4 or M5
R1 96 o 192 3 o 6 32 ~25% 1 o 2 2 3R2 168 6 28 ~28% 2 1 2R3 120 6 20 ~40% 2 1 2R4 144 6 24 ~33% 2 2 4
O.L per quadrant
Region M2 M3 M4 M5 totAmc40 per quadrant
R1 12 4 3 3 22 1
R2 12 4 2 2 20 1
R3 4 4 2 2 12 0,5
R4 4 4 2 4 14 1
These seem reasonable numbers – to be checked
4 TELL40 required to readout TDC information
A. Cardini for the Muon Upgrade Group 11
Muon Detector Control and Pulsing
Complex system with more than 600 microcontrollers and 150 flash-based FPGA, controlled by 6 computers, to set front-end parameters, pulse and monitor more than 120k physical channels
13 June 2013
A. Cardini for the Muon Upgrade Group 12
Muon Detector Control Upgrade• Up to now we were only considering to replace 8 PDM boards in M2-M5:
– Use new clock distribution via GBT– Use GBT-SCA to communicate with old Service Boards (SB), although through a
complicated protocol translation (GBT-SCA <-> I2C <-> CANbus <-> I2C)
• Experts however considered that:– Current SB are already 10 years old– Based on even older microcontroller boards (ELMB)– Protocol translation would be extremely complicated and absolutely inefficient– Bottleneck is CANbus: we will not improve the current situation where we need 5’ to
configure the detectors and more than 20’ to read all scalers
• We reached the conclusion that it is necessary to built 120 new SB:– All control electronics will be renewed and will be ready to operate up to ~2030– We will be able to use the high-speed communication via GBT for a fast
configuration/reconfiguration on-the-fly of the muon system (that we know is necessary) and for an accurate real-time monitoring of the front-end boards (that is something that we would really take advantage for)
13 June 2013
A. Cardini for the Muon Upgrade Group 13
nPDM block diagram
13 June 2013
Definition of all details is currently in progress
Do we need more than 1 bidirectional GBT link? NO
GBT has 16 I/O lines, to connect up to 16 GBT-SCA in parallel, each one directly driving 16 I2C lines
we could drive up to 256 I2C lines per crate OK with current setup
New backplane implementation?
YES: details still to be understood
A. Cardini for the Muon Upgrade Group 14
nSB block diagram
13 June 2013
Depending on new backplane implementation the design of these boards could be slightly different
A. Cardini for the Muon Upgrade Group 15
Conclusions• The definition of all details of the muon system upgrade is progressing fast
• Waiting for a feedback from Marseille colleagues on the new readout architecture, but preliminary discussions suggest that is it reasonably ok
• Muon system upgrade was presented last week to the INFN CSN1 (Commissione Scientifica Nazionale 1) where it was well received and we have been asked to proceed we are preparing a document for the INFN CTS (Comitato Tecnico Scientifico), to be presented on July 12
• Next steps:– July 2013: new architecture defined– October 2013: all details defined– November 2013: new electronics review– January 2014: Muon System Upgrade TDR
13 June 2013
Spare slides
A. Cardini for the Muon Upgrade Group 17
New high-granularity M2R1 detectors• Keeping the current logic layout every detector will have 384 pads
grouped in 4 TU of 87 logical pads– 2 nODE / detector 6 nODE / quadrant– Every nODE will have 174 active inputs (2 TU of 87 logical pads)– Need 12 links/ quadrant +16 nODE
13 June 2013
A. Cardini for the Muon Upgrade Group 18
New high-granularity M2R2 detectors• Keeping the current logic layout every detector will have 192 pads
grouped in 4 TU of 48 logical pads– 1 nODE / detector 6 nODE / quadrant– Every nODE will have 192 active inputs (4 TU of 48 logical pads)– Need 12 links/ quadrant +16 nODE
13 June 2013
Paolo
Cia
mbro
ne IN
FN-
LNF
O
DE U
pgra
de
• Senza ricablaggio (sono mostrati solo i primi 96 canali, i secondi 96 sono uguali)
ODE Input Connectors
M2/M3 R2:16 Logical channels per TU(Le pad catodiche passano per le IB, le pad dei fili vanno direttamente alle ODE)
M4/M5 R2:14 Logical channels per TU
M4/M5 R3/R4:10 Logical channels per TU
M4/M5 R1:24 Logical channels per TU
TU 1 TU 2 TU 3
TU 1 TU 2 TU 3 TU 4
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
M2/M3 R1:28 Logical channels per TU(2 tipi di Tb a causa dell’orientamento dei canali logici)
TU 1 TU 2 TU 3M2/M3 R3/R4:28 Logical Channels per TU
0 7 15 23 31 39 47 55 63 71 79 87 95
Canali logici attivi sul connettore di ingresso
Canali non “attivi” (floating) sul connettore di ingresso
Canali logici appartenenti alla stessa TU
19
Paolo
Cia
mbro
ne IN
FN-
LNF
O
DE U
pgra
de
20
TU 1 TU 2 TU 3
• Ricablando M4/M5 R1 (sono mostrati solo i primi 96 canali, i secondi 96 sono uguali, ad eccezione di M4/M5 R1 che deve avere un cablaggio speculare rispetto ai primi 96)
ODE Input Connectors
M2/M3 R2:16 Logical channels per TU(Le pad catodiche passano per le IB, le pad dei fili vanno direttamente alle ODE)
M4/M5 R2:14 Logical channels per TU
M4/M5 R3/R4:10 Logical channels per TU
M4/M5 R1:24 Logical channels per TU (nei secondi 96 canali il cablaggio è speculare)
TU 1 TU 2 TU 3
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
M2/M3 R1:28 Logical channels per TU(2 tipi di Tb a causa dell’orientamento dei canali logici)
TU 1 TU 2 TU 3M2/M3 R3/R4:28 Logical Channels per TU
0 7 15 23 31 39 47 55 63 71 79 87 95
Canali logici attivi sul connettore di ingresso
Canali non “attivi” (floating) sul connettore di ingresso
Canali logici appartenenti alla stessa TU
Paolo
Cia
mbro
ne IN
FN-
LNF
O
DE U
pgra
de
TU 1 TU 2
• Ricablando M4/M5 R1 e cambiando la TB (sono mostrati solo i primi 96 canali, i secondi 96 sono uguali)
ODE Input Connectors
M2/M3 R2:16 Logical channels per TU(Le pad catodiche passano per le IB, le pad dei fili vanno direttamente alle ODE)
M4/M5 R2:14 Logical channels per TU
M4/M5 R3/R4:10 Logical channels per TU
M4/M5 R1:24 Logical channels per TU
TU 1 TU 2 TU 3
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
TU 1 TU 2 TU 3 TU 4 TU 5 TU 6
M2/M3 R1:28 Logical channels per TU(2 tipi di Tb a causa dell’orientamento dei canali logici)
TU 1 TU 2 TU 3M2/M3 R3/R4:28 Logical Channels per TU
0 7 15 23 31 39 47 55 63 71 79 87 95
Canali logici attivi sul connettore di ingresso
Canali non “attivi” (floating) sul connettore di ingresso
Canali logici appartenenti alla stessa TU
TU 3
21
A. Cardini for the Muon Upgrade Group 2213 June 2013
(Hits)
(TDC)
6 optical links required / nODE
A. Cardini for the Muon Upgrade Group 2313 June 2013
nSYNC block diagram
A. Cardini for the Muon Upgrade Group 24
System readout architecture / 2• Design considerations and first decisions taken:
– Need to readout both HITs and their time (TDC): time information is required to time align the detector and to monitor it
– HIT and TDC info on same/different link? two different links
– Modularity: only one nODE type and we will not change the cabling on the back of the crates 96 bits for data (naturally compatible only with WideBus) + 16 bits for additional information (Bxid, …)
– GBT WideBus (112 bits) or not (80 bits)? WideBus
– A new ASIC for nODE a 32 channels nSYNC
13 June 2013
A. Cardini for the Muon Upgrade Group 25
System readout architecture / 3
• Current system layout…– 104 nODE, 4 TELL40 for HIT and L0muon readout, 3 TELL40 for muon TDC info– Very compact system, no need to exchange data between AMC40 outside
ATCA40
• + Remove IB from M5R4…– M5R4 is readout by 2 IB/quadrant– 108 nODE, 4 TELL40 for HIT and L0muon readout, 3 TELL40 for muon TDC info
• + New high-granularity detectors in M2R12 (see next 2 slides)– Need to add 32 nODE (or the equivalent of it) to readout 12 M2R1 detectors
with 384 pads and 24 M2R2 detectors with 192 pads– 140 (equiv.) nODE, 4 TELL40 for HIT and L0muon readout, 4 TELL40 for muon
TDC info
• Verification of all details with Marseille colleagues in progress
13 June 2013
Top Related