The Read Out Driver for the ATLAS Muon Precision …thei/HEF/atlas/Pub/lecc2002-prest-mrod.pdf11...

11 September 2002 LECC 2002, Colmar 1

The MRODThe Read Out Driver for the

ATLAS Muon Precision Chambers

Marcello Barisonzi, Henk Boterenbrood,Peter Jansweijer, Gerard Kieft, Jos Vermeulen

NIKHEF, AmsterdamAdriaan König,Thei Wijnen

NIKHEF and Univ. of Nijmegen, Nijmegen


Contents

• MROD System Overview• MROD-0 Feasibility Study• MROD-1 Prototype• First Results• Conclusions & Outlook


ATLAS MDT Muon Detector

~ 300.000 Drift Tubes

~ 1200 MDT Chambers

~ 200 Towers of 6 Chambers


System OverviewChamber Tower

24 ch. TDC

CSM18 x

24 ch. TDC

CSM-Link(GOL)*

6 x MROD 160 MBytes/sS-Link to ROB

24 ch. TDC

CSM18 x

24 ch. TDC

CSM-Link(GOL)

*) http://proj-gol.web.cern.ch/proj-gol


CSM Functionality

Serial to Parallel&

Clock Domain Separator

40 Mbit/sData/Clockfrom TDC

18 x

Serial to Parallel&

Clock Domain Separator

40 Mbit/sData/Clockfrom TDC

Separator

(GOL)

≈1 Gbit/s

1 Start bit32 Data bits

1 Parity bit2 Stop bits

36 bits @ 25 ns = 900 ns

1 Separator word (S)18 TDC data words19 words in 900 ns 85 MB/s

S1

18

CSM


TDC0, word 1

TDC2, word 4

TDC3, word 2

TDC0, word 1

TDC1, word 3

TDC2, word 5

TDC3, word 3

TDC3, word 0TDC2, word 0TDC1, word 0

TDC1, word 1

TDC1, word 2

TDC2, word 1

TDC2, word 2

TDC2, word 3

TDC3, word 1

Build events in a partitioned memoryfrom TDC data fragmentstim

e

(tdc 1) 000…000

Separator word

Separator word

Separator word MROD FunctionalityCheck (do not store)

Skip (do not store)

(tdc 0) 000…000

TDC0, word 0


MROD Form Factor• 9 U VME board (single slot), 6 CSM Inputs, 1 S-Link Output• Optionally 2 extra CSM Inputs with “extension” board to

accommodate some special towers with > 6 chambers• CSM Input Interfaces integrated on main board

• 1 MROD Crate (Sub rack) contains:12 MRODs (12 η Segments)Up to 4 MROD extension boards1 Crate Master with Ethernet Interface (ROD Crate DAQ)1 TIM: TTC-Rx Interface Module (incl. ROD Busy)

• @ 192 towers: 192/12 = 16 MROD Crates (1 per ϕ Sector)


MROD Crate

ROD CrateDAQ

MROD MROD… total ...12 x

6 CSMs 6 CSMs

TIM (TTCrxInterface)

ROD BusyROB ROB

DAQ / DCS

VME-bus

One MROD Crate services 12 towers (one full ϕ sector). In total 16 crates will be required for all MDT chambers. Some MRODs may have 7 or 8 input links via “slave” MROD input cards.

From TTC system

“TIM-bus”

Network


MROD Throughput

MRODMROD-in

MROD-out

Average 5 hits per TDC + header + trailer = 7 words/eventPer tower of 6 chambers max. 88 TDCs * 7 ≅ 600 words/event (= 2.4 kB/event)Worst case est.: @ 100 kHz L1A rate 240 MB/s per MRODCalculation based on actual tower layout (J.Chapman): max. rate < 60 MB/s per MROD

CSM-Link

S-Link

CSM-Link

CSM-Link

CSM-Link

CSM-Link

CSM-Link

MROD-in

MROD-in


MROD-0 Feasibility Study

MROD-in

MROD-in

MROD-in

MROD-in

MROD-in

MROD-in

MROD

MROD-out

CSM

CSM

CSM

ROB

CSM

CSM

CSM


(SHARC-I)


MROD-0 Prototype

MROD-out

SHASLINK

MROD-in

MCRUSHsorted

TDC-dataover

SHARC Link


MROD-0 Input Channel

1 MBZBT

Memory

SHARC

FPGA

Data FIFO

TetrisRegister

InputOutput

FIFO Control

Control/StatusError signaling

6 Sh

arc

l inks

@ 4

0 M

B/ s

eac

h

FIFO

Length FIFO

MCRUSH


MROD-0 Output Channel

SHaSLINK

PCI 9054

PCI b

us

SHARC

Altera10K10A

S-Link max.@ 160 MB/s

6 SHARC Links@ 40 MB/s each


MROD-0 Emulation Hardware

MROD-out

SHASLINK

1

0MROD-in

(3x)

MCRUSH

0

2

3

MROD-0

fragment lengths

sortedTDC-data

sorted +merged

TDC-data

optionally double/tripleMROD-in output thus

simulating 2 or 3 MROD-insevent fragment lengths

via SHARC-link simulatesfuture MROD-1 functionality

1 3

S-Link

Module typexxxxx

SHARC-links

ROBSIM

CRUSH +SHASLINK

SHASLINK dataTDC-

CSMSIM0


MRODOUT ROBSIM CSMSIM MRODIN

MROD

CSM-simulator performance

0.0

50.0

100.0

150.0

200.0

250.0

300.0

350.0

400.0

450.0

0 5 10 15 20 25 30 35

Words/TDC

Even

t rat

e [k

Hz]


MRODIN (1x) + MRODOUT + ROBSIM, 18 TDCs

0,0

20,0

40,0

60,0

80,0

100,0

120,0

140,0

160,0

180,0

0 5 10 15 20 25 30 35Words/TDC

Eventrate

[kHz]

MRODIN MRODIN+MRODOUT MRODIN+MRODOUT+ROBIN

MRODOUT MRODIN

MRODMROD-0PerformanceStudy Results

CSMSIM ROBSIM



0,0

20,0

40,0

60,0

80,0

100,0

120,0

140,0

160,0

180,0

0 5 10 15 20 25Words/TDC

Eventrate

[kHz]


MRODOUT MRODIN


CSMSIM ROBSIM



0,0

20,0

40,0

60,0

80,0

100,0

120,0

140,0

160,0

180,0

0 5 10 15 20 25Words/TDC

Event rate

[kHz]


MRODOUT MRODIN


CSMSIM ROBSIM


MROD-0 Performance Analysis• Measured event rate for single output SHARC-I @ 40

MHz with 2 and 3 emulated CSM inputs: maximumevent rates of 70 and 50 kHz respectively are measured.

• MROD-1 will use the SHARC-II @ 80 MHz: both the processing speed and the bandwidth increase proportionately → event rate ≈ 100 kHz ?

• → use 2 SHARC-II processors for MROD-out.


Memory

SHARC

FPGA

SHARC(2x)

MemoryFPGA

3x (in total)

VME64x

TTCInterface

Memory

SHARC

FPGA

MemoryFPGA

Sharc Links

MROD-1Prototype


SHARC-II


The ADSP-21060 and the ADSP-21160 SHARCs

• 40 MHz / 80 → 100 MHz CPU (SIMD mode)

• 512 KB / 512 KB internal memory

• 6 x 40 / 80 → 100 MB/s links. Throughput of all links simultaneously is 160 / 480 → 600 (?) MB/s, without disturbing the CPU.

• No handshaking on links, but hardware XON-XOFF protocol,

• 10 / 14 DMA channels

• Support for bus arbitration: at max. 6 SHARCs can be connected to a commonbus without glue logic. Each SHARC can access the internal memories of each otherSHARC. The SHARCs also provide support for a so-called host interface, which canact as an additional master on the common bus.

• Fast interrupt servicing due to the presence of shadow registers

• Two 40 Mbit/s / 40 → 50 Mbit/s (at max.) synchronous serial ports

• Can be booted via link 4


MROD-1 Form Factor• 9 U VME boards, 2 units wide

• 1 S-Link output on daughter board

• 6 GOL inputs on daughter boards

• SHARC II (ADSP21160) DSPs:(3 for input, 2 for output processing)

• Altera APEX FPGAs, 200k gates• TIM bus over special P3 back plane• VME64x interface

GOLdaughterboards

Input

Input

S-Linkdaughterboard

Input

Output

Motherboard


MROD-out Board


MROD-in Board


MROD-1 Prototype Status• Fully functional MROD-1 modules exist:• 7 MROD-in and 3 MROD-out boards have all

been extensively tested, also in conjunction with simulated data via the input link.

• Software to boot the SHARCs via the VME bus and a run-time environment providing file and terminal I/O via a server program under both the LynxOS and Linux operating systems on the VME processor is available.


MROD-1 Prototype Status (Contd.)

• Current MROD-1 SHARC software is largely ported from the MROD-0 prototype. Further development of this code is required.

• This process is hampered by a compiler which is not fully reliable, in particular when it comes to optimization which in turn is very important to obtain good performance.

• Transition to SHARC-II DSPs has not always been completely straightforward: work-aroundshave been implemented. See example below ….


MROD power-up modification


First MROD-1 Results

• Preliminary single channel tests with a CSM simulator show a sustainable 125 kHz event rate with 18 simulated TDCs with five 32-bit words per event.

• Equivalent of 45 MBytes/sec.• Rate of 125 kHz appears to be limited by the

input (i.e. the CSM simulator) rather than by the MROD-1 itself. This is being investigated.


Conclusions & Outlook• The first MROD-1 test results are in line with the

MROD-0 results and the SHARC-I to SHARC-II extrapolation.

• Results are encouraging. Improvements still possible. More tests need and will be done.

• Development of a GOL receiver card to connect to the CSM is well underway.

• System integration tests with both the CSM and the DAQ-1 and ROD CRATE DAQ follow soon.

• Application in NIKHEF local Cosmic Ray Test Stand.

The Read Out Driver for the ATLAS Muon Precision …thei/HEF/atlas/Pub/lecc2002-prest-mrod.pdf11...

Documents

Transcript of The Read Out Driver for the ATLAS Muon Precision …thei/HEF/atlas/Pub/lecc2002-prest-mrod.pdf11...