Post on 14-Dec-2015
Upgrade to the Read-Out Driver for ATLAS Silicon Detectors
Atlas Wisconsin/LBNL GroupJohn Joseph
March 21st 2007
ATLAS Pixel B-Layer Upgrade Workshop
2
Control Path:
ROD Controller FPGAMaster DSP
Program Reset Manager FPGA
Operation, Commands, Triggers
96
Bac
k o
f Cra
te C
ard
/ F
ron
t End
Ele
ctro
nic
s SL
ink
Ca
rd
Clo
ck
Fan
ou
t
Triggers and Event ID Data
Tim
ing
Inte
rfac
e
Mo
du
le
BOCSetup Bus
Serial Input Links
Xon/Xoff
ROD Busy
DSP Farm (4)
Event Trapping, Histograms
Data Path:
Formatter, Event Fragment Builder & Router FPGA,Diagnostic Memory, S-Link Interface & SDSP DMA
DMA 4
...
S-Link
48
System Clock
Serial Output Links
VME64x Signals
VM
E
Bu
s
40
DMA 1
ClocksHost Port
Silicon Read Out Driver (SiROD)
3
256 MB SDRAM
VME Slave FPGA
Configuration Controller
ROD BUSY Histogram
128Mb Flash CNFG & BOOT
Memory
ROD CONTROLLER
FPGA
XC4VFX20-10 Power PC Core
FORMATTER FPGA
XC3S4000-8
2x
EFB /ROUTER FPGA
XC4VLX60-10
DSP FARM TI 320C6713
220MHz 384MB SDRAM
4xDIAGNOSTIC
FIFOs
VME Bus
Back Of Crate (BOC)
CNFG DATA to All FPGAs
RESET
ROD BUSYD32
A32/D32
ROD Control/VME Path
D16
ADDR
D8
D48
1.5 GB SDRAM
ADDR
ARB
MOVE HPI INTERFACE
TO VME CONTROLLER
BOOT SLAVE DSP
FROM FLASH
4
ROD CONTROLLER
FPGA
XC4VFX20
FORMATTER FPGA
EFB/ROUTER FPGA
XC4VLX60
DSP FARM TI 320C6713
220MHz 384MB SDRAM
4x
XoffBOC
FORMATTER FPGA
INPUT FIFO 32K Deep
S-Link
Event Fragments
ROD BUSY
TTC
EVENT ID
READOUT
TIM
MDSP SP
BOCFE CMD (48)
XC3S4000
SiROD DATA PATH
Xoff
Xoff
DATA
DATA
STATUS
5
COMPONENT COMPARISONS: FORMATTER
• RevF SiROD
• Spartan2E Series: XC2S600-6FG456
• Cost: $100 x 8 = $800
• Available resources: 6912 Slices, 288K RAM Bits per FPGA
• Used resourses (1) : 2850 Slices (41%), 268K RAM Bits (93%)
• Used resourses (8) : 22.8K Slices (41%), 2140K RAM Bits (93%)
• Module FIFOs: 32 – 2048 x 32 Dual Port RAM Blocks
• Next Generation SiROD
• Spartan3 Series: XC3S4000-8FG900
• Cost: $220 x 2 = $440
• Available resources: 54K Slices, 3456K RAM Bits per FPGA
• Required resources (2) : 23K Slices, 1072K RAM Bits
• Module FIFOs: 32 - 4096 x 32 Dual Port RAM Blocks
6
COMPONENT COMPARISONS: EFB/ROUTER
• RevF SiROD
• Spartan2E Series: XC2S400-6FG676/XC2S400-6FG456
• Cypress 6 - 16K x18 15ns FIFOs
• Cost: $115 + $85 + $210 = $410
• Available resources: 9600 Slices, 1856K RAM + FIFO Bits
• Used resourses : 6080 Slices (63%), 1824K RAM + FIFO Bits
• Trap FIFOs: 4 1024 x 32 Dual Port RAM Blocks
• Next Generation SiROD
• Virtex4 Series: XC4LX60-10FG1152
• Cost: $558
• Available resources: 26K Slices, 2880K RAM Bits
• Required resources: 6080 Slices, 1824K RAM Bits
• Trap FIFOs: 4 - 4096 x 32 Dual Port RAM Blocks
7
COMPONENT COMPARISONS: CONTROLLER
• RevF SiROD
• Spartan2E Series: XC2S600-6FG676 / TI320C6201-200 DSP
• Cost: $130 + $100 = $230
• Available resources: 6912 Slices, 288K RAM + FIFO Bits
• Used resourses : 5130 Slices (74%), 188K RAM Bits (65%)
• DSP Processor
• Next Generation SiROD
• Virtex4 Series: XC4FX20-10FG
• Cost: $228
• Available resources: 8544 Slices, 1224K RAM Bits, Power PC Core
• Required resources: 5130 Slices, 188K RAM Bits, uProcessor
• Conversion from TI DSP to Xilinx Power PC core:
• Risk Evaluation Required
8
Silicon RevE(F) SiROD
MasterDSP
ControllerFPGA
FormatterFPGAs
Router FPGA
EventFragmentBuilderFPGA
SlaveDSPs
9U VME
VME Interface
Clock Distribution
9
ROD CONTROLLER
FPGA
XC4VFX20
128MB SDRAM
EFB/ROUTER FPGA
XC4VLX60
DSP FARM TI 320C6713
220MHz 384MB SDRAM
VME
J1
J2
INP
UT
FIF
O
32K
Dee
p
FORMATTER FPGA
XC3S4000
SiROD New Layout
BOC
J3
FORMATTER FPGA
XC3S4000
VME CONTROLLER
FPGA
XC3S400
DSP FARM TI 320C6713
220MHz 384MB SDRAM
DSP FARM TI 320C6713
220MHz 384MB SDRAM
DSP FARM TI 320C6713
220MHz 384MB SDRAM
128Mb FLASH
JTAG CONNECTOR ON FRONT PANEL TO PROGRAM VME CONTROLLER FPGA ROM
10
WHERE DO WE GO FROM HERE?
• Preliminary Design Study
• Recommendations:
• Purchase 2 Xilinx Virtex4 Evaluation Boards
• One for the LX series
• One for the FX series
• Combine the EFB/EventMemory/Router Designs to simulate and implement the proposed changes for verification
• Simulate and implement the Formatter as proposed for verification
• Simulate and implement the Controller to learn how to use the Power PC core and to plan a transition from the TI 6201 to the Xilinx PC core.
• Make the new design transparent to the DAQ system.
• Simulate the Control path to understand the limitations of VME BW
12
RODBus Interface
Control & Status Registers (32)
RODBus Data (16)
RODBus Address
RODBus Control
Link Input MUX
&
Half Clock
Counters
IN0
ModeBits FIFO 512x24
Link FIFO 2048 x 32
Link Decoder: 40/80/160MHz
Read Out Controller
DATA OUT (40)
TOKEN
FMT MB (12)
Link FIFO 2048 x 32
Link Decoder: 40/80MHz
Link FIFO 2048 x 32
Link Decoder: 40MHz
Link FIFO 2048 x 32
Link Decoder: 40MHz
IN1
IN2
IN3
IN4
IN5
IN6
IN7
IN8
IN9
IN10
IN11
FMT Type
FMT ID
HT LIMIT
ROD BUSY
Trailer DetectTRAILER (12)
PIXEL FORMATTER FPGA (8x)
13
EVENT FRAGMENT BUILDER FPGA
EventMemA
FIFO16Kx48
Engine 0
Engine 1
EventMemC To/F
rom
Ro
ute
rTo/F
rom
Fo
rmat
ter
To/F
rom
Fo
rmat
ter
To/F
rom
Co
ntr
olle
r
Output FIFO Data (43)
Header and Trailer Data (32)
Halt Output
Halt Output
Header/TrailerGenerator
Error Summary Word
Error Summary Word
Fo
rma
t &
Co
un
tF
orm
at
& C
ou
nt
L1
/BC
ID
Ch
ec
kL
1/B
C I
D C
he
ck
Err
or
Ch
ec
kE
rro
r C
he
ck
INC
/DE
C L
1ID
L1 & BC IDs
Event Data& Trigger Type
Dynamic Mask
Link Num. (6 bits)
Link Num. (6 bits)
Time-out Error
Time-out Error
L1ID Error
L1ID Error
BCID Error
BCID Error
Data (32 bits)
Data (32 bits)
Formatter Num. (2 bits)
Formatter Num. (2 bits)
Link Num. (4 bits)
Link Num. (4 bits)
Data (32 bits)
Data (32 bits)
Data Valid
Data Valid
Time Out
Time Out
INC
/DE
C L
1ID
Purpose: Collect Formatter output, check L1 and BC
IDs, count errors, generate Event Header & Trailer
FIFO Controller Xoff
EventMemB
FIFO16Kx48
14
ROD Router (FPGA) & Slave DSPs
SDSP 3SDSP 2SDSP 1SDSP 0
Read, formatand direct the
event data
2 TrapsEvent Type: ATLAS,
ROD or TIM
2 TrapsEvent Type: ATLAS,
ROD or TIM
2 TrapsEvent Type: ATLAS,
ROD or TIM
2 TrapsEvent Type: ATLAS,
ROD or TIM
DMA Transfer Engine1024 32-bit FIFO
DMA Transfer Engine DMA Transfer Engine DMA Transfer Engine
S-L
INK
To
/Fro
m R
OS
To/F
rom
Eve
nt
Fra
gm
ent
Bu
ild
er
XOn / XOff S-Link
S-Link DataDsp Halt Output
Halt Output
Error Format Data
Data Valid
Output FIFO (43b)
Event Header andTrailer Data (32b)
Texas Instruments 6713 floating point DSPs running at 220 MHz for monitoring and calibration histogramming
Purpose: Route formatted data to Level-2 and/or Slave DSPs (histogram)
1024 32-bit FIFO 1024 32-bit FIFO 1024 32-bit FIFO