Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden...

Frank Lemke

Design concepts and measurements of the CBM Design concepts and measurements of the CBM DAQ networkDAQ network

DPG – Frühjahrstagung Dresden 2013

HK 10.7

University of Heidelberg

Computer Architecture Group

Frank Lemke, Sven Schatral

04.03.2013

Frank LemkeDPG – Frühjahrstagung Dresden 20132

OutlineOutline

The Compressed Baryonic Matter Experiment Motivation CBM Network Structure Front-end ASIC Communication FPGA and ASIC Based Readout Controllers Conclusion

Frank Lemke3

FAIR at GSI Darmstadt GermanyFAIR at GSI Darmstadt Germany

Facility for Antiproton and Ion Research (FAIR) extends the existingGSI accelerator and synchrotron

Starting 2018, FAIR will be used to measure atomic nuclei and the particles they are made up of

( http://www.fair-center.de , January 2013 )


( https://www.gsi.de )

Construction work has already begun and makes visible progress

Demonstrator prototypes have been build and development for the experiment readout systems are ongoing


The Compressed BaryonicThe Compressed BaryonicMatter ExperimentMatter Experiment

One of the eight FAIR experiments is the Compressed Baryonic Matter (CBM) experiment

The CBM experiment Investigates highly compressed nuclear matter

using nucleus-nucleus collisions Is a self-triggered detector system Provides eight different kinds of detectors

( https://www.gsi.de )


MotivationMotivation

Create complete DAQ network solution usable within all stages of the network providing the required communication capabilities for CBM

Complete network approach allows design of straight forward and compact solutions fulfilling

Restricted space limits for hardware High bandwidth requirement Supporting CBM specific synchronization methodologies

No protocol conversion required Easy for user adaption and providing reusable blocks for the CBM network

Frank Lemke6

CBM Network Challenges

DAQ network system needs to deliver Flexibility for various build-up variants Efficiency for data aggregation Successive link rate increase

From 0.5 Gb/s up to 10Gb/s per lane

Clock distribution Reliable system clock from one single master Delivering up to 250 MHz over LVDS for stable FEE sampling

Precise time synchronization In the order of the link bit clock cycles -> 2 ns

Support for centralized control system solutions Dense interconnection solution Handling of up to several TB/s of raw data


Frank Lemke

CBM Network StructureCBM Network Structure

7 DPG – Frühjahrstagung Dresden 2013

( Walter F.J. Mueller, GSI, Darmstadt, 30 March 2012 )

Frank Lemke

Front-end ASIC ChallengesFront-end ASIC Challenges


Depending on type and system position FEE bandwidth varies Starting at 0.5 Gb/s using one lane

Supporting up to four lanes with 2 Gb/s

CBMnet has reusable design blocks Control and status Register File (RF) I2C support for RF access, debugging and bring-up CTRL decode for CBMnet ctrl messages ASIC SERDES implementations supporting synchronization CBMnet protocol module with Master/Slave support

Unbalanced links: 1 up-stream and up to 4 down-stream links Shift register chain and sub RF support for analog designed ASIC parts

Frank Lemke9

Front-end ASIC ModulesFront-end ASIC Modules


Frank Lemke10

CBMnet Implementation at CBMnet Implementation at SPADIC & STSXYTERSPADIC & STSXYTER


STSXYTER Device will be used for the Silicon Tracking System 2 Gb/s read-out bandwidth Currently in production

SPADIC - Self-triggered Pulse Amplification and Digitization asIC

( Please visit http://spadic.uni-hd.de/)

Device will be used for the transition radiation sub-detector (TRD)

Device to read-out and process small electrical detector signals on a single silicon

32-channel mixed signal Self-triggered hit detection and neighbor readout Full pulse recording 1 Gb/s read-out bandwidth Current version 1.0 using CBMnet protocol

communication and design modules

Frank Lemke

SPADIC MeasurementSPADIC Measurement


Synchronization accuracy on bit clock level of 2ns was shown

Configuration registers have been written and control communication using CBMnet protocol was approved

Data was successfully streamed from the SPADIC

Frank Lemke

FPGA Readout - EvalFPGA Readout - Eval


It provides

SFP to use old readout chainsFMC adapter to attach specially developed board connecting multiple FEEsHDMI cables used for first cablingIdentical FPGA type as new ROC used as emulation platform Small and low cost beam time and laboratory readout chainsLimited synchronization capability

Xilinx Spartan 6 eval-board SP605 with 45T device is used as first evaluation platform for the next generation FPGA Boards

Frank Lemke

FPGA Readout – ROC3FPGA Readout – ROC3


In the collaboration a new universal readout controller is in production using a Xilinx Spartan6 150T device

FMC adapters for attaching FEE readout Almost identical first readout using SP605 HDMI solution Designed to provide the full synchronization feature set Supporting up to 8 FEEs

Substitution of former concentrator and aggregator boards Clock source or even multiplier ECS/DCS emulator Data sink and readout chain source Emulation platform for HUB testing on limited small functional blocks

HK 34.6: Modular CBM-ROC Firmware - Was bisher geschah und wie es weitergeht. SEBASTIAN MANZ und UDO KEBSCHULL (Dienstag, 5. März, 15:30, HSZ-405)

Frank Lemke14

ASIC Readout ChallengesASIC Readout Challenges


Readout ASIC (HUB) needs to support at least 32 link FEE connections

Automatic and flexible initialization and synchronization of FEEs Aggregation of data from numerous FEE devices Rate conversion preserving reliable synchronization Creating an excellent ratio for bandwidth per area

Up to 4x, 5 Gb/s for each lane, 20 Gb/s per HUB

3 HUBs bundled to 12x links with 60 Gb/s total

Specific SERDES designs Standard cell SERDES to connect FEE devices (500Mb/s)

Full custom SERDES with CDR to connect back-end (at least 5Gb/s)

Frank Lemke15

HUB ASIC StructureHUB ASIC Structure


Direct extraction and distribution of synchronization

Deadlock avoiding for data and control communication (VCs)

Large crossbar structures enabling flexible FEE attachment and fault tolerance

Fully remote control through RF

Flexible FEE re-initialization and control

Frank Lemke16

ConclusionConclusion

The CBM protocol and the module concept have shown their usability SPADIC front-end ASIC was successfully tested Tests for STSXYTER are prepared and the chip was submitted Basic FPGA prototyping for ROC usage and HUB ASIC emulations

are ongoing

The goal for 2013 is preparing the first HUB ASIC prototype design as miniASIC for testing ofRadiation tolerance of design partsHigh-speed SERDES functionalityData aggregation and synchronization functionalities


Frank Lemke

Thank you for your attention !

Questions ?


Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden...

Documents

Transcript of Frank Lemke Design concepts and measurements of the CBM DAQ network DPG – Frühjahrstagung Dresden...