Post on 02-Feb-2016
description
Frank Lemke
DCB and CBM Net
13th CBM Collaboration - DAQ WorkgroupUniversity of Heidelberg
Computer Architecture Group
Frank Lemke
11.03.2009
2Frank Lemke
OutlineOutline
Data Combiner Board Functionality and Status CBM Net Overview Protocol Features and Implementation Current Status Conclusion & Outlook
3Frank Lemke
DCB FunctionalityDCB Functionality
HTX Connector with 16bit LVDS bidirectional interface FPGA Virtex-4 FX60 or FX100 speed grade -10,-11 or -12 6 SFPs 128MByte of DDR2 DRAM (optional 512MByte) 512 Mb of user FLASH, 16bit interface to FPGA 125MHz low jitter clock oscillator Power supply with only 12V and 3.3V from HTX connector Power consumption of 6 to 24W
4Frank Lemke
DCB StatusDCB Status
DCB is completely tested DCB is in use at our lab for prototyping research DCB is successfully in use by many partners at universities
and in industry The newest external project with integrated DCB for communication
is a 64 nodes research cluster in Valencia (1024 Processors) Bug fixes and optimizations from V 1.0 to V 1.2 Additional options for CBM Net included in V1.3
5Frank Lemke
CBM Net – Prototype StructureCBM Net – Prototype Structure
ABBDCB
ROC
ROC
…
FEE
FEE
FEE
FEE
Slow Control
Administration Packets
Data flow
Specific communicationprotocol for CBM usage:
Guarantee for Deterministic Latency Messages (DLM)
High data bandwidth for data streamROC => ABB
High reliability for control messagesABB => ROC
Fault detection and fault tolerance
ROC = Readout ControllerDCB = Data Combiner BoardABB = Active Buffer Board
6Frank Lemke
Message TypesMessage Types
Data Packet Data messages with CRC Only with error detection
Slow Control Packet Control messages with CRC Retransmission on error
Administration Characters 1-Bit error correction IDLE, INIT, ACK, NACK … Deterministic latency message (DLM)
7Frank Lemke
Protocol ImplementationProtocol Implementation
16 Bit alignment at 125 MHz Hamming distance for control signals Standard initialization sequence measures fixed latency DLM needs a well defined coding as a control signal with Hamming
distance of 3 for a 1-Bit error correction Different virtual channels for each message type
DATA CRCSOP EOP
16 Bit 8 – 64 Byte 16 Bit 16 Bit
Example packet structure of a data packet:
8Frank Lemke
Protocol FeaturesProtocol Features
Forward error correction of 1-Bit for all special characters Meets required hierarchical structure for CBM Net Optimized data bandwidth 91.428 % ( 73.148 % with 8b/10b) Fast and efficient administration packets Well defined Deterministic Latency Messages
for synchronization and special purposes Retransmission for Control Packets Different virtual channels for each message type
9Frank Lemke
Specialties in the Specialties in the DLM Implementation DLM Implementation
The guarantee for deterministic Latency lead to Problems:
Receive clock has to be used for sending
=>Implementation of Jitter cleaner extension for DCB
Specific MGT and GTP configuration to get always the same delay after initialization
Special virtual channel in the Verilog implementation including Priority Request Insertion
System source clock vs. recovered clock
10Frank Lemke
Current StatusCurrent Status
Basic Communication tested between: DCB – DCB DCB – ROC ABB – DCB
Deterministic Latency for MGTs and GTPs proved with new designs All existing coded parts are directly implemented in a clean modular
way for reusability in later design phases First version of Protocol implemented in Verilog Test setup for first protocol version is in preparation
=> Next step is the first hierarchical communication test with CBM protocol
11Frank Lemke
Demonstrator IDemonstrator I
ROC = Readout ControllerDCB = Data Combiner BoardABB = Active Buffer Board* Thanks to Walter F.J. Mueller for this picture.
12Frank Lemke
Conclusion & OutlookConclusion & Outlook
Setup and Test of hierarchical implementation in diverse steps towards the Demonstrator I.
Iterative improvement of CBM protocol and the attached system components towards the Demonstrator I.
Collect data from Demonstrator I for future implementations needs.
Complete system simulations should be done=> the ability to decide
System structure Network type and hierarchy Read-Out buffer features and dimensions Data and memory organization
=> Build next Generation prototype
13Frank Lemke
Thank you for your attention !
Questions ?