Active Buffer Status in CBM DAQ W. Gao, A. Kugel, R. Männer, G. Marcus, M. Stapelberg, A. Wurz 06...

Active Buffer Status in CBM DAQ

W. Gao, A. Kugel, R. Männer, G. Marcus, M. Stapelberg, A. Wurz

06 Oct 200914th CBM Collaboration Meeting

Split

06.10.2009 CBM[14] Split

Wenxue Gao ZITI.Uni-Heidelberg 2

Agenda

• Test set-up and traffic classes• Hardware preparation

– ABB DMA logic improvement– Larger Event Buffer: DDR FIFO– Optical Link (Frank, CA Group, ZITI)

• Software preparation– ABB Daemon– Misc. test programs

• Performance tests

06.10.2009 CBM[14] Split


Test Set-up

ABB2

ROCs

06.10.2009 CBM[14] Split


Traffic Classes in Link Protocol

• DAQ– Hit data packet– Uni-directional: downstream– Major data flow

• CTL– Commands and responses– Bi-directional

• DLM (Deterministic Latency Message)– Timing purpose– Synchronization

06.10.2009 CBM[14] Split


ABB2 Block Diagram

DDR2SFP-0

SFP-1

ABB2

4x PCIe

V5LX110T

LP

FIFOwrapper

DMAFibreBRAMFIFO

BRAM

DGen IGen

xbar

(CAG)

06.10.2009 CBM[14] Split


DMA Upgrade

• Interface upgrade– 32-bit to 64-bit– Back-to-back transactions

• BAR[2] access bugs• Time-out’s

– DMA time-out: does nothing– Event Buffer time-out: padding with -1– Tx arbitration time-out: padding with -1

06.10.2009 CBM[14] Split


DDR FIFO

• Reason– 128 KB BRAM FIFO is a little too small to avoid data loss

• Size– 256 MB = 64 bits × 32M

• Peak bandwidth– 8B × 2 × 125 MHz = 2 GB/sec

• Behaves as standard FIFO (not FWFT yet)– Reset takes longer time (3.6 ms) than normal FIFO

• Data count issue– Disagree between count and empty flag crashed the host

06.10.2009 CBM[14] Split


Optical Link Module(CAG, ZITI)

• 2.5 Gbps / link• Both for ABB2 and ROC• Protocol

– packet-based

• Combining 2 links outputs in ABB2– Additional XBAR module

for ABB2

06.10.2009 CBM[14] Split


ABB Daemon

06.10.2009 CBM[14] Split


ABB Daemon (cont’d)

• Uses UNIX IPC mechanism• A transparent replacement for direct driver usage • Control requests have a higher priority than data transfer• Takes care of the concurrent access of the device by

multiple software pieces • Provides a high-level interface of network messages for

the ROC– PUT, GET, NPUT, NGET, …

• Provides a DMA interface for the FIFO read• Minimal overhead

06.10.2009 CBM[14] Split


and, with all these …

06.10.2009 CBM[14] Split


Commissioning Tests• (nXyter –) ROC – DCB – ABB2 – ABB Daemon – upper-level test programs• Data generators available at two positions

– ABB2: without optical links– ROC: with optical links

• Loop-back– inside ABB2 before asymmetric DAQ channel– inside DCB: CTL verification in this way

• Traffic class tests status– CTL: no error– DAQ: no error. Much more effort than CTL– DLM: primarily tested

• 2 ROCs are also tested– Connected to ABB2

• Software support– PCI Driver + MPRACE Library + ABB Daemon + testNXyter

06.10.2009 CBM[14] Split


CTL Message Latency Tests

Write-Read Read

Latency (mSec) 0.138 0.053

• Software layer• 2 tests are made

– Read latency– Read-after-write latency

06.10.2009 CBM[14] Split


DAQ Performance

DMARead Perf.

DDR FIFO

(large)

BRAM FIFO

(small)1-ROC 272.7 MB/s 199.2 MB/s

2-ROC 308.1 MB/s 305.8 MB/s

• ROC DGen data source– Instead of nXyter format

• Software layer– Instead of data-link layer

06.10.2009 CBM[14] Split


Status Summary

• Hardware upgrade– DMA transportation– 256 MB DDR FIFO– CAG.ZITI: Optical link + XBar

• Software– ABB Daemon– nXyter-format data source DMA read test

• “United” tests in stability– DCB-ABB2 for CTL robustness test– ROC-ABB2– 2 ROCs - ABB2

06.10.2009 CBM[14] Split


End of Talk

Active Buffer Status in CBM DAQ W. Gao, A. Kugel, R. Männer, G. Marcus, M. Stapelberg, A. Wurz 06...

Documents

Transcript of Active Buffer Status in CBM DAQ W. Gao, A. Kugel, R. Männer, G. Marcus, M. Stapelberg, A. Wurz 06...