COPPER Status

T.Higuchi / KEK

Jul.4th, 2009. B2GM Meeting.

Belle II Data Stream

Belle II detector

SignalDigitizer

DigitalSignal TX Signal RX Readout

ElectronicsReadout

PC

AnalogSignal TX

SignalDigitizer

ReadoutElectronics

ReadoutPC

On/in/at the detector In the electronics hut

ReadoutPCReadout

PCReadoutPCReadout

PC

EventBuilder

OnlinePC Farm

StorageSystem STOR

~30m cable

R&D of the Readout ElectronicsPresent FASTBUS-based DAQ will fail around L1 rate~1 kHz with DAQ deadtime being 20%. And in Belle II, the L1 rate will be ~10kHz.

Call for a new readout electronicsthat works up to L1 rate ~30 kHz.

Keywordsof R&D

Equipped with pipeline (FIFO)

Equipped with online CPU for data processing

Usage of commercially available products

Modularized structure for easier maintenance

Seamless switch from the present system

COPPER-II BoardSignal RX x 4 or

Signal digitizer x 4Add-on modules

to the COPPER called “FINESSE” RadiSys EPC-6315– Intel P3 800 MHz– 256 MB memory– Network boot– RedHat Linux 9

Online CPU

VME9U size

100 BaseT port x 2Data transfer and Control

Trigger module

Onboard FIFO1MB x 4 on the tailside

Signal Digitizer – AMT3 FINESSE

• AMT-3 TDC chip 2 http://atlas.kek.jp/tdc

– 24 LVDS inputs / chip.– 256-edge deep L1 FIFO (pipeline), 8-event deep trigger FIFO.– Only matching edges to the trigger are output.– 0.7ns timing resolution.– Linearity = 251.0±0.9 ns (RMS) for 250.0 ns width pulse.

• Spartan3 FPGA 2– AMT3 register control.– Data format.– COPPER local bus interface,

COPPER FIFO interface.

“tandem”-structured FINESSE

PCI Mezzanine Card Standard

• 100% Compliant w/ PCI

• Good for a high density applications

•Many commercial products–Ethernet cards, CPUs, GbE cards, memory modules, etc.

• RadiSys EPC-6315–PMC sized CPU module.–Equipped with Intel PentiumIII 800 MHz w/ 512 MB memory.–RedHat Linux 7.3 or 9, or FedoraCore 1 Linux run on it.–Price = ~\120,000 / module.

Bootable from CF card or from network.

PCI NIC PMC NIC

Performance Review

RX

TDC FINESSEEmulator

TDC FINESSEEmulator

TDC FINESSEEmulator

TDC FINESSEEmulator

Clock/TriggerGenerator

PMC CPU

Ethernet

COPPER-II

The COPPER works> 30 kHz input rate

Factor-10 datareduction

Required trigger rate

Typical trigger rate

@ 416 bytes/ev/FINESSE

0 10 20 30 40Input trigger rate [kHz]

10

20

30

40

Acce

pted

trig

ger r

ate

[kH

z]

Performance Review

0

2

4

-4

-2AMT3

–FA

STBU

S(u

nt =

AM

T3 L

SB)

Resolution0.61 LSB (RMS)

Residual distribution

• Crosscheck of the COPPER output and the FASTBUS output.– Signals from the Q-T converter of Belle CDC are

branched to the FASTBUS and the COPPER.

COPPER FASTBUS

From CDC

Daisy-chain

Performance Review

# of hits/TDC

Dea

dtim

e du

ratio

n (μ

s)

Normal data size

～ 29.5 μs

～ 2.8 μs COPPER/AMT3

FASTBUS

We achieved 90%deadtime reductionwith pipelined DAQ.

S.Y.Suzuki

COPPER History in Belle DAQ• 2002-2003

– Start of R&D from a scratch.–COPPER prototype in hand.

• 2004–Revision to COPPER-II.– Study of maximum acceptable L1 rate: OK up to 30kHz.

• 2005–Debug of AMT3 FINESSE.–Replacement of the Belle EFC DAQ with 6 COPPERs.–Crosscheck with FASTBUS DAQ.

– a

• 2006–Replacement of part of the Belle CDC

DAQ with COPPERs.–Crosscheck with FASTBUS DAQ.–Deadtime performance study.

• 2007–Replacement of the Belle CDC DAQ

totally with 89 COPPERs.–Replacement of the Belle ACC DAQ with

24 COPPERs.

• 2008–Replacement of the Belle TRG DAQ with

26 COPPERs.

• 2009–Replacement of the Belle KLM DAQ with

34 COPPERs.– ECL and TOF DAQ are kept untouched.

Ready to go

COPPER-II COPPER-3

• Major motivation for the upgrade– Replacement of disappeared/disappearing parts from the market to

the available ones.– Bug fix (replacement bug fixing patches to onboard patterns).

• Minor modifications– Replacement of the onboard Ethernet to the Gbit-Ethernet.– Length normalization of CLK/TRG lines from TTRX to FINESSEs.– Easier access to a reset switch.– Easier access to VME base address configuration switches.– Change of PMC CPU interfacing connector.– Movement of VME J0 receiver location closer to the J0.– RoHS compliant parts.

COPPER-3 Board

Looks quite compatible to the COPPER-II.(The board color is changed to turquoise blue)

COPPER-3 Compatibility Study

• Before the study: Tips to operate the COPPER-3– Use specially created Linux image for COPPER-3

• A Linux image for the COPPER-II cannot be used for the COPPER-3.• The special Linux for COPPER-3 is ok for the COPPER-II.

COPPER-3 Compatibility Study

• Compatibility study setup– Same setup as the system test for

the COPPER-II and AMT3-FINESSE

Minor troubles in compatibility study• Swap of eth0eth1 configuration• A bug found in the AMT3 device driver

AMT3 FINESSE

AMT3 FINESSE

AMT3 FINESSE

AMT3 FINESSE

Clock/TriggerGenerator

PMC CPU

Ethernet

COPPER-3

1 to

96

sign

al fa

n-ou

t

RX

• Compatible data as the COPPER-II system test are readout in this study.– Trivially, the COPPER-3 can be

said OK.– The CPU is booted up properly.– The CPU, AMT3-FINESSEs and

trigger timing system works fine on the COPPER-3.

COPPER-II/COPPER-3 in Belle II DAQ

• The COPPER-3 is ready to use.– We use the COPPER-3 as the data RX in the electronics hut.

• We also use existing COPPER-IIs (~250 boards) together with the COPPER-3s.

• We will purchase ~250 COPPER-3s including spares in coming 2-3 years.

R&D of Processor PMC (PrPMC)

• Problems of EPC-6315 (existing PrPMC)– Future prospect of the product is unclear.

• Market trend is switching to ComExpress from PrPMC.• Vendor (US RadiSys) may stop the product supply immediately.

– Both vendor and Japanese HQ are quite unprofessional.• The first release of the product was with untested BIOS.• The product was revised with an incompatible network chip.• Japan US communication is poor.

Call for a new PrPMC withconcrete future prospect and kind support.

Now in bidding procedure. The company will be fixed on Jul.14th.

Specification of New PrPMC• CPU

– Intel Atom 1.6GHz or more.

• Main memory– 256MB or more.

• Mass storage– Onboard 4GB SSD or external

media like CF.

• Network I/F– GbE w/ Intel 82574.– 900Mbps or more.

• OS– Redhat Enterprise Linux 3 or

CentOS 3 or later.

• OS bootup method– From mass storage– From USB device– From network

• User I/F– Serial console or USB/PS2

keyboard+mouse + VGA.

• Power consumption– < 15W (EPC-6315 … 18W)

Schedule of PrPMC R&D

• Jun.19th Announcement of bid• Jul.1st Meeting w/ companies• Jul.7th Deadline of technical report from companies

– Block diagram– Layout proposal of major parts

• Jul.14th Box open– Company will be fixed on the day

• Feb.19th End of R&D. We have two PrPMCs in hand then.

Summary

• We are confident that the COPPER-based DAQ is in a “state of the art”.

• A new COPPER board, COPPER-3, is compatible with the COPPER-II. We are to employ the COPPER-3s for Belle II readout electronics.

• A new PrPMC, which replaces the EPC-6315 with several problems, is under bidding process. We have them by Feb.19th.