Post on 13-Dec-2015
PSI - Jul. 18th, 2007 1
Status of the electronics and DAQ systems of the MEG
experiment
PSI - Jul. 18th, 2007 2
Electronic chain
216
630
ActiveSplitter
1:11:14:1
ActiveSplitter
1:11:14:1
ActiveSplitter
1:11:14:1
Ramp60 120
Pre-Amp8:1
PMT
LXe
HV
PMT
lateral
front
TC
HV
HV
APD
PMTbars
fibers
DC
HV
Wires
Strips
Pre-Amp
Pre-Amp
576
1156
atten
DRSDRS
DRSDRS
DRSDRS
5 crates512
Hit registers
4 boards
TriggerTrigger
Trigger
32
3 crates
Aux. devices
INFN-Le
INFN-Pv
PSI
PSI
INFN-GeINFN-Ge
PSI - Jul. 18th, 2007 3
5 crates
DRSDRS
DRSDRS
DRS
Hit registers
TriggerTrigger
Trigger
3 crates20 MHz
clock
Trigger signalEvent numberTrigger type
TriggerStart
BusyError
E5 area ‘counting room’
PC (Linux)
PC (Linux)
PC (Linux)PC (Linux)PC (Linux)
PC (Linux)
PC (Linux)PC (Linux)PC (Linux)
Front-End PCs
GigabitEthernet
On-line farm
PC (Linux)PC (Linux)
storage
PC (Linux)
Event builder
Type3
1 crate
clockstartstopsync
Ancillarysystem
DAQ and control
Run startRun stopTrigger config
Main DAQ PC PC (Linux)
INFN-Pi
PSIPSI
INFN-Ge
INFN-Pi
INFN-Pi
PSI - Jul. 18th, 2007 4
HV systemActive down regulation of an external HV supply4 different requirements
• Lxe: 1000V , 100 uA• TC bars: 2400V, 1 mA• TC curved: 500V, <1 uA• DC: 2400V, ~1 uA
• 10 chn per board - 180 chn per crate• 24-bit ADCs for high accuracy (20mV)• Read out every 4 seconds of 900 chnCommercial HV supplies installedMass production completedInstallation completed in January ‘07Tests
– laboratory tests– 60 chn for TC ran stably for 2 weeks in
December
The system is ready
PSI - Jul. 18th, 2007 5
Splitter
4÷1 Trigger or DRS
Power
Inputs• Single coaxial
cable (RG178–9m)
• Negligible crosstalk
1÷1 DRS output• high bandwidth• High density
twisted pairs• crosstalk ~ 0.6%
1÷1 trigger or Type3 output Standard twisted
pairs cable
PSI - Jul. 18th, 2007 6
Splitter Summary
Splitter– 16 inputs– 16 output 1÷1 differential 400 MHz– 16 output 1÷1 differential 100 MHz– 4 outputs 4÷1 attenuated differential 100 MHz– Noise <1 mV rms– Cross-talk ~0.6% with LXe signals– Common calibration levels from backplane
Installation– Completed in September 06
Cables– All cables ready at PSI – Only TC cables installed during December 06 run
Test– Satisfactory test during December 06 run
The system is ready
PSI - Jul. 18th, 2007 7
TC APD-preamplifiers
APD pre amplifiers• First prototype with problems on IC and cross talk• Second prototype design and test completed • Mass production and test completed• Assembly and test completed
• System delivery in Aug ’07 (Glued
to the TC fibers)
PSI - Jul. 18th, 2007 8
TC APD hit registers
• Mezzanine board host on the PSI GP-VME boards (the same boards used for DRS)
• Design, production and test completed
• The system consists of 6 VME boards
• System delivery Aug ‘07 (with the detector)
PSI - Jul. 18th, 2007 9
TC Ramp
PMT ramp generator• Design of the final boards completed• Mass production in progress• system delivery 8 boards Aug 07
PSI - Jul. 18th, 2007 10
TriggerBoards
Type1
Front end
Type2
Ancill
PSI - Jul. 18th, 2007 11
Trigger system statusInstallation
- The entire system has been installed in Jul 06. - Active sub-detectors cabled in Nov 06- Pedestal and TC triggers provided during Dec 06 test run- Integration of the trigger and DAQ Nov-Dec 06
The system is ready
Configuration firmware- Baseline Version 3 ready for download Jul 07
• Scaler readout (implemented for Type 1.3 @Run06)• Memory space arrangement (faster read-out)• Revised trigger list
Trigger tree– Increased number DCH wire sum (individual inputs for each
end) – Cosmic ray counters
Analysis software– Base analysis tools available– Trigger parameters evaluation and download, under
development– Monitoring and efficiency evaluation tools
PSI - Jul. 18th, 2007 12
. . . 14 boards
14 x 48Type1Type1
Type116
4
LXe front face
(216 PMTs)
2 boards
. . . 5+5+2 boards
9 x 48
Type1Type1
Type116
4
LXe lateral facesback (216 PMTs) 4 in 1
lat. (144x2 PMTs) 4 in 1 up/down
(54x2 PMTs) 4 in 1
1 board
Timing counterscurved (512 APDs) 8 in 1
bars (30x2 PMTs)1 board
1 board
2 x48
Type2
Type2
Type2
Type2
Type2
. . . 9 boards
9 x 48Type1Type1
Type116
4
4 boards
2 x 48Type1Type1
16
4
16Drift chambers64 channels
2 x48
1 x48
Type12 x48
16Auxiliary devices16 channels
The trigger tree
Type2
4 x 48
Type1
16CR counters32 channels Type1
16
PSI - Jul. 18th, 2007 13
Trigger list (1)QTHQTL
MeV
DWW
DWN
LXe charge e+- direction e+- timing
trig.# name conditions
0 MEG QSUM > QTH && D < DN && |T| < TWN
1 MEG-Q QSUM > QTL && D < DN && |T| < TWN
2 MEG-D QSUM > QTH && D < DW && |T| < TWN
3 MEG-T QSUM > QTH && D < DN && |T| < TWW
4 RD-narrow QSUM > QTL && |T| < TWN
5 RD-wide QSUM > QTL && |T| < TWW
PSI - Jul. 18th, 2007 14
Trigger list (2)
trig.# name conditions
6 0 QSUM > QTH && QNaI > QN && |T| < TWN
7 0-NaI QSUM > QTL && |T| < TWN
8 NaI QNaI > QN 9 LXe-high QSUM > QTH
10 LXe-low QSUM > QTL
11 CW QTL < QSUM < QTH
12 neutron Qpatch < QTpatch && Qi< QTi
13 pulse-shape && QSUM < QTL
&& QWIRE> QTWIRE
14 laser QLAS > QTLAS
15 led
PSI - Jul. 18th, 2007 15
Trigger list (3)
trig.# name conditions
16 michel NDC 4 && QL > QT && QR > QT && (QL+QR) > QTS
17 DC track out NDC 4 && Iout
18 DC track NDC 4
19 DC + CR single DC && CR
20 DC
21 CR
22 TC QL > QT && QR > QT && (QL+QR) > QTS
31 pedestal internally generated random triggers
PSI - Jul. 18th, 2007 16
Cyclic buffers
Diff. driver FADC
Cyc.buff
Proces. Algor.
LVDS Tx
Cyc.buffType1 FPGA
Cyc.buff
Proces. Algor.
LVDS Tx
Cyc.buffType2 FPGA
LVDS Rx
Cyc.buff
Proces. Algor.
LVDS Tx
Cyc.buffType2 FPGA
LVDS Rx
Type1 layer
Type2 layer
Final Type2
Analog inputs
Trigger output
PSI - Jul. 18th, 2007 17
Monitoring: TRG Waveform
• 512 channels (5120 ns)
• 10 FADC bit data
• Range 0 V --> 1V
• 20 MHz Bandwidth
• Typical pulse 400 mV
• Baseline Fluctuation 1 mV (s 0.4 mV)
T (ns)
T (ns)
PSI - Jul. 18th, 2007 18
Monitoring: system operation
Periodic monitor of the system parameters performed automatically in the DAQ
• Clock locking • Synchronous operation • LVDS data transmission • Computer busy• Rates (trigger, channel)
Automatic alarms in case of failure
PSI - Jul. 18th, 2007 19
Detector monitoring: ratesRate of individual
channels (independent of trigger)
PSI - Jul. 18th, 2007 20
Efficiency: tools
1. Cyclic buffers- dump-mode used to take 5us depth snapshot of the trigger status
for monitoring, debugging and efficiency evaluation(PMT signals, physical quantities estimators: charge, time,
amplitude, position, patterns …)- source-mode used to process simulated events as well real
recorded events
2. Trigger – Prescaled unbiased triggers simultaneously acquired with meg
events
3. Algorithm emulators– Simulation of FPGA algorithms with Xilinx tool intensively used– Emulation by means of c++ code in progress
4. Simulated data– Meg events– Calibration events
PSI - Jul. 18th, 2007 21
DRS2
PSI GVME Board
FPGA with2 Power-PC
Mezzanine boards
PSI - Jul. 18th, 2007 22
DRS System
Optical fiberBack-end PC
Off-line cluster
Ethernet
Front-end PC
All channels equipped with DRS2
• 2636 channels
• 1024 cells per channel
• 0.5 - 2.5 GHz sampling speed
•After calibration pedestal noise at 0.5 mV RMS
PSI - Jul. 18th, 2007 23
DRS2 online calibration
mV
ADC counts / 10
The pedestal is dependent on the cell number
• Need of individual pedestal value for each bin
The non-linear response function depends on the cell number
• Need of different response functions for each bin
– Measure Vin – ADCout characteristics with precise DC power supply at the DRS2 input
– Fit and store parameters online– Write on disc linearized, pedestal
subtracted samples
PSI - Jul. 18th, 2007 24
DRS3 status
50 prototypes of DRS3 available for test
PSI - Jul. 18th, 2007 25
DRS3 linearity
DRS3 linearity
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
Uin [V]
Uo
ut
[V]
Uin (V)
Uou
t (V
)
PSI - Jul. 18th, 2007 26
DRS3 nonlinearity
Point deviation from linear interpolation for five arbitrary cells
30 deg. C30 deg. C
50 deg. C50 deg. C
• Integral nonlinearity is below 1mV with only one offset per cell used for correction
• Output changes by ~1mV in 20 deg. C --> Tc = 50 ppm
PSI - Jul. 18th, 2007 27
DRS3 summary
• No dangerous temperature instability• Readout speed increased 16 MHz (DRS2) -> 33 MHz• “Region Of Interest” – Readout mode works• Master clock signal (LVDS) can be digitized differentially
improvement in clock signal• Plan:
– VME boards with DRS3 test in July 2007 – Order engineering run after all tests have been finished– Get chips in ~December, but not in time for this year’s
beam time
PSI - Jul. 18th, 2007 28
Auxiliary digitizer:Type3
Modified Type1 boards to produce an auxiliary digitization of the LXe signals
PSI - Jul. 18th, 2007 29
Auxiliary digitizer status
• Higher input capability (32 channels), no LVDS transmission• 612 PMTs on LXe lateral+back sides 20 boards• Prototype tests: Jan 07
– Bit-stream downloading through VME
– Control signal (CLK, SYNC, START, STOP) distribution – FADC digitization and storage – RAM readout
• Production: Apr 07• Tests: May 07
• Installation: Aug 07
PSI - Jul. 18th, 2007 30
DAQ
80 GB System DiskRAID 1 (Mirror)
/home/meg
1.2 TG Data DiskRAID 5
80 GB System DiskRAID 1 (Mirror)
80 GB System DiskRAID 1 (Mirror)
Back-End
Front-End #1
Front-End #2
. . .NFS
VME-Interface
VME-Interface
SC-FE
Gig
abit
Sw
itch
PSI - Jul. 18th, 2007 31
Multi-threading model
VMETransferThread
CalibrationThread
CalibrationThread
CalibrationThread
CalibrationThread
CollectorThread
VME
Round-Robindistribution
Network
Zero-copy ring
buffers
Zero-copy ring
buffers
PSI - Jul. 18th, 2007 32
DRS Readout rateOptimal readout rate of DRS full waveforms with 4
calibration threads: 30 events/sDuring Dec 06 run max readout at 7 events/s
– Double event readout– Code optimization– Single calibration thread
PSI - Jul. 18th, 2007 33
DRS Readout rate
Assuming 50% occupancy Zero suppression done on the front-endMax transfer rate at 50 events/s
– Hits the VME transfer speedMuch larger than the expected maximum trigger rate
of 20 events/s
PSI - Jul. 18th, 2007 34
Trigger readout rate
• Type 3 boards (32 ch/board)– Standard 2EVME A32, D64 Readout
– For 1 cycle TAS 650 s/board ( v 50 MB/s)
– 20 boards/crate: 70 events/s (%Live) 80% @ trigger rate = 20 s-1
• Type 1 (16 ch/board) & Type 2 (18 ch/boards)– Custom 2EVME A32, D32 Readout (no access to A31-A0 during data broadcast)
– For 1 cycle TAS 1300 s/board ( v 25 MB/s)
– 20 boards/crate: 60 events/s (%Live) 75% @ trigger rate = 20 s-1
PSI - Jul. 18th, 2007 35
Higher DAQ rate
Higher rate is considered only for calibration events
• 50 Hz full waveform readout hits VME transfer limit (83 MB/sec)
• VME transfer size can be reduced by doing zero-suppression and ADC/TDC analysis on VME side
– Use embedded Power-PC CPUs (C-code)– Use FPGA (VHDL-code)
• Tools– Basic zero-suppression in VME in late 2007– ADC/TDC analysis in VME later in summer, need input
from sub-detector groups– Possibility of reaching 100 Hz
PSI - Jul. 18th, 2007 36
DAQ rate vs. amount of data
• DAQ speed is not a limiting factor• The total data size needs solution:
– 30 Hz is maximal VME speed for full waveforms → >270 MB/sec
– Data transmission limit is 20 MB/sec (=250TB/year)→ need online reduction 10x
– Storage limit is 30 TB/year→ need offline reduction 8x
PSI - Jul. 18th, 2007 37
Online data reduction
The factor 10 data size reduction obtained within the DAQ system:
• Level 3 trigger in the Event Builder task:– Fast linear fit of the LXe energy with trigger wfms – Presence of an e+ with a minimal momentum using
DC information
• Waveform data compression– Zero suppression– ADC/TDC like data for calibration– Waveform rebinning
PSI - Jul. 18th, 2007 38
Data size reductionPossible algorithms for data size
reduction• Zero suppression: hit if
max.ampl. > n x (baseline)
• Readout window at the trigger time
• Pile-up flag: Zero-crossings of first derivation
• Re-binning of signal tail4:1, 8:1
• ADC for calibration events: Numerical integration of signals over baseline
0.5 ns bins 4 ns bins
PSI - Jul. 18th, 2007 39
Conclusions
• Splitters: installed, operational, expected performances, test in Dec 06
• Fiber preamp: problem with an IC fixed, test passed, mounted on the TC detector, installation in Aug 07
• Hit registers: mezzanine boards produced, FPGA firmware ready (PSI GPVME board), installation in Aug 07
• Trigger: installed, operational, built-in debugging and control tool, need tuning after detector turn on, test in Dec 06
PSI - Jul. 18th, 2007 40
Conclusions • DRS2: installed, operational, good for timing,
temperature dependence, usable with DC• DRS3: prototype test phase, final solution, not
available in 2007• Aux digitizer: production problem solved, prototype
test completed successfully, ready for 2007 run • DAQ: installed, operational, good performances, test in
Dec 06 run
The electronics and the DAQ systems are expected to be ready for the 2007 run
PSI - Jul. 18th, 2007 41
Rack space
PSI - Jul. 18th, 2007 42
Trigger reminder
Digital approach – Flash analog-to-digital converters (FADC)– Field programmable gate array (FPGA)
trigger observables
– energy, direction and time (Lxe calorimeter)
– e+ time and approx. direction (Timing Counters)
Expected rate – For 108 muon stop rate
e
1-s 20 4
ff
TRfRRe
PSI - Jul. 18th, 2007 43
All triggers can be:• masked• prescaled (up to 32 bits)
Trigger features
Other information
Type 1• 100 MHz, 5 s depth, 10 bits, waveforms for all channels
• Single rate for each channel
Type 2• Rate for each trigger type
• Event Counter (hardware distributed to the DRS boards)
• Trigger pattern (hardware distributed to the DRS boards)
• Live Time and Dead Time
PSI - Jul. 18th, 2007 44
Trigger efficiency: example
events associated with the TRG STOP
( ~1s delay)
s s
low intensity
(slit = 10%)
high intensity
(slit = 100%)
– Extract pulses for
– “Trigger” events
– “Unbiased” events (out of trigger window and with 10 mV threshold)
PSI - Jul. 18th, 2007 45
Trigger efficiency: TC charge
UnbiasedTrigger
Npe
Landau peak for e+
Secondaryparticles
rescaled by
RpulseT
= NT/NU
almost full efficiency at Landau peak (~6MeV, ~500pe)
Interpolation by erf function
PSI - Jul. 18th, 2007 46
DRS2 primer
0.2-1 ns Inverter (“Domino”) chain Rotatingsignal
Input 1
Output 1
16 MHz
Input 2
Channels3 to 10
Analog
outputAnalog switch
Need of external buffer and FADC
PSI - Jul. 18th, 2007 47
DRS2 Temperature Dependence
T [º C]
Vout [V]
Vout vs. Temperature
y = -0.0139x + 1.3938
0.8
0.85
0.9
0.95
1
1.05
20 25 30 35 40 45
DRS2 has a marked dependence on the temperatureDRS2 has a marked dependence on the temperature
Tc ~ 1.4 % / ºCTc ~ 1.4 % / ºC