ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.
-
Upload
jason-stanley -
Category
Documents
-
view
221 -
download
0
Transcript of ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.
![Page 1: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/1.jpg)
ROOT in the PHOBOSOnline System
Peter SteinbergBrookhaven National Laboratory
Upton, NY, USA
![Page 2: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/2.jpg)
PHOBOS experimental setup
Spectrometer
Multiplicity/Vertex
![Page 3: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/3.jpg)
Readout Channels• PHOBOS is meant (CDR) to be a high-rate detector• However, while a “small” detector, it has a “big”
channel count.• Spectrometer
• 73,728 channels
• Octagon Multiplicity + Vertex detectors• 28,612 channels
• Ring Multiplicity Detectors• 16,384 channels
• Over 120,000 channels to read out -> 150kB/central event
• Main Constraint: average bandwidth 10MB/sec allowed by RHIC computing facility (RCF)
![Page 4: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/4.jpg)
PHOBOS online data flow
Run ControlRun ControlSlow ControlSlow Control Online MonitoringOnline Monitoring
FASTBUS
Si TOF RHIC
ORACLEData
Validation
EventBuilder
(PhATDAQ)
Mercury
RCF
Trigger
![Page 5: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/5.jpg)
PhATDAQ• PhAT is the general PHOBOS analysis toolkit (see
Gunther Roland’s talk)• Originally used for offline tasks
• Data analysis, algorithms, etc.
• But what about DAQ? • Original DAQ group started to develop using
CODA from CEBAF• But one day, Andrei Sukhanov (BNL) mentioned
that he would try and perform the event building directly using ROOT objects and I/O
• (very) soon thereafter, PhATDAQ was born
![Page 6: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/6.jpg)
PhATDAQ Architecture
EB interface
Data taking
ROC:
LengthSeq.number
Command Source Type
SubEvent or CommandSubEvent SubEvent…
Message:Length
Event# from Event ManagerHeader length
0x12345678
SubEvent:
Data
Run number
Structure of the communication messages
Legend:
Working
Being developed
Work not started yet
Work started
EventBuilderphatdaq.exe
Run Control(root/phat)
UDPsocket
TCP TSocket
Data & Commands
Data
PhatOnlineSilicon
TOF
Trigger
ROC
ROC
ROC
ROCDBGateway
Data BaseSlow Monitoring
events
(Andrei Sukhanov, BNL)
Gigabit
![Page 7: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/7.jpg)
PhATDAQ data format• Large segments of detector are read out by
“Front-End Controller” (FEC)• Each FEC consists of several “strings” of “chips”• The chip data is concatenated in each string.• Thus, channels are indexed by
FEC/String/ChannelFEC FEC
StringString
FECFEC
StringString StringString StringString
ChipChip ChipChipChipChipChipChip ChipChip ChipChipChipChipChipChip
“channels”
![Page 8: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/8.jpg)
ROOT in PhATDAQ• ROOT objects are used to store and access raw
data• Each FEC is its own object, which contains arrays of
string data.
• Trees are used for raw data files• one event after the other, no tags as of yet.
• Heavy use of ROOT containers for flexibility• Our TPhEvent class is a hash table which store relevant
objects (TPhDataObject, TPhObjectContainer)• However, hashtables are “heavy” -- we have a substitute
object (TPhSubEventArray) which can be turned into a TPhEvent and vice versa.
• PhATDAQ has been working perfectly for 6 months• Extensive performance benchmarks to come.
![Page 9: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/9.jpg)
Run Control
• So far, CINT is our run control interface• .L runcontrol.cxx• rc_start_run()• rc_end_run(), etc.
• Works very well• flexible• robust
• DAQ team is conspiring to move to a GUI• Inspired by NA45 RC GUI
![Page 10: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/10.jpg)
Online Monitoring• Designed and implemented by a team at
University of Illinois, Chicago • Judith Katzy, Clive Halliwell, Don McLeod, Mike Reuter
RCFPHOBOS PhATDAQ
TCP/IPDual Pentium Machines
ORACLE
![Page 11: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/11.jpg)
Monitoring Implementation• Based fully on PhAT and ROOT• Class library encompassing various functions
• Main process - event loop • Display classes• Histogram filling and managing• Histogram selecting
• Geometry and electronics setup is self-configuring• GUI controls are created dynamically
• Easily extendable to add new histogram controls • All available online application options available
through user interface:• Program control, histogram selection, display functions
![Page 12: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/12.jpg)
Online Monitoring GUI
![Page 13: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/13.jpg)
ROOT-based event display
• Module of Online Monitoring package
• Allows selection of sub-detectors
• Hits can be displayed either on the pixel, or perpendicular (hedgehog display)
• 2/3 dimensional display
• Different display options:• Lego• Color coded
Octagon
Spectrometer
Planes
(Don McLeod, UIC)
![Page 14: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/14.jpg)
More event displays
![Page 15: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/15.jpg)
Oracle, ROOT & PhAT• Oracle was chosen long ago as the only PHOBOS database• Oracle is used comprehensively in the whole system
• Silicon testing• Pedestals and Calibrations• Data Tracking• Experimental Logbook• Slow control information (from Labview & PhATDAQ)
• However, Oracle is not OO• For storing certain objects, simple tables will not do.• Hough tables, tracking templates,...
• Want to store and retrieve arbitrary objects (as BLOBs) by means of a “generic” key. • Should also work with ROOT TTrees
• For more info, ask George Heintzelman ([email protected])
![Page 16: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/16.jpg)
Signal Processing in PhAT• PHOBOS requires extensive amounts of low-level
signal processing before any hits can be found• Requirements:
• Granular - algorithms should be “plug-and-play”. • Geometric -- wanted to be able to easily analyze data by
subsection (FEC, String, Chip, Channel) without overhead of traversing object trees.
• Easy to adapt to DSP code (c code, Mercury platform)• Easy to debug -- simple interface between data and
histograms• Need ROOT compatibility
• Led to development of TPhModules and several new data structures to satisfy these requirements...
![Page 17: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/17.jpg)
TPhTopology
FEC Detector
FEC FEC
StringString
FECFEC
StringString StringString StringString
ChipChip ChipChipChipChipChipChip ChipChip ChipChipChipChipChipChip
• Any particular geometry has a fixed electronics structure• Thus, every channel has a well defined location• This lets us store a set of values as a flat array.• The detector topology can then be used to find the start of data! We
can iterate from there.
“TopoData”
(George Heintzelman,
BNL)
![Page 18: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/18.jpg)
TPhTopoData• With this “topology” we can create data
structures of different “depths”• “Channel Depth” - one entry for each channel• “Chip Depth” - one entry for each chip (e.g. for CMN
studies…)• etc.
• The contents of this structure can in principle be any object ( working w/ templates in CINT)• Integer, Float, etc.• Any TObject - TPhHistogram, TPhCalibration, TPhFrog,
etc.
NEW
![Page 19: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/19.jpg)
Application to Signal Processing
• Visualization of arrays
• Makes histograms merely a visualization tool, rather than a fundamental object to be moved around
Pedestals stored in a TPhTopoData<int>
“TPhTopoData<TH1F>”
Visualize “by string”
![Page 20: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/20.jpg)
Things we’d like to see...• A collection from the collaboration:
• More & Better Template support• reduces code duplication, increases type-safety• need working examples, esp. w/ STL
• Simple shared memory interface• At least a way to directly share simple structures
with an object interface• Maybe not as critical if thread support improves
• Rationalization of the relationship between objects and TDirectories
• “now, where did my pointer go?…”• Instead, had to roll our own (TPhObjectManager)
![Page 21: ROOT in the PHOBOS Online System Peter Steinberg Brookhaven National Laboratory Upton, NY, USA.](https://reader035.fdocuments.us/reader035/viewer/2022062315/5697c0151a28abf838cce056/html5/thumbnails/21.jpg)
And something we really need• Help!
• PHOBOS group at BNL is also looking for a postdoc (or even two)
• Available immediately• Scope very flexible, but a few ideas:
• Improving Oracle classes (could be of general use to community)
• We have a very powerful multiple-DSP system (~8 G4’s) between the silicon readout and the event builder.
• Not fully exploited yet. • Plans for Level 3 triggering : multiplicity, vertex, tracking
• Physics! RHIC is a very exciting place to be right now. Offline framework is still under intensive development.