Post on 29-Jan-2016
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
A Modular Object Oriented Data Acquisition System
for the Gravitational Wave AURIGA experiment
Antonio CeseracciuGabriele VedovatoAntonello Ortolan
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
2- The AURIGA Gravitational Wave detector
Location: Laboratori Nazionali di Legnaro – INFN First Acquisition Run: jun 1997 – nov 1999 Detector upgrades: transducer, suspension Next Acquisition Run: apr 2003
Detector: resonant bar• Operating conditions: T=0.1K, attn 300dB• Electronic readout: dc-SQUID, є=100 h• Sensitivity: Δl/l ≈ 10-20
Antenna readout: 5 kHz, 1.5 Gb/day Secondary readouts: seismometers, magnetometers, power monitor; 24 channels @200 Hz, 1 Gb/day
International GW Observatory:•UTC synchronization: GPS based system,accuracy 1 µsData Exchange: common data format
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
3- DAQ hardware system
HP1430
HP1413
RS232 MXI
GPSSatellite
AURIGADetector
4882.8125 Hz – 23 bitData Sampling
24 channels200 Hz – 16 bitData Sampling
10 MHzSynchronization Clock
ADC InterruptsData Synchronization
GPS DateString
DAQ Workstation
GPS
Environment
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
4- Requirements and Building Blocks
Main Requirements for the DAQ system:● Reliability: for long uninterrupted runs;● Fault-Tolerance;● Flexibility: modularity, real-time monitoring, components reuse,
extensibility, network transparence.
To fulfill at design level the above requirements, the envisioned architecture:
● is a multi-agent one, with multiple asynchronous processes doing specific tasks;
● Processes must be allowed to stop/reset/die at any time without forcing to reset the whole system.
Multi-agent design needs a powerful framework to take care of all the common needs of processes control and intercommunication
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
5- Software Technology
Fundamental platform choices (and specific implementation):● Object Oriented design methodology● C++ language● CORBA interprocess communication (OmniORB, AT&T)● O.S. GNU/Linux (RedHat); ● Compiler GCC● Persistence format: IGWD Frame (Interferometric Gravitational Wave Detector) :the format adopted by GW experiments● QT graphic toolkit (Trolltech)
Development tools:● CVS ● IDE: Kdevelop (KDE project), Designer, Trolltech
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
6- DAQ software architecture
Log
AntennaANT
AuxiliaryAUX
GPS
RS232
BuilderBLD
DiskDSK
MXI
gps time raw data
Raw data
4882.8125 Hz – 23 bit
Data Sampling
24 channels200 Hz – 16 bitData Sampling
GPS Date String
LoggerLog
messages
Acquisition info
commands
PermanentData storage
860 880 900 920 940 960
1
10
100
1000
AU
Hz
Frequency (Hz)
Monitor
Monitordata
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
7- the PCL (Process Control Library)
WorkingThread
Log msg queue
Commands
Spy Queue (frame)
From Remote Queue
C
OR
BA
In
terf
ace
Devic
e Inte
rface
From Device
Data queue (frame)
Data output
Data Input
CommandsStatus Info
● Defines a common architecture for processes, thus called PCL processes● All the DAQ processes are PCL processes
● The PCL takes full care of: a) InterProcess Communication b) Synchronization c) Transient and Persistent storage d) Process Control
● The architecture provided by the PCL is inherently multithreaded● Part of the PCL is provided as a shared library; part as template classes
All of the custom process code is embedded in the Working Thread
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
8- The PCL intercommunication model
ACTIVELOADED CONFIGUREDINITIAL
FAILURE
Boot
StopAbortShutdown
StartConfig
Reset
Reset
Process control is modeled by ● defining a small FSM● associating controlcommands to statetransitions
● All produced data is stored / transmitted by pushing it into a queue ● Consumers will poll asynchronously for new data.● At every poll request, all the pending items in the data queue are fetched ● Benefits: asynchronous intercomm, simple design, but safe and efficient
W TData queue
WorkingThread
Data queue Data queuePoll Request
Producer Consumer
WorkingThreadDATA
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
9 – PCL process model and CORBA
PclInterface_impl+pclProducer: T*
+Boot()+Configure()
+Start()+Stop()
+Abort()+Shutdown()
T
POA_PclInterface
PortableServer::RefCountServantBase
PclProducer#qpcl: FrameQueues*
#qpclx: FrameXQueues*+Boot()
+Config()+Start()
+Stop()+Abort()
+Shutdown()+Work()
main
«bind»ProcessProducer
omni_thread
Producer
PCL process
PCL
PclOmniBind+startServer(prc:int,nQueues:int)
T
OmniORB
● Specialized codeis in Producer● Producer inherits the process model, and is bound as template parameter
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
10- performance
The performance requirements for the AURIGA experiment are not very high. The requested throughput is about 30kB/sec, and the DAQ system easily proved more than one order of magnitude above that.
Peak performance tests were conducted on two networked (100 Mb)Intel(R) Xeon(TM) CPU 2.40 Ghz machines
For any block size, a limiting factor of 50Hz, probably due to the CORBA data transport, is present. The maximum throughput at 9600kB is limited by the network interface
AntennaANT
BuilderBLD
DiskDSK
100Mb 100Mb
Block size (kB) Hz Throughput (kB/sec) 16 50 800 64 50 3200192 50 9600
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment
11- Summary
The first achievement for the Data Acquisition System: Providing all the desired functionality.
Many improvements were brought in: Dataflow and persistent data: the frame format has been adopted to
transport and store any information (GW and aux data, configuration...)
Object Oriented design proved consistent and flexible; basic classes reused within the analysis environment Open source components and tools proved flexible and powerful, and
ensure the longevity of the system
The DAQ system has been in use for more than one year, requiring no maintenance nor code tweaking. It is ready for the upcoming new acquisition run of the AURIGA experiment.
CHEP 2003 – Antonio Ceseracciu – A Modular Object Oriented Data Acquisition System for the Gravitational Wave AURIGA experiment