A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow...
-
Upload
bryan-ramsey -
Category
Documents
-
view
218 -
download
0
Transcript of A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow...
![Page 1: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/1.jpg)
A. Amorim
1
The HERA-B database servicesdetector configuration, calibration,
alignment, slow control, data classification
• The HERA-B detector• The database problem• The Architecture• The Berkeley-DB DBMS• The client/server integration• The domains and solutions• Conclusions and Outlook
A. Amorim, Vasco Amaral, Umberto Marconi, Tome Pessegueiro, Stefan Steinbeck, Antonio Tome, Vicenzo Vagnoni and Helmut Wolters
![Page 2: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/2.jpg)
A. Amorim
2
HERA-B Experiment B/B-tagging B/B-tagging
B0/B0 J/KS B0/B0 J/KS
Vertex Detector Si strip 12 m resolution
RICH (/K)multianode PMT
TRD (eh)straw tubes+thin fibers
ECAL(+eh)W/Pb scintillator
shashlik
MUON (h)tube,pad and gaspixel chambers
Tracking: - ITR(<20cm): MSGC-GEM - OTR(>20cm): 5+10mm drift cells
Magnet: 2 Tm
C4F10
HiPt triggerpad/gas pixel
![Page 3: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/3.jpg)
A. Amorim
3
The main challenge: Selecting
![Page 4: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/4.jpg)
A. Amorim
4
HERA-B DAQ
L2-farm: 240 PC’s L4-farm 200 PC’s
1000 SHARC (DSP)
Detector Front End Electronics
FCS
DSP SWITCH DSP SWITCHEvent Control
TriggerPC
TriggerPC
TriggerPC
INTERNET SWITCH
4LTPC
4LTPC
LoggerPC
SLT/TLT
4LT
![Page 5: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/5.jpg)
A. Amorim
5
The HERA-B database problem
To provide persistence services (including online-offline replication) to:
• Detector configuration
• Common accepted schema
• Calibration and alignment • Distributing information to the reconstruction and trigger farms
• Associate each event with the corresponding database information
•Slow control
• Manage updates without data redundancy
•Data set and event classification
•Online Bookkeeping
• Detector Configuration
• Calibration and Alignment
• Slow Control
• Data Set and Event Classification
• Online Bookkeeping
![Page 6: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/6.jpg)
A. Amorim
6
Characterizing the context
Inter-processdata-flow rate
Complexity
Persistent Data-Volume
ARTE in memory structures:· Accessed very often by 1 process· Transient data limited to memory· No versioning, high complexity
Event Stream:· Accessed 2-3 times by 1 process· Large integrated data volume· Relatively simple structure
Database Services:· Accessed every data-set or slow control
update by Several hundred processes· Persistent data always available for retrieval· Time and version, high complexity
Analysis
20 Terabytes /year
Gigabyte/update
Large number of classesMerging data and updates
Complex queries
![Page 7: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/7.jpg)
A. Amorim
7
Keys, objects and client/server
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
client/server at the SDB level +RPM -> an UDP based communication package.
/PM/ Descrip. field1 ; field 2; ...
.2
.5-.1
56892
...Db: /RICH/HV/
Db: /RICH/HV/
versions
Key= name+ version
Machine independent blub of DATA
![Page 8: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/8.jpg)
A. Amorim
8
The Berkeley DB
See http://www.sleepycat.com/
• Embedded transactional store with: logging, locking, commit and roll back, disaster recovery.
•Intended for: high-concurrency read-write workloads, transactions and recoverability. • Cursors to speed access from many clients..
• Open Source policy The license is free for non-commercial purposes - rather nice support
•No client/server support is provided
![Page 9: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/9.jpg)
A. Amorim
9
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
Slow Control Interface
Slow Control interface Client/server
Metadata Object Data Object Update
time
Pmt1000Pmt1003Pmt2000
Update
1 ...1.5 ...2 ...
1.2 ...
1.6 ...2.3 ...
slctemp
nfields : intcolumsize : intfieldtype : int*xbuf : void *ids : int *columdata : void *
LocCopy2Table( )LocCopyTable( )LocMergeSlcTable( )
MIZZItable
SlcDatabase
x : Tablesdbname : char *
SlcBookDatabase( )
db
1..1
.sch tupd.dat
tdat
SlcTable
nused : int = 0nmax : intcarray : char **tabledes : char *tabledat : char *tableupd : char *maxnupd : intslctemp : slctempdat *nchardef : int
SlcBookTable( )SlcUnbookTable( )SlcGetDatabase( )SlcOpenTables( )SlcCloseTables( )SlcAdd2Table( )SlcFindInTable( )SlcGetDescTable( )SlcPutDescTable( )SlcStoreUpdTable( )SlcStoreDataTable( )SlcMemTable( )SlcTableValues( )SlcTableGetpt( )SlcTableGetSpt( )
1..1
(trough the tabledes name )
.upd
Optimized Queries
![Page 10: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/10.jpg)
A. Amorim
10
Associations to Events
Index Objects (referenced by events) Client/server
Active server interface
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
Slow Control interface Client/server
Index Obj. Index Obj.
... ...
Index Obj.
...
Revision 0 - online
Index Obj.
Calibrating
1 - offline
Dynamic Associations
Index Obj. Created in active
Servers
![Page 11: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/11.jpg)
A. Amorim
11
Basic n-n associations (LEDA)
LEDA - Object Manager (hash table implemented associations)
Key objects (referenced by events) Client/server
Active server interface
SDB/local thin layerMIZZI
C- formatter/client
Datastore Server
SDB/client thin layer
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
Berkeley DB / Sleepycat embedded database managerOpen Source
Slow Control interface Client/server
Key key’
Key key’
•Associations are navigated with iterators•Using hash tables.•Keys as OID’s with the scope of classes. •Explicitly loaded or saved (as containers)
![Page 12: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/12.jpg)
A. Amorim
12
GUI for editing and drawing
From R&D: JAVA, TCL/TK, gtkReusing and extending widget.
Data hidden from TCL/TK
ROOTdatabaseBindingSocket:
Client/ Server
![Page 13: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/13.jpg)
A. Amorim
13
General Architecture
SDB/local thin layerMIZZI
C- formatter/client
Datastore ServerSlow Control interface
Key objects (referenced by events)
Replication
Tcl/TkOpen Source
SDB/client thin layer
ROOTOpen Source
HERA-B client server infrastructure (DAQ software)
C- formatter/localMIZZI
LEDA - Object Manager(hash table implemented associations)
Berkeley DB / Sleepycat embedded database managerOpen Source
Database GUI Spreadsheet
Db GUIPlotting
Root ObjectPersistency
Client/server
Client/server
Socket
Active serverinterface
MySql, RDBMSOpen Source
Tag and Data Quality
Tag and D/Q replication
109 Evt./y
![Page 14: A. Amorim 1 The HERA-B database services detector configuration, calibration, alignment, slow control, data classification The HERA-B detector The database.](https://reader030.fdocuments.us/reader030/viewer/2022032612/56649ec75503460f94bd3c41/html5/thumbnails/14.jpg)
A. Amorim
14
ConclusionsONLINE:• Large number of Clients => Gigabytes per Update
•broadcast simultaneously to SLT• tree of cache database servers to the 4LT
• Correlates (dynamically) each event with the databases objects• 600 k SLC parameters using data and update objects
•parameter history is re-clustered on the database servers • The online database system has been successfully commissioned
OFFLINE:• Replication mechanism decouples online from offline
•also provides incremental backup of the data • TCP/IP gateways and proxies
• “data warehousing” for data-set classification -> MySQL • Relation to event tag under evaluation• Also providing persistency to ROOT objects
Using Open Source external packages has been extremely useful.