A. Amorim

1

The HERA-B database servicesdetector configuration, calibration,

alignment, slow control, data classification

• The HERA-B detector• The database problem• The Architecture• The Berkeley-DB DBMS• The client/server integration• The domains and solutions• Conclusions and Outlook

A. Amorim, Vasco Amaral, Umberto Marconi, Tome Pessegueiro, Stefan Steinbeck, Antonio Tome, Vicenzo Vagnoni and Helmut Wolters

A. Amorim

2

HERA-B Experiment B/B-tagging B/B-tagging

B0/B0 J/KS B0/B0 J/KS

Vertex Detector Si strip 12 m resolution

RICH (/K)multianode PMT

TRD (eh)straw tubes+thin fibers

ECAL(+eh)W/Pb scintillator

shashlik

MUON (h)tube,pad and gaspixel chambers

Tracking: - ITR(<20cm): MSGC-GEM - OTR(>20cm): 5+10mm drift cells

Magnet: 2 Tm

C4F10

HiPt triggerpad/gas pixel

A. Amorim

3

The main challenge: Selecting

A. Amorim

4

HERA-B DAQ

L2-farm: 240 PC’s L4-farm 200 PC’s

1000 SHARC (DSP)

Detector Front End Electronics

FCS

DSP SWITCH DSP SWITCHEvent Control

TriggerPC

TriggerPC

TriggerPC

INTERNET SWITCH

4LTPC

4LTPC

LoggerPC

SLT/TLT

4LT

A. Amorim

5

The HERA-B database problem

To provide persistence services (including online-offline replication) to:

• Detector configuration

• Common accepted schema

• Calibration and alignment • Distributing information to the reconstruction and trigger farms

• Associate each event with the corresponding database information

•Slow control

• Manage updates without data redundancy

•Data set and event classification

•Online Bookkeeping

• Detector Configuration

• Calibration and Alignment

• Slow Control

• Data Set and Event Classification

• Online Bookkeeping

A. Amorim

6

Characterizing the context

Inter-processdata-flow rate

Complexity

Persistent Data-Volume

ARTE in memory structures:· Accessed very often by 1 process· Transient data limited to memory· No versioning, high complexity

Event Stream:· Accessed 2-3 times by 1 process· Large integrated data volume· Relatively simple structure

Database Services:· Accessed every data-set or slow control

update by Several hundred processes· Persistent data always available for retrieval· Time and version, high complexity

Analysis

20 Terabytes /year

Gigabyte/update

Large number of classesMerging data and updates

Complex queries

A. Amorim

7

Keys, objects and client/server

SDB/local thin layerMIZZI

C- formatter/client

Datastore Server

SDB/client thin layer

HERA-B client server infrastructure (DAQ software)

C- formatter/localMIZZI

Berkeley DB / Sleepycat embedded database managerOpen Source

client/server at the SDB level +RPM -> an UDP based communication package.

/PM/ Descrip. field1 ; field 2; ...

.2

.5-.1

56892

...Db: /RICH/HV/

Db: /RICH/HV/

versions

Key= name+ version

Machine independent blub of DATA

A. Amorim

8

The Berkeley DB

See http://www.sleepycat.com/

• Embedded transactional store with: logging, locking, commit and roll back, disaster recovery.

•Intended for: high-concurrency read-write workloads, transactions and recoverability. • Cursors to speed access from many clients..

• Open Source policy The license is free for non-commercial purposes - rather nice support

•No client/server support is provided

A. Amorim

9

SDB/local thin layerMIZZI

C- formatter/client

Datastore Server

SDB/client thin layer

HERA-B client server infrastructure (DAQ software)

C- formatter/localMIZZI

Berkeley DB / Sleepycat embedded database managerOpen Source

Slow Control Interface

Slow Control interface Client/server

Metadata Object Data Object Update

time

Pmt1000Pmt1003Pmt2000

Update

1 ...1.5 ...2 ...

1.2 ...

1.6 ...2.3 ...

slctemp

nfields : intcolumsize : intfieldtype : int*xbuf : void *ids : int *columdata : void *

LocCopy2Table( )LocCopyTable( )LocMergeSlcTable( )

MIZZItable

SlcDatabase

x : Tablesdbname : char *

SlcBookDatabase( )

db

1..1

.sch tupd.dat

tdat

SlcTable

nused : int = 0nmax : intcarray : char **tabledes : char *tabledat : char *tableupd : char *maxnupd : intslctemp : slctempdat *nchardef : int

SlcBookTable( )SlcUnbookTable( )SlcGetDatabase( )SlcOpenTables( )SlcCloseTables( )SlcAdd2Table( )SlcFindInTable( )SlcGetDescTable( )SlcPutDescTable( )SlcStoreUpdTable( )SlcStoreDataTable( )SlcMemTable( )SlcTableValues( )SlcTableGetpt( )SlcTableGetSpt( )

1..1

(trough the tabledes name )

.upd

Optimized Queries

A. Amorim

10

Associations to Events

Index Objects (referenced by events) Client/server

Active server interface

SDB/local thin layerMIZZI

C- formatter/client

Datastore Server

SDB/client thin layer

HERA-B client server infrastructure (DAQ software)

C- formatter/localMIZZI

Berkeley DB / Sleepycat embedded database managerOpen Source

Slow Control interface Client/server

Index Obj. Index Obj.

... ...

Index Obj.

...

Revision 0 - online

Index Obj.

Calibrating

1 - offline

Dynamic Associations

Index Obj. Created in active

Servers

A. Amorim

11

Basic n-n associations (LEDA)

LEDA - Object Manager (hash table implemented associations)

Key objects (referenced by events) Client/server

Active server interface

SDB/local thin layerMIZZI

C- formatter/client

Datastore Server

SDB/client thin layer

HERA-B client server infrastructure (DAQ software)

C- formatter/localMIZZI

Berkeley DB / Sleepycat embedded database managerOpen Source

Slow Control interface Client/server

Key key’

Key key’

•Associations are navigated with iterators•Using hash tables.•Keys as OID’s with the scope of classes. •Explicitly loaded or saved (as containers)

A. Amorim

12

GUI for editing and drawing

From R&D: JAVA, TCL/TK, gtkReusing and extending widget.

Data hidden from TCL/TK

ROOTdatabaseBindingSocket:

Client/ Server

A. Amorim

13

General Architecture

SDB/local thin layerMIZZI

C- formatter/client

Datastore ServerSlow Control interface

Key objects (referenced by events)

Replication

Tcl/TkOpen Source

SDB/client thin layer

ROOTOpen Source

HERA-B client server infrastructure (DAQ software)

C- formatter/localMIZZI

LEDA - Object Manager(hash table implemented associations)

Berkeley DB / Sleepycat embedded database managerOpen Source

Database GUI Spreadsheet

Db GUIPlotting

Root ObjectPersistency

Client/server

Client/server

Socket

Active serverinterface

MySql, RDBMSOpen Source

Tag and Data Quality

Tag and D/Q replication

109 Evt./y

A. Amorim

14

ConclusionsONLINE:• Large number of Clients => Gigabytes per Update

•broadcast simultaneously to SLT• tree of cache database servers to the 4LT

• Correlates (dynamically) each event with the databases objects• 600 k SLC parameters using data and update objects

•parameter history is re-clustered on the database servers • The online database system has been successfully commissioned

OFFLINE:• Replication mechanism decouples online from offline

•also provides incremental backup of the data • TCP/IP gateways and proxies

• “data warehousing” for data-set classification -> MySQL • Relation to event tag under evaluation• Also providing persistency to ROOT objects

Using Open Source external packages has been extremely useful.