Evolution of the Evolution of the Configuration Configuration

Database DesignDatabase Design

Andrei Salnikov, SLACAndrei Salnikov, SLAC

For BaBar Computing GroupFor BaBar Computing Group

ACAT05 – DESY, ZeuthenACAT05 – DESY, Zeuthen

May 23, 2005 ACAT 2005 2

————————————————————————————————————————————

Talk overviewTalk overview

BaBar configuration databaseBaBar configuration database Migration of BaBar databasesMigration of BaBar databases New configuration database APINew configuration database API Choice of technologiesChoice of technologies Implementation dynamic loadingImplementation dynamic loading Lessons learned from migrationLessons learned from migration

May 23, 2005 ACAT 2005 3

————————————————————————————————————————————

BaBar on-line BaBar on-line databasesdatabases Conditions databaseConditions database

Calibrations, geometry, alignment, etc.Calibrations, geometry, alignment, etc. Data accessed with the event timeData accessed with the event time

Ambient database (simplified conditions)Ambient database (simplified conditions) Time history of the data-taking conditions (voltages, Time history of the data-taking conditions (voltages,

currents, temperatures, etc.)currents, temperatures, etc.) Part of the on-line detector controlPart of the on-line detector control

Configuration databaseConfiguration database Details followDetails follow

Prompt Reconstruction databasesPrompt Reconstruction databases Support for multi-node calibrationsSupport for multi-node calibrations

Electronic LogbookElectronic Logbook Etc.Etc.

May 23, 2005 ACAT 2005 4

————————————————————————————————————————————

ConConfifiguration guration databasedatabase Important part of the on-line system Important part of the on-line system Configuration database keeps Configuration database keeps

configuration data - detector and software configuration data - detector and software settings for data takingsettings for data taking

Support configuration of the on-line Support configuration of the on-line hardware and software in preparation for hardware and software in preparation for data takingdata taking

For more details see CHEP’03 talkFor more details see CHEP’03 talk Worked reasonably well since the Worked reasonably well since the

beginning of Run 1 in 1999beginning of Run 1 in 1999

May 23, 2005 ACAT 2005 5

————————————————————————————————————————————

BaBar database BaBar database migrationmigration BaBar was using Objectivity/DB ODBMS for BaBar was using Objectivity/DB ODBMS for

many of its databasesmany of its databases About two years ago started migration About two years ago started migration

from Objectivity to ROOT for event store, from Objectivity to ROOT for event store, which was a success and improvementwhich was a success and improvement

No reason to keep pricey Objectivity only No reason to keep pricey Objectivity only because of “secondary” databasesbecause of “secondary” databases

Migration effort started in 2004 for Migration effort started in 2004 for conditions, configuration, prompt conditions, configuration, prompt reconstruction, and ambient databasesreconstruction, and ambient databases

May 23, 2005 ACAT 2005 6

————————————————————————————————————————————

Scope of the migrationScope of the migration Configuration database currently holds Configuration database currently holds

about 60MB (uncompressed) of about 60MB (uncompressed) of configuration dataconfiguration data

About 30 persistent classes for About 30 persistent classes for representation of the configuration datarepresentation of the configuration data

Rather compact and simple compared to Rather compact and simple compared to other BaBar databases (e.g. conditions other BaBar databases (e.g. conditions database is 40GB and 400 classes)database is 40GB and 400 classes)

Due to its size and simplicity configuration Due to its size and simplicity configuration database is a perfect candidate for a pilot database is a perfect candidate for a pilot project within migration effortproject within migration effort

May 23, 2005 ACAT 2005 7

————————————————————————————————————————————

Configuration Configuration database APIdatabase API Main problem of the old database – API exposed Main problem of the old database – API exposed

too much of the implementation technology:too much of the implementation technology: Persistent object, handles, class names, etc.Persistent object, handles, class names, etc.

API has to change but we don’t want to make the API has to change but we don’t want to make the same mistakes again same mistakes again (new mistakes are more (new mistakes are more interesting)interesting)

Pure transient-level Pure transient-level abstract APIabstract API independent on independent on any specific implementation technologyany specific implementation technology Much easier to re-implement in different technology if Much easier to re-implement in different technology if

we don’t like one particular technologywe don’t like one particular technology Can support more than one technology if needed Can support more than one technology if needed [vital][vital] Client code independent of any persistency technologyClient code independent of any persistency technology

May 23, 2005 ACAT 2005 8

————————————————————————————————————————————

Prototyping New APIPrototyping New API To test the details of new API and one To test the details of new API and one

specific implementation, a prototype specific implementation, a prototype implementation was builtimplementation was built Great prototyping language used – PythonGreat prototyping language used – Python MySQL used for data storageMySQL used for data storage

Prototype is not “exact” because Prototype is not “exact” because languages are very different: Python – languages are very different: Python – dynamic, C++ – staticdynamic, C++ – static

But helped a lot with quick testing of But helped a lot with quick testing of different ideas and optimizing SQL tablesdifferent ideas and optimizing SQL tables

May 23, 2005 ACAT 2005 9

————————————————————————————————————————————

New Database APINew Database API Defining new API is easy – after five years Defining new API is easy – after five years

of experience it can’t be done wrongof experience it can’t be done wrong Expressing this API in C++ is not so easy Expressing this API in C++ is not so easy

one particular problem – how to pass non-one particular problem – how to pass non-polymorphic types through abstract interface polymorphic types through abstract interface

in C++ interfaces and templates do not mixin C++ interfaces and templates do not mix solution is to use RTTI at transient levelsolution is to use RTTI at transient level smart AnyPtr (void* with enough RTTI)smart AnyPtr (void* with enough RTTI) mechanism is hidden from client code, clients mechanism is hidden from client code, clients

only see type-safe methodsonly see type-safe methods

May 23, 2005 ACAT 2005 10

————————————————————————————————————————————

API migration – clientsAPI migration – clients All clients should be changed to use new APIAll clients should be changed to use new API All client code should be made free of the old All client code should be made free of the old

implementation detailsimplementation details Most time-consuming task in the whole migrationMost time-consuming task in the whole migration Re-examining dependencies, re-packaging code, Re-examining dependencies, re-packaging code,

re-thinking design at global levelre-thinking design at global level Process is hard, sometimes conflicting with other Process is hard, sometimes conflicting with other

development, but the result is beneficial for development, but the result is beneficial for everybody – cleaner code and fewer everybody – cleaner code and fewer dependenciesdependencies

May 23, 2005 ACAT 2005 11

————————————————————————————————————————————

Bridge ImplementationBridge Implementation Having an abstract API we need to build Having an abstract API we need to build

specific implementation(s)specific implementation(s) Start with obvious – we still have data in Start with obvious – we still have data in

Objectivity. Build bridge implementation Objectivity. Build bridge implementation on top of old database on top of old database proof of a working APIproof of a working API being used until migration is completebeing used until migration is complete but the goal is to have different but the goal is to have different

technology(-ies)technology(-ies)

May 23, 2005 ACAT 2005 12

————————————————————————————————————————————

Choosing TechnologyChoosing Technology

Time to choose implementation Time to choose implementation technologies, real alternatives to technologies, real alternatives to ObjectivityObjectivity

Many considerations – data access and Many considerations – data access and distribution, reliability, data modeling distribution, reliability, data modeling capabilities, etc.capabilities, etc.

May 23, 2005 ACAT 2005 13

————————————————————————————————————————————

Choosing technologyChoosing technology Two classes of clients with different Two classes of clients with different

requirements:requirements: production site needs reliable, fault-tolerant, production site needs reliable, fault-tolerant,

concurrent read-write access to the dataconcurrent read-write access to the data remote sites need zero-management, easy-to-remote sites need zero-management, easy-to-

use, fast, scalable read-only access to the datause, fast, scalable read-only access to the data

It should be possible to use different It should be possible to use different implementation technologies at production implementation technologies at production and remote sitesand remote sites

May 23, 2005 ACAT 2005 14

————————————————————————————————————————————

Read-only Read-only implementationimplementation ROOT is an obvious decision for storing ROOT is an obvious decision for storing

read-only data:read-only data: Data definition using C++ constructs allows Data definition using C++ constructs allows

easy migration of Objectivity schemaeasy migration of Objectivity schema No servers and no management needed for No servers and no management needed for

small installationssmall installations For large installations For large installations xrootdxrootd server could be server could be

used for load-balancingused for load-balancing BaBar data distribution knows already about BaBar data distribution knows already about

ROOT ROOT

May 23, 2005 ACAT 2005 15

————————————————————————————————————————————

Read-only Read-only implementationimplementation Migration was straightforwardMigration was straightforward

ROOT-persistent classes are (almost) exact ROOT-persistent classes are (almost) exact copy of Objectivity DDLscopy of Objectivity DDLs

Data size is small, everything fits in one fileData size is small, everything fits in one file Metadata and data objects are stored in Metadata and data objects are stored in

TTrees, TTrees, àà la relational tables in SQL database la relational tables in SQL database Some complications due to lack of proper Some complications due to lack of proper

indexing support in ROOT TTrees, needed indexing support in ROOT TTrees, needed some additional structuressome additional structures

May 23, 2005 ACAT 2005 16

————————————————————————————————————————————

Read-write Read-write implementationimplementation Need something more robust for this – real Need something more robust for this – real

databasedatabase Practically only option is relational database – Practically only option is relational database –

either commercial like Oracle, or free as MySQLeither commercial like Oracle, or free as MySQL Need to augment it with the object-relational Need to augment it with the object-relational

mapping, not quite trivial despite many mapping, not quite trivial despite many databases call themselves object-relationaldatabases call themselves object-relational

But we could reuse data definitions from ROOT But we could reuse data definitions from ROOT read-only implementationread-only implementation Not so critical for configuration database, but will be Not so critical for configuration database, but will be

essential for conditionsessential for conditions

May 23, 2005 ACAT 2005 17

————————————————————————————————————————————

MySQL implementationMySQL implementation Initial implementation in MySQL, but can switch to Initial implementation in MySQL, but can switch to

Oracle later if MySQL failsOracle later if MySQL fails Configuration objects are self-contained, store Configuration objects are self-contained, store

them as BLOBs in the relational tablesthem as BLOBs in the relational tables Need to serialize object data for storageNeed to serialize object data for storage

convert to ROOT objectconvert to ROOT object call Streamer() to stream data into buffercall Streamer() to stream data into buffer compress buffer, store as BLOBcompress buffer, store as BLOB

De-serialization works in opposite directionDe-serialization works in opposite direction Reuse of the persistent classes from ROOT read-Reuse of the persistent classes from ROOT read-

only implementation!only implementation!

May 23, 2005 ACAT 2005 18

————————————————————————————————————————————

Building applicationsBuilding applications Now we have three implementations: Now we have three implementations:

Bridge (temporary), ROOT, and MySQLBridge (temporary), ROOT, and MySQL Decision on which one to use should be Decision on which one to use should be

delayed until run time, so that the same delayed until run time, so that the same application could run everywhereapplication could run everywhere

Should link all implementations into all Should link all implementations into all applications?applications?

Not good, don’t want to install MySQL if Not good, don’t want to install MySQL if need only ROOT read-onlyneed only ROOT read-only

Load implementations dynamically at run Load implementations dynamically at run timetime

May 23, 2005 ACAT 2005 19

————————————————————————————————————————————

Dynamic loadingDynamic loading In principle it should be possible to split In principle it should be possible to split

things into static/dynamicthings into static/dynamic In practice it is very complex and needs In practice it is very complex and needs

extreme careextreme care In BaBar we ended up with the dynamic In BaBar we ended up with the dynamic

loading of loading of libmysqlclient.solibmysqlclient.so only only ROOT implementation is linked into applicationROOT implementation is linked into application part of MySQL implementation toopart of MySQL implementation too but no link dependency on MySQLbut no link dependency on MySQL

May 23, 2005 ACAT 2005 20

————————————————————————————————————————————

Current StatusCurrent Status New API and all three implementations are New API and all three implementations are

now in productionnow in production Bridge implementation is what we are Bridge implementation is what we are

currently using in productioncurrently using in production Data in two other implementations are Data in two other implementations are

constantly updated from production dataconstantly updated from production data Preparing data distribution for ROOT Preparing data distribution for ROOT

configuration dataconfiguration data Planning to switch production to use Planning to switch production to use

MySQL implementationMySQL implementation

May 23, 2005 ACAT 2005 21

————————————————————————————————————————————

Lessons learnedLessons learned Always make abstract APIs to avoid problems in Always make abstract APIs to avoid problems in

the future (this may be hard and need few the future (this may be hard and need few iterations)iterations)

Client code should be free from any specific Client code should be free from any specific database implementation detailsdatabase implementation details

Early prototyping could answer a lot of questions, Early prototyping could answer a lot of questions, but five years of experience count toobut five years of experience count too

Use different implementations for clients with Use different implementations for clients with different requirementsdifferent requirements

Implementation would benefit from features Implementation would benefit from features currently missing in C++: reflection, introspection currently missing in C++: reflection, introspection (or from completely new language)(or from completely new language)

May 23, 2005 ACAT 2005 22

————————————————————————————————————————————

ConclusionsConclusions We have redesigned and reimplemented We have redesigned and reimplemented

BaBar configuration databaseBaBar configuration database Different alternative implementations exist Different alternative implementations exist

for clients with different requirementsfor clients with different requirements No specific performance figures, but it No specific performance figures, but it

should be better than original Objectivity should be better than original Objectivity performanceperformance

Good experience which will be used in Good experience which will be used in migration of other BaBar databasesmigration of other BaBar databases