Design, Development and Evolution of the ROOT system Rene Brun February 21 2010 ACAT2010 Jaipur...

Design, Design, DevelopmentDevelopmentand Evolutionand Evolutionof the ROOT of the ROOT

systemsystemRene BrunRene Brun

February 21 2010February 21 2010ACAT2010 JaipurACAT2010 Jaipur

René Brun, ACAT2010, René Brun, ACAT2010, JaipurJaipur11ROOT: design,development, evolutionROOT: design,development, evolution

planplanROOT in 2010ROOT in 2010

packagespackages

usersusers

How did we reach this situation?How did we reach this situation?after-PAW fightsafter-PAW fights

Objectivity era and big confusion about object Objectivity era and big confusion about object persistencypersistency

LCG project and problems with the committee LCG project and problems with the committee approachapproach

the ramping and consolidation yearsthe ramping and consolidation years

DirectionsDirections


René Brun, ACAT2010, René Brun, ACAT2010, JaipurJaipurROOT: design,development, evolutionROOT: design,development, evolution 33

OS & compilers

Networking

Frameworks likeROOT, Geant4

ExperimentSoftware

End userAnalysis software

Hardware

20 MLOC

2 MLOC

2 MLOC

0.1 MLOC

HardwareHardware

HardwareHardware

Software Software hierarchyhierarchy

ROOT todayROOT todaya quick reviewa quick review


15 years of ROOT15 years of ROOT


1995 2000 2005 2010

afterPAW

GEANT3

afterPAW

GEANT3

PAW-likeROOT

PAW-likeROOT

Objectivity

era

Objectivity

eraLCG

projectsstart

PI, SEAL, POOL

LCG projects

startPI, SEAL,

POOLROOT

mainstream

ROOTmainstrea

m

Modular Modular structure?structure?

Nice and Nice and modular modular building.building.

But can But can we use we use only what only what we need?we need?


Project StructureProject Structure

CORECORE

I/OI/O

GRAF3D

GRAF3DGRAF2

DGRAF2

DMAT

HMAT

H

HISTHIST

NETNET Trees

Trees

TGEOTGEO

GUIGUI

PROOF

PROOF

xrootd

xrootd

Roofit

Roofit

TMVATMVAEVEEVE

reflex

reflex

CINTCINT VMCVMC

g3rootg3root

Tfluka

Tfluka

g4root

g4root

RooStatsRooStats

pyroot

pyroot

77ROOT: design,development, evolutionROOT: design,development, evolution René Brun, ACAT2010, René Brun, ACAT2010, JaipurJaipur

frameworkframeworkLike in the Like in the Albert Albert Hall Museum,Hall Museum, we we created base created base classes (the CORE), classes (the CORE), doors and windows doors and windows (IO) and utility (IO) and utility libraries on top.libraries on top.

Can be seen as a Can be seen as a mess or a very mess or a very functional building.functional building.


InterpretersInterpretersIn this talk, I have no time to discuss issues about In this talk, I have no time to discuss issues about interpreters like interpreters like CINT CINT and and PyrootPyroot, or the internals , or the internals of the reflexion system (see my slide about of the reflexion system (see my slide about LLVM LLVM later).later).

CINT and its reflexion system is the cornerstone of CINT and its reflexion system is the cornerstone of ROOT for the IO system and scripts.ROOT for the IO system and scripts.

CINT was not designed to be a full C++ CINT was not designed to be a full C++ interpreter. We came several years ago with interpreter. We came several years ago with ACLIC ACLIC that automatizes the interface with the compiler.that automatizes the interface with the compiler.

We strongly recommend the use of ACLIC for We strongly recommend the use of ACLIC for medium/large size scripts. LLVM will simplify even medium/large size scripts. LLVM will simplify even more the problem once the system will be more the problem once the system will be operational.operational.



root > .x script.C

root > DoSomething(…);

root > .x script.C++

root > .x script.C+

gROOT->ProcessLine(“.L script.C+”);

gROOT->ProcessLine(“DoSomething(…)”);

execute file script.C

execute function DoSomething

compile file script.Cand execute it

compile file script.Cif file has been

modified.execute it

same from compiledor interpreted code

Interpreter & Compiler Interpreter & Compiler integrationintegration

1111

ROOT Math/Stat LibrariesROOT Math/Stat Libraries

TMinuitTMinuitTFumiliTFumili

Minuit2Minuit2

Fitting and Minimization

Linear FitterLinear Fitter

QuadpQuadp

Extra algorithmsExtra algorithmsExtra Math Extra Math functionsfunctions

GSLGSL

MathMore

Random NumbersRandom Numbers

MLPMLP

Statistical Libraries

Limit classesLimit classes

YYMath Math

functionsfunctions

Basic Basic algorithmsalgorithms

Math InterfacesMath Interfaces

MathCore

TComplexTComplex

TRandomTRandom

Fitting Fitting ClassesClasses

FFTWFFTW

Extra Libraries

UnuranUnuran

FoamFoamSpectruSpectrumm

SMatrixSMatrixLinear Algebra

TMatrixTMatrix

GenVectorGenVector

Physics Vectors

TVector3/TVector3/TLVTLV

RooFitRooFit TMVATMVA

High Level Analysis Libraries

RooStatsRooStats

ROOT: design,development, evolutionROOT: design,development, evolution

The new CERNLIB/M

ATHLIB

The new CERNLIB/M

ATHLIB

With better

algorithms

With better

algorithms

René Brun, ACAT2010, René Brun, ACAT2010, JaipurJaipur

ROOT 2-D ROOT 2-D graphicsgraphics

based on 10 years experience with PAW and based on 10 years experience with PAW and 15 years of operation with ROOT itself.15 years of operation with ROOT itself.

zillions of tiny new features always zillions of tiny new features always requested by usersrequested by users

works in batch (canvas.works in batch (canvas.rootroot, canvas., canvas.pdfpdf, , canvas.canvas.gifgif,..),..)

object editorsobject editors

C++ script generated from canvas C++ script generated from canvas (TObject::SavePrimitive)(TObject::SavePrimitive)


2-D Graphics2-D Graphics

1313

New functions New functions added at each added at each new release.new release.

Always Always requests for requests for new styles and new styles and new ways to new ways to display data.display data.

ps,pdf,svg,gif,ps,pdf,svg,gif,

jpg,png,c,root, jpg,png,c,root, etcetc

Move to GL ?

ROOT: design,development, evolutionROOT: design,development, evolution René Brun, ACAT2010, René Brun, ACAT2010, JaipurJaipur

ROOT: design,development, evolutionROOT: design,development, evolution 1414 René Brun, ACAT2010, René Brun, ACAT2010, JaipurJaipur

Image processingImage processing


GUIGUIPortable GUI with all widgets found in Portable GUI with all widgets found in modern UI toolkits.modern UI toolkits.

Many enhancements in the GUI classes: Many enhancements in the GUI classes: browser, html browser, TABs, EVE widgets, browser, html browser, TABs, EVE widgets, FitPanel.FitPanel.

GUI builder with C++ code generator. GUI builder with C++ code generator.

Session recorder tool (class Session recorder tool (class TRecorder)TRecorder)can store and replay a GUI session:can store and replay a GUI session:All mouse eventsAll mouse eventsKeyboard input, including macro executionKeyboard input, including macro execution

QT interfaces: a big pain, difficult to QT interfaces: a big pain, difficult to maintain with the successive versions of Qt.maintain with the successive versions of Qt.


GUIGUI


ROOT: design,development, evolutionROOT: design,development, evolution 1818

GUIGUIOne of the most demanding areasOne of the most demanding areas

Must follow technology and not standard Must follow technology and not standard systems (gks -> phigs ->Motif ->Qt ->?)systems (gks -> phigs ->Motif ->Qt ->?)

GUI based on OpenGL too !GUI based on OpenGL too !

Plug-ins must work on the web (but 2-d and 3-d Plug-ins must work on the web (but 2-d and 3-d graphics too!)graphics too!)

GUI builder extensions (code generators)GUI builder extensions (code generators)

GUI must be scriptable and theme-ableGUI must be scriptable and theme-able

HELP + language internalisationHELP + language internalisation

Context menus help access to documentationContext menus help access to documentation


ROOT 3-D ROOT 3-D graphicsgraphics

OpenGL based: graphics acceleratorsOpenGL based: graphics accelerators

detector geometry visualizationdetector geometry visualization

event displays (event displays (EVEEVE))

interaction between GL view and interaction between GL view and original objects (picking/editing)original objects (picking/editing)


3-D 3-D GraphicsGraphics

Hig

hly o

ptim

ized G

L view

s H

igh

ly op

timized

GL

views

inin T

Pad

TP

ad

the G

L view

erth

e GL

viewer

2020ROOT: design,development, evolutionROOT: design,development, evolution

Mathematical

objects

Mathematical

objectsDetector hitsDetector hits



Graphics (2D-3D)Graphics (2D-3D)

“SURF”“LEGO”

TF3

TH3

TGLParametric


The Geometry Package The Geometry Package TGEOTGEO

The The TGeoTGeo classes are now stable (thanks classes are now stable (thanks ALICEALICE).).

Can work with different simulation engines Can work with different simulation engines (G3,G4,Fluka) (See (G3,G4,Fluka) (See VVirtual irtual MMonte onte CCarlo)arlo)

G3G3->->TGeoTGeo, , G4G4->->TGeoTGeo, , TGeoTGeo>>GDMLGDML

Used in online systems and reconstruction Used in online systems and reconstruction programsprograms

Built-in facilities for alignmentBuilt-in facilities for alignment

Impressive gallery of experiments (35 Impressive gallery of experiments (35 detectors in detectors in $ROOTSYS/test/stressGeometry$ROOTSYS/test/stressGeometry))



OpenGLOpenGL

see $ROOTSYS/tutorials/geom


Event Event DisplayDisplay: : EVEEVEa GL-based package for event displays.a GL-based package for event displays.

Developed in collaboration with Developed in collaboration with ALICEALICE ((AliEveAliEve) and ) and CMSCMS ( (FireWorksFireWorks).).

Provides all the GUI widgets, browsers, GL Provides all the GUI widgets, browsers, GL infrastructure (far better than the old infrastructure (far better than the old OpenInventor).OpenInventor).

Used now by many experiments (see eg Used now by many experiments (see eg FAIRROOTFAIRROOT, , ILCROOTILCROOT) to display raw data, MC ) to display raw data, MC events or detector oriented visualization.events or detector oriented visualization.


PROOFPROOFMassive data analysis with parallel systems. Now Massive data analysis with parallel systems. Now in production in many places . Growing success.in production in many places . Growing success.

A lot of developments for a better integration A lot of developments for a better integration with dataset management systems and to with dataset management systems and to support priority and accounting management.support priority and accounting management.

PROOF-LITE is a mini version to run on a multi-PROOF-LITE is a mini version to run on a multi-core.core.


ROOT: one core only with PROOF

with PROOF-LITE


IO and TreesIO and TreesThey are the most important They are the most important components of the system because they components of the system because they have a deep impact on the experiment have a deep impact on the experiment frameworks and their data handling frameworks and their data handling strategy.strategy.

Shaping these 2 systems has been a Shaping these 2 systems has been a long process.long process.



Memory <--> TreeMemory <--> TreeEach Node is a branch in the Each Node is a branch in the

TreeTree0

12

34

56

789

1011

1213

1415

1617

18T.Fill()

T.GetEntry(6)

T

Memory

Automatic branch Automatic branch creation from object creation from object

modelmodel

float a;int b;double c[5];int N;float* x; //[N]float* y; //[N]Class1 c1;Class2 c2; //!Class3 *c3; std::vector<T>;std::vector<T*>;TClonesArray *tc;

float a;int b;double c[5];int N;float* x; //[N]float* y; //[N]Class1 c1;Class2 c2; //!Class3 *c3; std::vector<T>;std::vector<T*>;TClonesArray *tc;


branch buffersbranch buffers

branches created from a branches created from a dynamic listdynamic list

aa

bb

cc

dd

ee

aaaa

aaaa

aa

Each named member of a TList

Is a top level branch

Each named member of a TList

Is a top level branch


Top level branches Top level branches created by usercreated by user

aa

bb

cc

dd

ee

Top level branchesAre created manually

Top level branchesAre created manually


Branches created from Branches created from STL collectionsSTL collections

std::vector<T>

std::vector<T>

Use split=199 to reduce

The number of branchesIn case of

polymorphic collections

Use split=199 to reduce

The number of branchesIn case of

polymorphic collections

std::vector<T*>

std::vector<T*>

std::vector<T*>

std::vector<T*>

TClonesArray

TClonesArray

std::vector<T>

std::vector<T>

split = 0

split >0

split <100

split =199

split >0


Trends in Tree Trends in Tree structurestructure

Maximum split level, ie more branches is Maximum split level, ie more branches is betterbetter

eg, Atlas 9785, CMS 1436, LHCb 1167, Alice eg, Atlas 9785, CMS 1436, LHCb 1167, Alice >1000>1000

Many branches is good for the future Many branches is good for the future because branch reading can be executed because branch reading can be executed in parallel.in parallel.

many branches is better because the access many branches is better because the access granularity is better for fast selections.granularity is better for fast selections.


ObjectWise/MemberWise ObjectWise/MemberWise StreamingStreaming

abcd

abcd

a1b1c1d1a2b2c2d2…anbncndn

a1b1c1d1a2b2c2d2…anbncndn

a1a2..anb1b2..bnc1c2..cnd1d2..dn

a1a2..anb1b2..bnc1c2..cnd1d2..dn

a1a2…ana1a2…an

b1b2…bnb1b2…bn

c1c2…cnc1c2…cn

d1d2…dnd1d2…dn

3 modes to stream

an object

3 modes to stream

an object

object-wise to a buffer

member-wise to a buffereach m

ember to a buffer

member-wise gives

better compressio

n

member-wise gives

better compressio

nRené Brun, ACAT2010, René Brun, ACAT2010,

JaipurJaipur3333ROOT: design,development, evolutionROOT: design,development, evolution

SupportedSupported PlatformsPlatforms

• Linux (RH, SLC4, SLC5, Suse, Debian, Linux (RH, SLC4, SLC5, Suse, Debian, Ubuntu)Ubuntu)– gcc3.4,…, gcc4.4 (32 and 64 bits)gcc3.4,…, gcc4.4 (32 and 64 bits)– icc10.1, icc11.1icc10.1, icc11.1

• MacOS X (Intel/PPC, 10.4, 10.5, 10.6)MacOS X (Intel/PPC, 10.4, 10.5, 10.6)– gcc4.0.1, gcc4.2.1,gcc4.4.1, llvm-g++gcc4.0.1, gcc4.2.1,gcc4.4.1, llvm-g++– icc10.1, icc11.1icc10.1, icc11.1

• Windows (XP, Vista)Windows (XP, Vista)– VC++7.1, VC++9VC++7.1, VC++9– Cygwin gcc3.4, 4.3Cygwin gcc3.4, 4.3

• Solaris + OpenSolarisSolaris + OpenSolaris– CC5.2CC5.2– gcc3.4, gcc4.4gcc3.4, gcc4.4


Support for many compilers adds

robustness at a minor development cost.

It facilitates the migration to new compiler versions.

Support for many compilers adds

robustness at a minor development cost.

It facilitates the migration to new compiler versions.


Distribution StatsDistribution Stats


More than 20000 users35000 FTP-only

distributionsof binaries per month.Many more from SVN

sourceor AFS access

More than 20000 users35000 FTP-only

distributionsof binaries per month.Many more from SVN

sourceor AFS access


How did we reach How did we reach the current the current situationsituation

Rien ne sert de courir, il faut partir a pointRien ne sert de courir, il faut partir a pointJean de la FontaineJean de la Fontaine


one cannot forget one cannot forget the pastthe past

• Systems like Systems like ROOT ROOT and and G4 G4 have been have been strongly influenced par their predecessors, strongly influenced par their predecessors, by politics, by sociology.by politics, by sociology.

• Like Jumbo jets, large systems that required Like Jumbo jets, large systems that required a lot of manpower evolve with time with the a lot of manpower evolve with time with the pros and cons of their inertia.pros and cons of their inertia.

• Once a system is massively used, people Once a system is massively used, people expect backward and forward compatibility.expect backward and forward compatibility.

• However ROOT in 2010 is very much different However ROOT in 2010 is very much different of ROOT in 1995 or 2000 or 2005of ROOT in 1995 or 2000 or 2005


Software in HEPSoftware in HEP


1970 1980 1990 2010

cernlib ===============-------------------------……………….. . . . . .

paw---====================-----------------… . . . .

zebra---=================------------------------…… . .geant1,2

==--hbook===============================------------------------………….. .

geant4----===================

piaf====

geant3 ======================------------------……… . . .

ROOT---===================

JAS--===----…. . .

most systems used over 2,3 or 4 decades

most systems used over 2,3 or 4 decades

Main parameters Main parameters who shaped ROOTwho shaped ROOT• Long experience with Long experience with GEANT GEANT and and PAWPAW

• Reuse of componentsReuse of components

• Concept of packages within a Concept of packages within a frameworkframework

• Optimal for batch, optimal for Optimal for batch, optimal for interactivityinteractivity

• User support quasi onlineUser support quasi online


CERNLIBCERNLIB• 19701970: main.f links with cernlib.a where cernlib : main.f links with cernlib.a where cernlib

is a collection of independent subroutines.is a collection of independent subroutines.

• 19751975: subroutines are grouped into packages : subroutines are grouped into packages identified by the letters of the alphabet, eg identified by the letters of the alphabet, eg Y250Y250

• 19801980: cernlib accepts large packages, eg : cernlib accepts large packages, eg hbook, minuit,gd3hbook, minuit,gd3

• 19851985: cernlib is a collection of large packages, : cernlib is a collection of large packages, mathlib, kernlib,packlib,graflib,pawlib,geant3mathlib, kernlib,packlib,graflib,pawlib,geant3


hbook : 1973 hbook : 1973 20xy20xy

• first large package in first large package in cernlibcernlib

• histogram-oriented system including histogram-oriented system including visualization (from line-printer to graphics) and visualization (from line-printer to graphics) and persistency.persistency.

• interface with interface with MinuitMinuit and and FumiliFumili

• interface with interface with zbook in zbook in 1975, then 1975, then zebrazebra in in 19831983

• extended in 1985 to support extended in 1985 to support row-wise ntuples row-wise ntuples and and column-wise ntuplescolumn-wise ntuples in 1990. in 1990.

• the heart of the heart of PAWPAW


geant3 : 1980-geant3 : 1980-20xy20xy

• successor of successor of geant1geant1 (1975), (1975), geant2geant2(1978)(1978)

• includes a powerful geometry package includes a powerful geometry package used to model all existing HEP detectors.used to model all existing HEP detectors.

• interfaces with external hadronic shower interfaces with external hadronic shower packages like packages like tatinatatina,,gheishagheisha,,flukafluka..

• includes visualisation: detector cross-includes visualisation: detector cross-sections, event displays.sections, event displays.

• includes IO for geometry and events includes IO for geometry and events ((zebrazebra))


PAW : 1984PAW : 198420xy20xy• started by a committee (design for the past)started by a committee (design for the past)

• quickly reshaped in 1985 with quickly reshaped in 1985 with KUIPKUIP, , HIGZHIGZ, , ntuplesntuples

• query processor: query processor: ntuple/plot 10.pxntuple/plot 10.px

• analysis patterns indicates row-analysis patterns indicates row-wisewisecolumn-wise storage and access.column-wise storage and access.

• developments using developments using zebra RZ IO zebra RZ IO systemsystem

• more and more graphics optionsmore and more graphics options


PIAF : 1992-1994PIAF : 1992-1994• PAW PPAW Parallel arallel IInteractive nteractive AAnalysis nalysis

FFacilityacility

• very important step to understand the very important step to understand the issues with parallelism with the IO sub issues with parallelism with the IO sub system and the implications on the system and the implications on the users analysis code.users analysis code.

• very important to understand the very important to understand the balance between CPU, disk IO, network balance between CPU, disk IO, network IO.IO.


software crisissoftware crisis1992-19941992-1994

• It was clear in 1992 that our Fortran-based It was clear in 1992 that our Fortran-based systems will be impacted by new languages systems will be impacted by new languages and new techniques.and new techniques.

• Many people thought that Fortran was the Many people thought that Fortran was the future.future.

• MOOSE MOOSE project investigating languages (C++, project investigating languages (C++, Eiffel) and techniques (UML, Rose, OO-design)Eiffel) and techniques (UML, Rose, OO-design)

• commercial software or home-grown dilemmacommercial software or home-grown dilemma

• OODBMS OODBMS at the horizonat the horizon


software crisissoftware crisis1992-1994 -more1992-1994 -more

• many computer scientists in the discussionmany computer scientists in the discussion

• most of them with no experiencemost of them with no experience

• existing experience with large software existing experience with large software neglected as well as feedback coming from neglected as well as feedback coming from experienced physicistsexperienced physicists

• west coast syndromewest coast syndrome

• Babar as guinea pig (C++ and OODBMS)Babar as guinea pig (C++ and OODBMS)

• Revolution, not EvolutionRevolution, not Evolution


Towards Towards ObjectivityObjectivity

1994-20001994-2000• Gurus predict that Gurus predict that OODBMS OODBMS will replace will replace RDBMS RDBMS

systems in 2000.systems in 2000.

• Post-mortem influence of defunct Post-mortem influence of defunct SSCSSC

• dream of a central data base accessed world-dream of a central data base accessed world-wide at the bit level for writing and reading.wide at the bit level for writing and reading.

• Don’t worry about containers structure, CPU Don’t worry about containers structure, CPU and network performance; the technology will and network performance; the technology will solve your problems.solve your problems.

• Most physicists trust gurus for reaching NirvanaMost physicists trust gurus for reaching Nirvana


1994-20001994-2000Lost in the clouds before Lost in the clouds before the grid and the cloudsthe grid and the clouds

• Experienced physicists busy with LHC exp design.Experienced physicists busy with LHC exp design.

• It took many years (of frustration) to move them It took many years (of frustration) to move them from good old from good old Fortran Fortran to the magic of to the magic of OO OO and and C++C++

• catastrophic catastrophic LCB LCB (LHC Computing Board) (LHC Computing Board) committee with always the same referees with no committee with always the same referees with no experience in the new technology. As a experience in the new technology. As a consequence the LCB has no choice but trusting consequence the LCB has no choice but trusting the gurus.the gurus.

• Despite early signs of problems, gurus and Despite early signs of problems, gurus and referees persist in their recommendations.referees persist in their recommendations.


1995-19981995-1998ROOT is born in the ROOT is born in the

meadowsmeadows• While many people were actively working in While many people were actively working in

committees and expecting magic from committees and expecting magic from ObjectivityObjectivity, , a small group of anarchists is exiled in the CERN 2 a small group of anarchists is exiled in the CERN 2 meadows.meadows.

• free of committees and not spending time only with free of committees and not spending time only with UML diagrams , but concentrating their work on UML diagrams , but concentrating their work on key issues and delivering frequent releases of key issues and delivering frequent releases of prototypes tested in real life experiments (eg prototypes tested in real life experiments (eg NA49).NA49).

• While the first ROOT prototype was a C++ version While the first ROOT prototype was a C++ version of PAW, of PAW, it was rapidly realized that the key problem it was rapidly realized that the key problem was with a general object persistency solution was with a general object persistency solution much different than the Objectivity solutionmuch different than the Objectivity solution..


Original Objectives Original Objectives in 95in 95

• Continue the tradition started with Continue the tradition started with CERNLIB CERNLIB to to develop packages targeted to the needs of our develop packages targeted to the needs of our experiments and with a long life time.experiments and with a long life time.

• Eg Eg hbookhbook (36 years), (36 years), geant3 geant3 (28), (28), pawpaw(25): still (25): still in use!in use!

• Focus on data analysis (PAW in the oo world)Focus on data analysis (PAW in the oo world)

• OOpen pen SSource model to get contributions and ource model to get contributions and feedback as soon as possible.feedback as soon as possible.

• First ROOT prototype for NA49 in 1995, rapidly First ROOT prototype for NA49 in 1995, rapidly followed by many more adopters. Bottom-up followed by many more adopters. Bottom-up (user-driven) instead of top-down (committee) (user-driven) instead of top-down (committee) approach. Users workshops.approach. Users workshops.


ROOT in 1995ROOT in 1995• During the first 3 months of 1995 we During the first 3 months of 1995 we

prototyped several versions of the histograming prototyped several versions of the histograming system, including a version based on templates.system, including a version based on templates.

• In October 1995 we presented a first prototype In October 1995 we presented a first prototype with histograms, functions, graphics (a la PAW).with histograms, functions, graphics (a la PAW).

• This prototype included a command line This prototype included a command line interface supporting very primitive calls to interface supporting very primitive calls to ROOT classes member functions.ROOT classes member functions.

• This prototype generated a lot of attention but This prototype generated a lot of attention but also fears.also fears.


ROOT in 1996ROOT in 1996• Instead of our very basic interpreter, we Instead of our very basic interpreter, we

decided to use decided to use CINT CINT developed since 1991 by developed since 1991 by Masa GotoMasa Goto from HP Japan. from HP Japan.

• We used CINT not only as an interpreter, but We used CINT not only as an interpreter, but also as a parser for the ROOT header files to also as a parser for the ROOT header files to generate automatically a dictionary such that:generate automatically a dictionary such that:– one could call class member functions from C++ one could call class member functions from C++

interpreted scriptsinterpreted scripts– one could stream objects to a buffer with explicit one could stream objects to a buffer with explicit

streamers.streamers.

• ROOT version 1 included also many more ROOT version 1 included also many more classes and better graphics.classes and better graphics.


ROOT in 1997ROOT in 1997• The IO Streamer functions generated by The IO Streamer functions generated by rootcint rootcint

were quite simple to understand and were were quite simple to understand and were efficient.efficient.

• However, we realized soon that classes were However, we realized soon that classes were evolving. We had to introduce class versions and evolving. We had to introduce class versions and the possibility for users to write their own the possibility for users to write their own custom streamers where they could take into custom streamers where they could take into account the class schema evolution.account the class schema evolution.

• At the same time, we made At the same time, we made rootcint rootcint more general more general such that it could parse many more classes, in such that it could parse many more classes, in particular non-ROOT classes developed by our particular non-ROOT classes developed by our experiments guinea-pigs.experiments guinea-pigs.


ROOT in 1998ROOT in 1998• We released version 2 with more features We released version 2 with more features

(IO, math, graphics and more sophistication (IO, math, graphics and more sophistication in in TreesTrees, our main container for event data., our main container for event data.

• In fall 1998 In fall 1998 ROOT was selected by FNAL & ROOT was selected by FNAL & RHICRHIC as: as:– the persistency solutionthe persistency solution– the data analysis solutionthe data analysis solution

• This decision generated a big chock-wave in This decision generated a big chock-wave in the establishment. It also generated more the establishment. It also generated more work for us.work for us.


ROOT in 1999ROOT in 1999• The ROOT The ROOT Streamers Streamers were very efficient. However were very efficient. However

the library containing the Streamers was required the library containing the Streamers was required when reading the data sets.when reading the data sets.

• While this requirement was not a problem when While this requirement was not a problem when reading complex object collections and the classes reading complex object collections and the classes used to write were necessary in the read phase, it used to write were necessary in the read phase, it was a serious drawback for files with simple was a serious drawback for files with simple Trees Trees like PAW ntuples.like PAW ntuples.

• At this point we implemented an automatic system At this point we implemented an automatic system to stream data using the information in the to stream data using the information in the dictionary instead of using the generated C++ dictionary instead of using the generated C++ StreamersStreamers


StreamerInfoStreamerInfo• With the automatic streaming, we also With the automatic streaming, we also

saved to the ROOT file the meta data saved to the ROOT file the meta data information describing all classes such information describing all classes such that ROOT files are now that ROOT files are now self-describingself-describing..

• With time and in the following years we With time and in the following years we continued to optimize this system:continued to optimize this system:– to support the full C++ language in to support the full C++ language in

general.general.– to support more sophisticated cases of to support more sophisticated cases of

class schema evolution.class schema evolution.


ROOT vs ROOT vs ObjectivityObjectivity

• In 99.99% cases people In 99.99% cases people write a file once write a file once and and read it many times. No need for a locking read it many times. No need for a locking process when reading.process when reading.

• ROOT files are ROOT files are self-described self-described and and independent of the location where they are independent of the location where they are processed.processed.

• ROOT files are ROOT files are compressedcompressed. Typical . Typical compression factors for LHC experiments is compression factors for LHC experiments is between 3 and 5.between 3 and 5.

• ROOT files are ROOT files are machine independentmachine independent


ROOT vs ROOT vs Objectivity 2Objectivity 2

• ROOT does not stream transient data membersROOT does not stream transient data members

• ROOT can stream collections ROOT can stream collections member-wisemember-wise, , optimizing both speed and disk space.optimizing both speed and disk space.

• ROOT can read files containing different class ROOT can read files containing different class versions: automatic class versions: automatic class schema evolutionschema evolution..

• ROOT has a complete ROOT has a complete query/visualization systemquery/visualization system..

• ROOT files can be processed in parallel with ROOT files can be processed in parallel with PROOFPROOF

• ROOT was processing files orders of magnitude ROOT was processing files orders of magnitude fasterfaster

• ROOT has a powerful ROOT has a powerful caching caching mechanism allowing mechanism allowing to read files very efficiently in wide area networksto read files very efficiently in wide area networks


DirectionsDirections


Our ChallengesOur Challenges

• Keep a stable core, improving robustnessKeep a stable core, improving robustness

• At the same time follow the technology changesAt the same time follow the technology changes– Distributed and parallel computingDistributed and parallel computing– Multi-core systems, GPU processing (OpenCL)Multi-core systems, GPU processing (OpenCL)– New version of C++, etcNew version of C++, etc– CINT to LLVMCINT to LLVM

• Improve algorithms (quality and speed)Improve algorithms (quality and speed)

• Improve doc , user interface, user supportImprove doc , user interface, user support

• Add support for new communities, LC,etcAdd support for new communities, LC,etc

• And, of course, provide a robust and efficient And, of course, provide a robust and efficient data analysis tool for the LHC experimentsdata analysis tool for the LHC experiments..



Will have to deal with many shared libs

Only a small fraction of code used

Problem Problem decompositiondecomposition


ROOTROOTinin

19951995

ROOTROOTinin

20102010

10 shared libs10 shared libs

200 classes200 classes

100 shared libs100 shared libs

2000 classes2000 classes

PAW modelPAW model

Plug-in managerPlug-in manager

Some FactsSome Facts


• many_x.f -> many_x.o many_x.f -> many_x.o

• many_x.o + some_libs.amany_x.o + some_libs.a

• + many_libs.so -> x.exe+ many_libs.so -> x.exe

x.exe

Input.dat(free format)

Output.log

Objectivity?ROOT ?

Zebra fileRFIO

Executable module Executable module in 2000in 2000


x.exe

Config.C(interpreter)

Output.log

ROOT files

ROOT files

OracleMysql

Dcachecastor

xrootd

a.so b.so

u.so

Shared libs dynamically

loaded/unloaded

by the plug-in manager

LAN



x.exe

Config.C(interpreter)

Output.log

ROOT files

ROOT files

OracleMysql

Cache

Proxy

manager

http

a.cxx

http

b.cxx

http

u.cxx

Local shared libs dynamically

Compiled/loaded/unloaded

from a source URL

WANx.exex.exex.exe

ROOT bufferslocal cache

Multi-threaded

Core executor

Executable module in Executable module in 2020 ?2020 ?

The LLVM systemThe LLVM system• We are having big hopes with We are having big hopes with LLVM LLVM (open source (open source

project with people from Apple, Adobe, Google, project with people from Apple, Adobe, Google, etc).etc).

• LLVM is a LLVM is a gcc compatible compilergcc compatible compiler, parser and , parser and JIT JIT ((JJust ust IIn n TTime) compiler.ime) compiler.

• Our idea is that the interpreter will be the compiler Our idea is that the interpreter will be the compiler itself (itself (C++0x compliantC++0x compliant) and offering many ) and offering many additional possibilities like quality and speed for additional possibilities like quality and speed for important classes like important classes like TFormulaTFormula, , TTreeFormulaTTreeFormula, etc, etc

• We have a working prototype today, but a fully We have a working prototype today, but a fully operational use of LLVM in ROOT is probably 2 operational use of LLVM in ROOT is probably 2 years away.years away.



• The GRID is a parallel engine. However it is The GRID is a parallel engine. However it is unlikely that you will use the GRID software unlikely that you will use the GRID software on your 32-core laptop.on your 32-core laptop.

• There is a lot of work to be done within ROOT There is a lot of work to be done within ROOT to use parallelism internally or making ROOT-to use parallelism internally or making ROOT-based applications more easily parallelizable.based applications more easily parallelizable.

• Think Top->Down and Bottom->UpThink Top->Down and Bottom->Up

Coarse grain: job, event, trackFine grain

vectorization

Design for Design for ParallelismParallelism


100,000 computers

in 1000 locations

5,000 physicists

in 1000 locations

WAN

LAN

Towards a distributed Towards a distributed systemsystem


Few big users submittingmany long jobs (Monte Carlo,

reconstruction)They want to run many jobs in

one week

Many users submittingmany short jobs (physics

analysis)They want to run many jobs in one hour or less

GRID: Users GRID: Users profileprofile

Difficult to fit all Difficult to fit all sizessizes

• Like the Taj Mahal, ROOT has many Like the Taj Mahal, ROOT has many different facets and the visitor may different facets and the visitor may have different impressions depending have different impressions depending on his viewpoint and time of day.on his viewpoint and time of day.

• We try to do our best to satisfy very We try to do our best to satisfy very heterogeneous users. Some use ROOT heterogeneous users. Some use ROOT as a framework, others as a simple tool, as a framework, others as a simple tool, others as a set of libraries.others as a set of libraries.


Taj Mahal & ROOTTaj Mahal & ROOT• The Taj Mahal combines elements from The Taj Mahal combines elements from

PersianPersian, , IndianIndian, , Islamic Islamic architectural styles, architectural styles, an integrated complex of structures. an integrated complex of structures. Building began around Building began around 1632 1632 and was and was completed around completed around 16531653, and employed , and employed thousands of artisans and craftsmen.thousands of artisans and craftsmen.

• ROOT combines ideas and techniques that ROOT combines ideas and techniques that proved to be successful in the proved to be successful in the geant3geant3, , zebra zebra and and PAW PAW eras. Building began in eras. Building began in 1995 1995 and in and in 2010 2010 is still under construction by many is still under construction by many motivated people.motivated people.


Many ThanksMany Thanks• I have been pleased to follow the ACAT I have been pleased to follow the ACAT

meetings since the beginning in Lyon meetings since the beginning in Lyon (1989 ?)(1989 ?)

• Many thanks to my colleagues organizing Many thanks to my colleagues organizing this event.this event.

• This is my last talk at ACAT as ROOT This is my last talk at ACAT as ROOT project leader. I take this opportunity to project leader. I take this opportunity to thank all my colleagues who are working thank all my colleagues who are working hard in this project and I wish them a hard in this project and I wish them a healthy future.healthy future.


Input/Output: Major StepsInput/Output: Major Steps


parallel mergeparallel merge

generalized schema evolution

generalized schema evolution

member-wise streamingfor STL collections<T*>member-wise streamingfor STL collections<T*>

member-wise streamingfor TClonesArray

member-wise streamingfor TClonesArray

automatic streamers from dictionary with StreamerInfosin self-describing files

automatic streamers from dictionary with StreamerInfosin self-describing files

streamers generatedby rootcint

streamers generatedby rootcint

User written streamersfilling TBuffer

User written streamersfilling TBuffer


CMZ…………………………………CVS………………………………………………………….SVN……………

Major Technical StepsMajor Technical Steps

7474

P R O O F

ReflexFrom SEAL to

ROOT


LLVMLLVM

CINT7CINT7

GL/EVEGL/EVE

RooFitTMVARooFitTMVA

MathLibsMathLibs

ReflexFrom SEAL to ROOT

ReflexFrom SEAL to ROOT

Trees automatic

split

Trees automatic

split

GUI based on

signal/slots

GUI based on

signal/slots

I/O based on

dictionaries in memory

I/O based on

dictionaries in memory

Graphics based on TVirtualX

Graphics based on TVirtualX

CINTCINT


Testing, Nightly Testing, Nightly buildsbuilds

• Several thousand tests run every night on many Several thousand tests run every night on many platforms.platforms.

• Code checking with a coding convention rule checkerCode checking with a coding convention rule checker

• Code coverage to be improved, but already much Code coverage to be improved, but already much better than any other package reviewed.better than any other package reviewed.

• Frequent Frequent valgrindingvalgrinding and profiling and profiling


PROOF-LITEPROOF-LITE

7676

PROOF optimized for single many-core machinesZero configuration setup. No config files and no daemonsWorkers are processes and not threads for robustnessLike PROOF it can exploit fast disks, SSD’s, lots of RAM, fast networks and fast CPU’s Once your analysis runs on PROOF Lite it will also run on PROOF (with exactly the same user code)


ROOT: one core only

with PROOF-LITERené Brun, ACAT2010, René Brun, ACAT2010,

JaipurJaipur


• many_x.f -> many_x.o many_x.f -> many_x.o

• many_x.o + many_libs.a -> x.exemany_x.o + many_libs.a -> x.exe

x.exe

Input.dat(free format)

Output.logportableZebra file


Design, Development and Evolution of the ROOT system Rene Brun February 21 2010 ACAT2010 Jaipur...

Documents

Transcript of Design, Development and Evolution of the ROOT system Rene Brun February 21 2010 ACAT2010 Jaipur...