Blueprint RTAG Status Torre Wenaus, BNL/CERN SC2 Meeting July 5, 2002.

Blueprint RTAG Status

Torre Wenaus, BNL/CERN

SC2 Meeting

July 5, 2002

SC2 meeting, July 5 2002 Slide 2 Torre Wenaus, BNL/CERN

Blueprint RTAG Status

Current RTAG page: Linked from the applications page

Activity so far: Meetings Wed June 12, Fri June 14

‘Opening statement’ type talks and discussions Between these meetings: a long email dialogue on architecture

and design Particularly on the role and architecture of ROOT, following on

from a proposal made by Rene on June 14 All day meeting Wed July 3

Response/reaction to Rene’s proposal and the email dialogue Some resolved issues and decisions Domain decomposition On to developing the architectural blueprint


The main software areas

GRID

middleware

RDBMS

run/file

catalogs

Object

persistencyv

2-d, 3-d

graphics

GUI

Toolkits

Math Libs

Statistics

Detector

Geometry

Event

Generators

Dectector

Simulation

Object

persistency

Histograming

Fitting

Event Models

Folders

Event Display

Ntuple

analysis

Interpreters

DAQ

Online

System

services

ETC...

Rene Brun


Framework with Object bus

Object bus: Object dictionary

Data Interface (I/O): Functional Interface

User

Applications

Higher level

framework

services

Higher level

framework

services

Experiment

framework

Higher level

framework

services

Higher level

framework

services

Higher level

framework services

User

Applications

Rene Brun


Python as a Software Bus

Python is an object-oriented scripting language commonly used in a wide variety of domains

But, it could also be seen as framework where you can plug easily “extension modules” in binary form implemented using other languages.

Very easy and natural to interface to C++ classes (C++ API) Python should only be the “glue” between modules developed in C+

+ or other languages The interface (API) for Python extension modules is quite simple

and at the same time flexible

Pere Mato


GUI

Python as a Software Bus

Python

mathmath

shellgaudipython

DatabaseEDG API

GUI

Very rich set of Python standard modules

Several GUI toolkits

XML

Very rich set specialized generic modules

GaudiFramework

rootpython

RootClasses

PVSS

JPE

JavaClasses

LHC modules

Gateways to

other frameworks

Pere Mato


Architecture: Devil is in the Details

Build independent components: Avoid Dependencies among components at the same level Gratuitous and exaggerated re-use

One hammer does not fit all screws global states (even cout) Exposure of internal relationships (a->b()->c(i)->d(“b”)) assumptions on higher level behavior (lent pointers) Interfaces that force your environment on user code

Balance inheritance (white box) vs composition (black box) Distinguish Framework API, Client API and User API

These are Architectural issues NOT coding guidelinesI do not mind of “#define int float” in your .cc, I mind if in a .h

Vincenzo Innocente


Toward a Project Praxis

Define the global software model Granularity, role and nature of “Modules”

Physical vs logical modules

(yesterday at CMS plenary M.Livny concluded asking for staticly linked, check-pointable executables…)

Reuse model of sub-components Which “glues” have to be used, where and how

Define THE set of basic components Agree on Metrics to measure modularity

Not only Frameworks, also applications based on them

Vincenzo Innocente


Rene’s Proposal

The existing set of ROOT libs is the starting core of the LCG

software.

Because the system is already widely distributed and used, it guarantees the initial acceptance

to a wide community.

We invite architects and key developers to review the current organisation of libraries and to propose an evolution if it proves necessary, keeping in mind that this may affect thousands of

existing users.


LCG Software Architecture and ROOT

Architectural principles that CMS, LHCb, ATLAS want to follow were developed in presentations by Pere and Vincenzo and in a long email dialogue

Exchanges indicated sharply the different views on design between the ROOT/ALICE team and CMS/LHCb/ATLAS

A clear conclusion, agreed in Wed meeting… It is not going to work for ROOT to be taken as the starting

core of LCG software The design differences are too great, and the ROOT team is not

going to redesign ROOT What we know is going to happen is that ROOT will be used heavily

by the LCG software and all four experiments And has been taken directly as the core foundation for one


How will ROOT be used?

While design discussions should continue, we cannot rely on resolving them in order to progress…

We accepted that We are not going to converge anytime soon on a common

approach to architecture and design What we must take up and resolve is working out an

architecturally acceptable way to make use of a big grey (not black) ROOT box

With accommodation on both sides Changes to ROOT library organization? More substantive changes to improve factorization? Dealing constructively with ‘linking in the kitchen sink’ issues Agreed metrics meaningful for usability, maintability, modularity

etc.


How will ROOT be used?

LCG software developed as a ROOT user will draw on a great ROOT strength: users are listened to very carefully!

Much more carefully than software designers proposing major design changes!!

The ROOT team has been very responsive to needs for new and extended functionality coming from the persistency effort

Drawing on ROOT in a user-provider relationship matches much better the reality of the ROOT development model of a very small number of ‘empowered’ developers

The ROOT development team is small and they like it that way

While ROOT will be used at the core of much LCG software for the foreseeable future, we agree there needs to be a line with ROOT proper on one side and ‘LCG software’ on the other.


LCG Software Architecture

With the ‘ROOT relationship’ resolved to be not architecture/design wars to be (endlessly) fought, but rather designing how ROOT will be used in a distinct LCG architecture, we can (and we began to) move on to developing the LCG architectural blueprint


Views I expressed, to no vocal objection

I think that within the LCG software architecture ‘bare ROOT’ should be an integral, trivially accessible part of the architecture

e.g., most obviously, affording the interactive user the ability to easily move between a Python prompt (see coming slide) and a ROOT prompt with access to their object model from ROOT; possible now given the foreign class support in ROOT

As Rene often says: Users vote with their feet The design and evolution of LCG software should be well attuned to

the user, as is ROOT LCG software we develop that does not use ‘bare ROOT’, or ROOT

at all, will live or die on its merits We should build it so that it lives on its merits, not on life support

through architecturally walling off ROOT in some way Plenty of people have bet against ROOT in the past. They have all

been wrong. LCG software should not make this mistake


Interactivity and software buses

Points of agreement A common object dictionary will be a key component at the

foundation of the architecture A Python-based interactive environment and ‘software bus’ will

be part of the architecture ROOT and the ROOTCINT interpreter will be trivially accessible

from the Python environment and vice versa The architecture will provide for access to data objects from

programmatic and interactive environments throughout the system


Python in the architecture

Rene requested expression of his views… Python should not be the primary interactive interface.

Personally I agree; I think that ROOTCINT and Python should essentially be on a level playing field

Will lead to confusion and waste of time by users Actually I think users will insist on easy availability of their preferred

interactive environment, as I expressed on the ‘views’ slide a few slides ago

Python should be available, to take advantage of services for which a Python interface is already available

In the same way, we must encourage cross-language communication Rene in favour of a public presentation of a complete and coherent

framework I think the outcome of this RTAG should be presented clearly in a public

form, eg. in an applications area meeting (with the anticipation of larger than average attendance!)


Notes on Domain Decomposition

- NB user and software provider perspectives on domain

decomposition will be different

- basic layer - software bus / object dictionary

- persistency/data management

- object persistency

- relational cataloging

- event-specific data management

- conditions-specific data management

- event processing framework

- event models

- event generation **defer

- detector simulation **defer

- geometry and materials

- persistent representation

- transient geometry model

- reconstruction

- reconstruction

- scripting (Python) interpreter tool as software bus

- GUI **defer

- NB requirements coming from support for picking

- ability to put any kind of object into a list of primitives

- 2D & 3D graphics **defer

- analysis tools **defer

- ntuple analysis

- math libraries, statistical analysis

- histogramming, fitting

- grid middleware **defer

- meta-services (Pere will elucidate; cf. Gaudi)

- utility and foundation libraries

- system services. OS interaction

- packaging and distribution


Blueprint RTAG plans

Another series of 3 meetings next week Into the specific details of techniques and patterns via proposals

presented as concrete examples Refining the domain decomposition; walking through it and identifying

candidate implementations; interdependencies The following week:

Begin assembling the report Too many absences for meetings

Then more meetings, and on and on… Will invite some speakers (Tony Johnson, There is some skepticism within the RTAG that we will make early Sep (we

will not make early Aug), but this is our objective We recognize the importance of getting this done in good time It has to be done right

I think the RTAG is off to a good start and has made some important, clarifying decisions.

Blueprint RTAG Status Torre Wenaus, BNL/CERN SC2 Meeting July 5, 2002.

Documents

Transcript of Blueprint RTAG Status Torre Wenaus, BNL/CERN SC2 Meeting July 5, 2002.