SEAL Project Overview

SEAL Project Overview

Lorenzo Moneta/ CERN-EP on behalf of the SEAL team

ACAT03

IX International Workshop on Advanced Computing and Analysis Techniques in Physics Research

1-5 December, 2003KEK

ACAT03, 1-5 Dec. 2003

SEAL Project Overview L. Moneta / CERN2

Outline Introduction

– Scope and motivation SEAL Work Packages

– Description – Current status

Future plans and summary

ACAT03, 1-5 Dec. 2003


Project motivation What is SEAL ?

– project in LCG Application Area – started last year after BluePrint RTAG recommendations– common project on core tools and services

» SEAL: Shared Environment for Applications at LHC SEAL aims to

– Provide the software infrastructure, basic frameworks, libraries and tools that are common among the LHC experiments

– Select, integrate, develop and support foundation and utility class libraries

– Develop a coherent set of basic framework services to facilitate the integration of LCG and non - LCG software

Do not re-invent the wheel– Try to use as much as possible of existing software– Develop/adapt/generalize to achieve necessary level of

coherency

ACAT03, 1-5 Dec. 2003


Project scope Foundation Class Libraries

– Basic types (STL, Boost,…), utility libraries, system isolation libraries, domain specific foundation libraries

Mathematical Library– Basic mathematical functions,

linear algebra, fitting and minimization libraries

Basic Framework Services– Component model, reflection, plug-

in management, incident (event) management, distributed computing, grid services, scripting

Basic Framework

Foundation Libraries

Simulation Framework

Reconstruction Framework

Visualization Framework

Applications

. . .

Optional Libraries

OtherFrameworks

Software Structure

ACAT03, 1-5 Dec. 2003


Project Work Packages

Foundation Foundation and Utility Libraries and Plug-in Manager

MathLibs Math Libraries Support and CoordinationDictionary LCG Object DictionaryFramework Component Model and Basic Framework servicesScripting Scripting ServicesGrid Grid ServicesDocumentation Education and Documentation

Not yet active

ACAT03, 1-5 Dec. 2003


Foundation and Utility Libraries Inventory of existing C++ utilities (http://seal.cern.ch/components.html)

– Recommended classes useful for LCG projects and experiments – Large number from Boost (open source project)

» Support for Boost is provided (in contact with developers) Develop in SEAL what is missing in STL and Boost Have a set of low-level, fairly independent class libraries

200 classes originated mainly from classlib (CMS) – Basic abstraction and foundation utilities classes– Timing utilities – Stream-oriented I/O, compression and decompression

High quality, efficient, well tested and documented library – Recommend best practice to LHC developers

http://seal.cern.ch/snapshot/main/components.html




ACAT03, 1-5 Dec. 2003


Plug-in Manager Plug-in Management

– Service in charge of managing, querying, [un]loading plug-ins Low level plug-in management (no framework

constraints)– Ability to create “module” libraries containing “plug-ins”– Ability to instantiate concrete implementations knowing the

plug-in category and the concrete type– Caching information for fast loading

» what modules contain » what kind of plug-ins exist and which categories

Imported from CMS IGuana– Basic concept: advanced object factory

» Interfaces for object instantiation and plug-in provider– Dynamic loading completely orthogonal — and optional!

ACAT03, 1-5 Dec. 2003


Mathematical Libraries GSL (Gnu Scientific Library) evaluation

– Extensive comparison with NagC– Validation and test suite

Re-implementation of MINUIT in C++– Prototype available with Migrad and Minos– Undergoing numerical and performance tests

» Comparison tests with Fortran version (compatibility within errors) Linear Algebra

– Prototype wrapper around GSL matrices. – Comparison of various linear algebra packages (CLHEP, GSL,

[u]CBLAS) Participation in CLHEP project Provide to experiments math and statistics libraries to be

used in analysis, reconstruction and simulation. – support, consultancy, etc..

ACAT03, 1-5 Dec. 2003


Object Dictionary Adding “introspection/reflection” capabilities to C++ Use for persistency and interactivity (e.g. Python) Modern languages provide reflection inherently

– In C++ reflection is very limited (RTTI) The basic C++ concepts shall be supported

– e.g. inheritance, methods, data members, accessibility, templates Seal provides :

– Reflection Builder and Reflection (API interface) packages– Dictionary generator to produce dict. source from C++ files

» Uses gcc-xml + Python script– Dictionary for some standard packages (STL, CLHEP)

Used by POOL (persistency)

ACAT03, 1-5 Dec. 2003


gcc-xml to parse header-files (extension to gcc ) XML file generated with dictionary information Python script to generate dictionary C++ code

– use an additional XML file to customize class selection Compiled and loaded at run-time to create

dictionary – Provide meta information about the object to the user

Dictionary Generator

.xml

.h

.h lcgdict.h.h

gccxml .xml filter+

extengendictpar

ser_dict.cpp ..Dict.somake

selection file

(Python script)#include files

ReflectionLibraries Clients

User

ACAT03, 1-5 Dec. 2003


Framework Services Motivation:

– Allow LCG developers and users to write portable and modular code

– Provide abstractions for modelling software systems Component model definition

– Basic Components» Set of “base classes” and interfaces needed to build more

complex components» Provide standard functionality

instantiation, configuration, lifetime management, etc Users (framework developers) should model their

services using these proposed base classes SEAL provide in addition a set of basic services

implemented following the component model

ACAT03, 1-5 Dec. 2003


SEAL Component Model

Hierarchy of bases classes– Components living in a Context– Services are Components providing their local Context

User classes inherit from Component or Services– Plug-in functionality for free

ContextApplication C1C1 C1C2

Component

Svc2C1C6 C1C7

Svc1C1C3 C1C4

C1C5

Svc1

ServiceC1C7

ACAT03, 1-5 Dec. 2003


Basic Common Framework Services Application Service

– Defines the top level Context– Defines initial set of Components to be loaded in the application

Message Service– Message composition, filtering and reporting

Configuration Service– Management of Component properties and loading configurations– Multiple backends foreseen:

» Gaudi style options, .INI style, CMS style, XML, ... Other services will come later:

– Object Whiteboard (Transient Object repository) – Dictionary Service (Loading Dictionaries in the application)

ACAT03, 1-5 Dec. 2003


Scripting Services Scripting needed for Interactivity, Configuration, Rapid prototyping, Integration

(Component bus), etc. Main scripting language :

– Python

Define guidelines for developing Python bindings : – Various tools available

(Boost.Python, SWIG)– Recommend solutions:

»Boost.Python »LCG Dictionary

GUIPython

mathmathshell

PyGaudi

DatabaseEDG APIGUI

XML

GaudiFramework

PyROOT

RootClasses

PVSS

JPE

JavaClasses

ACAT03, 1-5 Dec. 2003


Scripting Packages PyLCGDict

– Python extension module to the LCG dictionary– Provide access to C++ libraries from Python, “for

free”– Automatically generates Python proxies for C++

objects using dictionary information»Everyone can create binding effortlessly

PyROOT– Python bindings for ROOT classes

»access to ROOT functionality from Python– Binding using the ROOT dictionary (CINT)– Missing gateway ROOT -> Python

ACAT03, 1-5 Dec. 2003


Education/Documentation Produce documentation

– Reference documentation (generated with doxygen) – HowTo pages for the released elements

» Plugin Manager, Dictionary generation, PyROOT, PyLCGDict, etc.

– Release notes Training material (tutorials)

– Help developers incorporating SEAL into the LCG projects and the experiment frameworks

Python courses– Hands-on introduction course to Python programming– Very positive feedback received

ACAT03, 1-5 Dec. 2003


Software Process and QA Design

– Team design sessions– Prototypes using Pythons

Code Review– No formal code reviews– Coding done very often in pairs (XP style)– More than one developer knowledgeable for each package

Testing– More than 200 SEAL unit tests based on CppUnit– All are driven and run automatically by QmTest

Bug reporting and tracking– Savannah Portal (http://savannah.cern.ch/projects/seal)

http://savannah.cern.ch/projects/seal/




ACAT03, 1-5 Dec. 2003


Future plans (1) Foundation libraries (Plug-in Manager)

– Follow Boost developments– Little development planned in SEAL (hash-maps)

MathLibs: – Produce inventory of mathematical functions– Evolve Minuit C++ into a final product– Linear Algebra studies– Generate dictionary library for Python and CINT

Dictionary: – Implementation of new reflection model

» Overcome some existing limitations (typedef,…)– Converge to a common dictionary with CINT ?

ACAT03, 1-5 Dec. 2003


Future plans (2) Framework:

– Develop Whiteboard (object repository) and Dictionary service (loading of dictionary libraries on-demand)

Scripting : – PYLCGDict:

» Migrate much of functional core from C++ to Python » Support more natural Python features (eg iterator protocol)

– PyROOT:» Unify core with PyLCGDict » Develop gateway ROOT -> Python

Grid : – New LCG project (ARDA) is starting– Expect need to interface to some of the ARDA services

Documentation: – Produce tutorials for foundation classes, Boost, etc..

ACAT03, 1-5 Dec. 2003


Summary SEAL has delivered basic foundation, utility libraries

and object dictionary– The main “client” have been POOL (persistency project) and PI

» Plug-in manager is also used in CMS – Very positive feedback received for plug-in manager – Currently being integrated into experiments’ frameworks of

LHCB (Gaudi) and ATLAS (ATHENA) The first version of the Component Model and

Framework services is available– Start integrating in POOL and the LHC experiments– Wait for feedback before developing more services

Scripting based on Python– Boost.Python and PyLCGDict recommended for Python

bindings– PyLCGDict is used to provide bindings for classes of Physics

Interfaces (PI) project

ACAT03, 1-5 Dec. 2003


http://seal.cern.ch/

Acknowledgments:Seal team members: R. Chytracek, J. Generowicz, W. Lavrijsen, M. Marino, P. Mato, S. Roiser, L. Tuura, M. Winkler

Conclusions

ACAT03, 1-5 Dec. 2003


Additional Slides

ACAT03, 1-5 Dec. 2003


LCG Application Domain Coverage

EventGeneration

Core Services

Dictionary

Whiteboard

Foundation and Utility Libraries

DetectorSimulation

Engine

Persistency

StoreMgr

Reconstruction

Algorithms

Geometry Event Model

GridServices

I nteractiveServices

Modeler

GUIAnalysisEvtGen

Calibration

Scheduler

Fitter

PluginMgrMonitor

NTupleScripting

FileCatalog

ROOT GEANT4 DataGrid Python Qt

Monitor

. . .MySQLFLUKA

EventGeneration

Core Services

Dictionary

Whiteboard

Foundation and Utility Libraries

DetectorSimulation

Engine

Persistency

StoreMgr

Reconstruction

Algorithms

Geometry Event Model

GridServices

I nteractiveServices

Modeler

GUIAnalysisEvtGen

Calibration

Scheduler

Fitter

PluginMgrMonitor

NTupleScripting

FileCatalog

ROOT GEANT4 DataGrid Python Qt

Monitor

. . .MySQLFLUKA

ACAT03, 1-5 Dec. 2003


Module Library

PluginManager

PluginFactoryBase

PluginFactoryT

PluginInfo

XXXFactory XXXInfo

Module ModuleDef

ModuleBase

MyModuleDef*

*

PluginCache

Client

*

Plug-in Manager Design Originated from CMS Iguana

ACAT03, 1-5 Dec. 2003


PropertyListItem

MethodField

ItemBuilder

MethodBuilderFieldBuilderClassBuilder

NamespaceBuilder

Reflection

ReflectionBuilder

ClassNamespace

Reflection System Reflection Builder: the loading interface Reflection : the user API

ACAT03, 1-5 Dec. 2003


LCG Object Dictionary: Usage

ROOT I/O

LCGDictionaryCINT

DictStreamer

.h

in

out

ROOTCINT

CINT generatedcode Dict generating

code

.adl.xml

ADL/GOD

OtherClients:

(python, GUI, etc.)

LCG

to C

INT

Dict

gat

eway

(2)

(1)

Population

Conversion

Reflection

GCC-XML

ACAT03, 1-5 Dec. 2003


Seal Component Model Hierarchy of bases classes Each Component lives in a Context Contexts form their own hierarchy Support for component look-up A Service provides its own local

Context– Components of a Service live in the

scope defined by its own Context User classes inherit from

Component or Service– Plug-in functionality for free

Component

Service

MyComponent MyService

Context

*

*

1

ComponentFactory

ComponentLoader

1

SEAL Project Overview

Documents

Transcript of SEAL Project Overview