Software Projects in the IT/API Group API = Applications for Physics and Infrastructure

/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 1

Software Projects in the IT/API GroupAPI = Applications for Physics and

Infrastructure

Software Panel of the LHC Computing Review

Jürgen Knobloch

Slides available on: http://nicewww.cern.ch/~knobloch/slides/hhapi/index.htm


IT Division - Structure from January 2000

Technical ServicesTS

Communication Services CS

User Support and Training US

Div

isio

na

l P

lan

nin

g /

Ad

min

istr

ati

ve

Se

rvic

es

Pro

jec

t Su

pp

ort T

ea

m

Division Leader’s Office / Divisional Officers DL, DDL, TLO, DSO, DTO, DAO

User and Management Committees CAEC, DTF, ELEC, FOCUS, LEPC, LHCC, MB, RB, SPSC

Co

mp

uti

ng

fo

r E

ng

ine

eri

ng

C

E

Co

ntr

ols

CO

Inte

rne

t A

pp

lic

ati

on

s

IA

Da

tab

as

es

(O

bje

ct

an

d R

ela

tio

na

l)

DB

Win

do

ws

In

fra

str

uc

ture

a

nd

Se

rvic

es

W

IN

Ap

pli

ca

tio

ns

fo

r P

hy

sic

s

an

d I

nfr

as

tru

ctu

re

AP

I

Ph

ys

ics

an

d D

es

kto

p P

art

ne

rsh

ip

PD

P

Ap

pli

ca

tio

ns

fo

r P

hy

sic

s

an

d I

nfr

as

tru

ctu

re

AP

I

Controls

CO

Computing for Engineering

CE

Internet Applications

IA

Database(Object and Relational)

DB

Windows Infrastructure and Services

WIN

Physics and Desktop Partnership

PDP

Applications for Physicsand Infrastructure

API

Shrinking staff numbers

Half of the people are working on general CERN infrastructure

Outsourcing of standard tasks


API Group

Technical Services TS

Communication Services CS

Co

mp

uti

ng

fo

r E

ng

inee

rin

g

CE

Co

ntr

ols

CO

Inte

rne

t A

pp

licat

ion

s

IA

Dat

abas

es

(Ob

ject

an

d R

ela

tio

na

l)

DB

Win

do

ws

Infr

astr

uct

ure

and

Ser

vice

s

WIN

Ap

plic

atio

ns

for

Ph

ysic

s an

d In

fras

tru

ctu

re

AP

I

Ph

ysi

cs

and

Des

kto

p P

art

ner

ship

P

DP

User Support and Training US

Div

isio

nal

Pla

nn

ing

/ A

dm

inis

trat

ive

Ser

vice

s

Pro

ject S

up

po

rt Tea

m

Division Leader’s Office / Divisional Officers DL, DDL, TLO, DSO, DTO, DAO

User and Management Committees CAEC, DTF, ELEC, FOCUS, LEPC, LHCC, MB, RB, SPSC

Ap

plic

atio

ns

for

Ph

ysic

s an

d In

fras

tru

ctu

re

AP

I

Group Organization

Contact with & Support of other projects:JAS, OpenScientist, EDMS, Pythia 7, CORAL ...

Software process supportLanguages, Software Development Tools,

Documentation (LIGHT)Testing, Distribution, Installation, License Management

GE

AN

T 4

AID

A/L

iza

rd/H

TL

/n-

Tu

ple

Fit

tin

g

Co

mm

erc

ial

Co

mp

on

en

ts

CE

RN

LIB

-PA

W

GE

AN

T 3

CL

HE

P

WIR

ED


What is the role of a CERN-IT software group?

Experiments have recognized the need of “common solutions”

– Historic example: CERNLIB

Cannot do all the work in IT– Example GEANT 4:

• 3 staff in API, 2 FTE on average as visitors in API• 60 people in the experiments and other institutes

Act as catalyst:– Work with experiments on specific solutions

• Small components - short “time to market”

– Productize for other experiments– Share the work with other labs and institutes


Products

OO toolkit for simulation

Event display in JAVA

Mathematical libraries, graphics

(Mostly) FORTRAN softwareLimited maintenance: next and final release of CERNLIB ever: 1Q/2002

Libraries for HEP ComputingWas called LHC++C++ replacement of CERNLIB

GEANT 4

AIDA/Lizard/HTL/n-Tuple Fitting

Commercial Components

CERNLIB-PAW

GEANT 3

CLHEP

WIRED

7 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch

Geant4: object oriented toolkit for simulation,

Slides prepared by

John Apostolakis, CERN/IT/API


Geant4 Collaboration

Collaboration based on MoU, consists of• Laboratories: CERN, SLAC, KEK, TRIUMF, Jefferson,• Experiments: Atlas, CMS, LHCb, BaBar, • Institutes, Groups: IN2P3, IGD, TERA, Frankfurt

– for the production service, maintenance and development of Geant4

About 80 collaborators worldwide– physicists and computer scientists

G4 was developed by worldwide RD44 project– RD44 ended with first production release in

December 98


Class Category Diagram

RunRun Geant4Geant4ReadoutReadout

EventEvent Event Gen.Event Gen.DigiDigi

HitHit MagneticFields

MagneticFields

TrackingTrackingDetectorDetector

PhysicsPhysics

TrackTrack

MaterialMaterial

ParticleDefinitionParticle

Definition

VisualizationVisualization

ODBMSODBMS

UtilitiesUtilities

GeometryGeometry CADInterface

CADInterface


In Context: Theoretical Hadronic Processes

Theoretical models, from low E to high E– Pre-Compound Model + Evaporation Phase– Cascades, CHIPS and QMD models– String models– Interface to event generator(s)


Pre-Compound Model + Evaporation Phase

Traditional pre-equilibrium model– as good as existing ones

Evaporation:– Weisskopf-Ewing model – Fermi breakup model– Model for fission– Multifragmentation model (Bondarf)– Photon Evaporation– only missing Internal Conversion

• (suppressed by more than 10^4)

Future: 2nd Pre-Compound, from HETC re-eng. (in 2000)


Other Hadronics

Data driven and parameterized approaches

Neutrons• from numerous evaluated data libraries• down to thermal energies, up to 20 MeV

– Isotope production (see earlier slide)– Induced Fission & Capture (H.Fesefeld)

• used above 20 MeV

Parameterized models– Gheisha + fixes– New parameterizations


High Energy K,pi on Al, Au


Photon evaporation

Sn(120), 80 MeV Sn(112), 80 MeV

MeV MeV

Photon energy, discrete levels


Induced fission U235, 10 MeV neutrons


Isotope Production

Isotopes produced by neutrons on Lead 208

Small dots: evaluated data

Circles with error bars: Geant

– latest model– is in currrent

release: Geant4 1.0


Electro-Magnetic physics

Gammas: – Gamma-conversion, Compton scattering, Photo-electric effect

Leptons(e, mu) + charged particles(hadrons, ions): – Ionisation, Bremstrahlung, Energy loss, Multiple scattering, transition radiation,

Synchrotron radiation, PI model energy loss

Photons: – Cerenkov, Rayleigh, Reflection, Refraction, Absorption, Scintillation

High energy muons and lepton-hadron interactionsImplementation of EM physics to 250 eV

– using ICRU and Ziegler parameterisations for hadrons, ions – using evaluated data libraries (for e-, gammas)


Shower profile

1 GeV electron in H2O

G4

Data

G3

Very good agreement seen with the data


Sampling calorimeter

Sampling calorimeter

– visible energy

tests– all EM processes

for e-, e+ and photon

Data from Sicapo Col. NIM A332 (85-90) 1993


Multiple scattering

Example of comparisons:

– 6.56 MeV proton • on 93 microns Si


Results very stable with variation of cuts

– even track length

Shower profiles for different cuts stable

– between 10mm and 50 microns

Changing cuts


Electromagnetic processes

All processes at least at level of Geant-3

New process: Transition radiation

Multiple Scattering: new model– no path length restriction– added lateral displacement

• measured effect on result

Energy Loss: two types of approaches• two approaches: differential and integral• several alternatives: PAI model (thin),

Super E-loss

– Integration of cross section over Energy• hadronic resonances can be seen (future)


Low energy EM processes

Photons, electrons down to 250 eV

– Xsec from evaluations

– with thanks to J. Stepanek

CERN/ ESAA. FortiM.G. Pia

P. Nieminen


Software Engineering

Software Engineering in HEP

The software process: distributed development

ESA PSS-05: URD v.06

Object-Oriented methodology: Booch+unified

OOA&D: Rational Rose CASE

QA: Insure, Logiscope, code inspection, coding guidelines, scripts

Testing at class-level (ex: 375 test-cases for processes)

System integration testing


Geant4 geometries

EM Calorimeter, BaBar (SLAC)

Borexino Detector for Low Energy Solar NeutrinosURL: http://almime.mi.infn.it/


CMS Geometry in GEANT4April 1999

Beam Pipe Tracker

– Si Pixel Detectors• Barrel Si Pixel • Forward Si Pixel

– Si Strip Detectors • Barrel Si Strip

– MSGC • Barrel MSGC

Calorimeters – Electromagnetic Calorimeter

• Barrel ECAL – Hadron Calorimeter

• Barrel HCAL Muon System

– Barrel Muon


Prototype Atlas Geometry


Milestones for Geant4 in 2000

Comparison projects with ‘experiments’• agree mini-projects, work & publish together

Create a training kit • and continually improve documentation, examples

Re-examine & improve the software process

Other goals– new development– publishing results


Milestone 1: comparison projects

Select key ‘detector’ setups– from experimental groups (eg test beams)

Perform in-depth simulations– with combined resources

Analyse results and publish together

Set of ‘detectors’ should be complementary– short projects (~ 6 months), few at a time (~3);– e.g. Calorimeters - Trackers - Radiation studies


Other Proposed Milestones

Create a training kit • to aid in teaching the capabilities of G4 toolkit• bringing together all GEANT 4 training experience

– Must address needs of user communities• type, length of training , proposed audiences

Software Process Improvement

– Identify areas needing new analysis/design cycle

– Create team for QA & optimisation monitoring– Fully integrate testing & QA in development cycle.


Libraries for HEP Computing (LHC++)

Slides extracted from CHEP2000 talk by

Andreas PfeifferCERN/IT/API

[email protected]

Name is being changed!


Strategy for AIDA, Lizard et al. (was LHC++)

Based on components

Defining abstract interfaces

Phasing out of Explorer

Reuse of components developed previously

C++ is still required

Two purposes:– “Out of the box” analysis– Toolset - Library

Mathematical functions provided by external source


LHC++

Aim: physics data analysis in OO-environment

Needs libraries and tools– functionally as complete as possible (full

replacement of CERNLIB)– making use of OO tools and methodologies– started in late 1995


LHC++: what it is (I)

Modular replacement of current CERNLIB for use in HEP experiments

– memory management (C++)– persistency (“I/O”)– mathematical library – foundation classes – random number generators – histogramming – fitting – simulation


LHC++: what it is (II)

Based on OO methodology– eases long term maintenance and extensibility– quick way to (basic) functionality– flexible wrt. changing user requirements

Initial choice: C++ – performance close to C/FORTRAN– Java for the future ?

In close collaboration with LHC (and other) experiments

“use standards as far as possible”


Present configuration

Object persistency– from RD45 collaboration (Objectivity/DB)

Foundation classes– HEP specific foundation classes (CLHEP)– Random number generators (CLHEP)

Mathematical library from NAG (NAG_C)– covers broad range of functionality– extensions required by CERN will be added in next

release (Mark 6)– quality assurance


Present configuration (cont.)

Histogramming: HTL

Fitting: Gemini, HepFitting packages– interface to any minimizer (at present: NAG, Minuit)

Event generators– Lund people started Pythia-7 (C++)– StdHep++ in process to become part of CLHEP


LHC++ packages and dependencies

Being

replaced


New Data Analysis Tool

First iteration on physics data anlysis tool (based on IRIS Explorer)

– data driven approach– GUI based, not command line driven– majority of users didn’t like it

Request made in September 1999 to create new physics analysis tool

– new requirements defined together with experiments– Try to maximise re-use of existing components


OO approach to create tool

Start with OO analysis– collection of user requirements

Study critical path(s)– user interface, distributed processing, ...

OO design phase– define categories and classes, find patterns

Create prototype– get feedback from users

Iterate


User requirements for a physics analysis tool

Easy to use for “end user”– "like PAW”

Foresee customization/integration wrt. existing frameworks of experiments

– e.g., use persistency/messaging/... from experiment– needs to be compatible with experiment’s framework

Plan for extensions

Maximize flexibility/interoperability– "plug-and-play-like" use of components from other

frameworks (shared libs using the same interfaces)


Categories and dependencies


Designing the Abstract Interfaces

AIDA project started by HepVis’99 workgroup: Abstract Interfaces for Data Analysis

– http://wwwinfo.cern.ch/asd/lhc++/AIDA/index.html

In close collaboration with users and developers from experiments and providers of other packages

– Iguana, HippoDraw, JAS, OpenScientist

Starting with Histogram classes– presently in final iteration

Next items are Ntuples, Vectors and Fitting


User interface design

Create abstract interface for UserIF– decouple functionality from presentation– makes UI independent of implementation– using SWIG for flexibility wrt. scripting language

Starting with Python based UserIF– OO scripting language

Other implementations will come

Also GUI based user interfaces easy by design – "Controller" category defines interface to user

interface


Status

Libraries are ready– http://wwwinfo.cern.ch/asd/lhc++/index.html

Data analysis tool– first prototype ready– http://wwwinfo.cern.ch/asd/lhc++/Lizard/index.html

Work on defining interfaces– AIDA project (from HepVis’99 workgroup)– http://wwwinfo.cern.ch/asd/lhc++/AIDA/index.html


Plans

AIDA– Continue abstract interface definitions together with

users and other projects (JAS, OpenScientist,…)

Lizard– Use AIDA interfaces where available– First release in October 2000 (LINUX only)

• Histogramming, Ntuples, Vectors, Persistency• Functions, plotting, fitting• Scripting, GUI• Release on LINUX first

– Next release in April 2001


Lizard - October 2000 (1)

Histograms• create, destroy, manipulate, store/retrieve from DB, plotting

Ntuples• retrieve from DB, scan/project according to 'full' C++ cuts

Vectors• retrieve/store from ASCII file, creation from histogram,

plotting

Functions• definition of arbitrary functions, plotting• Fitter: chisquare/ML fitting of vectors/histograms with simple

models (Gaussian, Exponential, Polynomial and linear combinations).


Lizard - October 2000 (2)

Plotter• plotting on several zones, overlaying plots, colors,

titles/labels• 1D representations (histo,line,errors,markers) • 2D representations (Box, Contour)• Postscript• Locate feature.

Controller: PAW like interface

Analyzer• dynamic compiling and loading of experiment specific

code,creation of histograms which can then be accessed from Lizard


Lizard - April 2001

Ntuples: creation/storing in DB

Fitter• fitting with user defined functions, store/retrieve fit context

from DB.

Plotter• more graphics representations, more powerful locate

feature. Visual editor.

Lizard available on Linux, Solaris (and NT?).

Another script interface (Tcl, Perl, CINT?)

AIDA interfaces used where available.


WIRED

World-Wide Web Interactive Remote Event Display

Slides extracted from CHEP2000 talk by

Mark Dönszelmann, IT/API, CERN


Features

– Framework for building Event Displays• 10% of code is experiment specific

– Client Server Architecture• Client can be used as Application or Applet (browser mode)

– Written entirely in Java• Client in Java, Server can be in a different language

– Access to data• reads files or connects to Java, FORTRAN, C and C++

– Graphics Engine to do HEP specific projections• 3D as well as non-linear (fish-eye) projections

– ATLAS, BaBar, CHORUS, DELPHI and LHCb• Development infrastructure available for collaboration


WIRED as Application (Delphi)


WIRED as Applet in Netscape (BaBar)


GUI (Views and Controls)

ViewTop

ControlTop

ToolBar

MenuBar

Console


Infobus to connect Views to Controls

Info

Bus

Info Bus

Views

Comm. StatusEvent Selector Event Info

Command

Parts Select


Infrastructure for development

Unix and NT support for development

AFS to access repository --> migrate to CVS server

CVS repository and CVSWeb

GNUMake

Nightly build and test on different platforms

JIndent for coding convention

JInstall Toolkit to create distribution


WIRED - Summary

– Now• Graphics Engine, including picking, reasonably complete• Printing in PostScript• Easy event selection• Access to Data in XML, RMI and CORBA• Integration - Coupling to JAS

– Near Future• Further Infobus integration• Additional Controls• Addition of the scripting engine• Configuration saving and restoring (in XML)• HepRep implementation• Access to C++ objects directly (without CORBA)


Software Development Tools (SDT)

Providing expertise, making tools available at the server at CERN.

Tools in production:– Analysis and Design

• Rational Rose and StP - OMT/Booch/UML analysis and design tools with code generation

– Programming and Debugging• SNiFF+ : Integrated development environment for C, C++,

Java, Fortran and CORBA IDL

– Graphical User Interface Builder• X-Designer : Interactive X-platform GUI builder with

OSF/Motif toolkit widgets, C, JAVA and Windows codegeneration.


SDT (2)–Quality, Metrics and Testing

• Logiscope Tool Kit: Source code analyser and test coverage for C/C++/Fortran.

• Look!: C++ dynamic visualization system. • Insure++: Comprehensive tool for error detection in C/C++ programs

–Documentation• LIGHT: LIfecycle Global Hypertext • FrameMaker: Complete and integrated publishing tool. • IXgen: Indexing and marker management tool for FrameMaker. • SoDA : Software Documentation Automation. • VisualThought : Multipurpose drawing, diagramming and flowcharting

tool. • SDLT (API product) : Software Documentation Layout Templates, for

FrameMaker. • User Guide Templates (API product) : For FrameMaker.


SDT Statistics

Web accesses Licenses served


Conclusions

Geant 4• Comparisons with data very promising - continued• Training program being developed• Software process: monitoring, improvements

Analysis tools and libraries• C++ components reused in adapted environment• IRIS-Explorer will be replaced• Role of Java increasing• Java - C++ co-existence to be understood

Software Development Tools• Ensuring/enabling software quality

API - few dedicated people ready to help the experiments!

Software Projects in the IT/API Group API = Applications for Physics and Infrastructure

Documents

Transcript of Software Projects in the IT/API Group API = Applications for Physics and Infrastructure