CERN/IT/API/J.KnoblochLHC Computing Review - Software Panel1 Software Projects in the IT/API Group...
-
Upload
marlene-horton -
Category
Documents
-
view
213 -
download
0
Transcript of CERN/IT/API/J.KnoblochLHC Computing Review - Software Panel1 Software Projects in the IT/API Group...
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 1
Software Projects in the IT/API GroupAPI = Applications for Physics and
Infrastructure
Software Panel of the LHC Computing Review
Jürgen Knobloch
Slides available on: http://nicewww.cern.ch/~knobloch/slides/hhapi/index.htm
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 3
IT Division - Structure from January 2000
Technical ServicesTS
Communication Services CS
User Support and Training US
Div
isio
na
l P
lan
nin
g /
Ad
min
istr
ati
ve
Se
rvic
es
Pro
jec
t Su
pp
ort T
ea
m
Division Leader’s Office / Divisional Officers DL, DDL, TLO, DSO, DTO, DAO
User and Management Committees CAEC, DTF, ELEC, FOCUS, LEPC, LHCC, MB, RB, SPSC
Co
mp
uti
ng
fo
r E
ng
ine
eri
ng
C
E
Co
ntr
ols
CO
Inte
rne
t A
pp
lic
ati
on
s
IA
Da
tab
as
es
(O
bje
ct
an
d R
ela
tio
na
l)
DB
Win
do
ws
In
fra
str
uc
ture
a
nd
Se
rvic
es
W
IN
Ap
pli
ca
tio
ns
fo
r P
hy
sic
s
an
d I
nfr
as
tru
ctu
re
AP
I
Ph
ys
ics
an
d D
es
kto
p P
art
ne
rsh
ip
PD
P
Ap
pli
ca
tio
ns
fo
r P
hy
sic
s
an
d I
nfr
as
tru
ctu
re
AP
I
Controls
CO
Computing for Engineering
CE
Internet Applications
IA
Database(Object and Relational)
DB
Windows Infrastructure and Services
WIN
Physics and Desktop Partnership
PDP
Applications for Physicsand Infrastructure
API
Shrinking staff numbers
Half of the people are working on general CERN infrastructure
Outsourcing of standard tasks
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 4
API Group
Technical Services TS
Communication Services CS
Co
mp
uti
ng
fo
r E
ng
inee
rin
g
CE
Co
ntr
ols
CO
Inte
rne
t A
pp
licat
ion
s
IA
Dat
abas
es
(Ob
ject
an
d R
ela
tio
na
l)
DB
Win
do
ws
Infr
astr
uct
ure
and
Ser
vice
s
WIN
Ap
plic
atio
ns
for
Ph
ysic
s an
d In
fras
tru
ctu
re
AP
I
Ph
ysi
cs
and
Des
kto
p P
art
ner
ship
P
DP
User Support and Training US
Div
isio
nal
Pla
nn
ing
/ A
dm
inis
trat
ive
Ser
vice
s
Pro
ject S
up
po
rt Tea
m
Division Leader’s Office / Divisional Officers DL, DDL, TLO, DSO, DTO, DAO
User and Management Committees CAEC, DTF, ELEC, FOCUS, LEPC, LHCC, MB, RB, SPSC
Ap
plic
atio
ns
for
Ph
ysic
s an
d In
fras
tru
ctu
re
AP
I
Group Organization
Contact with & Support of other projects:JAS, OpenScientist, EDMS, Pythia 7, CORAL ...
Software process supportLanguages, Software Development Tools,
Documentation (LIGHT)Testing, Distribution, Installation, License Management
GE
AN
T 4
AID
A/L
iza
rd/H
TL
/n-
Tu
ple
Fit
tin
g
Co
mm
erc
ial
Co
mp
on
en
ts
CE
RN
LIB
-PA
W
GE
AN
T 3
CL
HE
P
WIR
ED
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 5
What is the role of a CERN-IT software group?
Experiments have recognized the need of “common solutions”
– Historic example: CERNLIB
Cannot do all the work in IT– Example GEANT 4:
• 3 staff in API, 2 FTE on average as visitors in API• 60 people in the experiments and other institutes
Act as catalyst:– Work with experiments on specific solutions
• Small components - short “time to market”
– Productize for other experiments– Share the work with other labs and institutes
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 6
Products
OO toolkit for simulation
Event display in JAVA
Mathematical libraries, graphics
(Mostly) FORTRAN softwareLimited maintenance: next and final release of CERNLIB ever: 1Q/2002
Libraries for HEP ComputingWas called LHC++C++ replacement of CERNLIB
GEANT 4
AIDA/Lizard/HTL/n-Tuple Fitting
Commercial Components
CERNLIB-PAW
GEANT 3
CLHEP
WIRED
7 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
Geant4: object oriented toolkit for simulation,
Slides prepared by
John Apostolakis, CERN/IT/API
9 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
Geant4 Collaboration
Collaboration based on MoU, consists of• Laboratories: CERN, SLAC, KEK, TRIUMF, Jefferson,• Experiments: Atlas, CMS, LHCb, BaBar, • Institutes, Groups: IN2P3, IGD, TERA, Frankfurt
– for the production service, maintenance and development of Geant4
About 80 collaborators worldwide– physicists and computer scientists
G4 was developed by worldwide RD44 project– RD44 ended with first production release in
December 98
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 10
Class Category Diagram
RunRun Geant4Geant4ReadoutReadout
EventEvent Event Gen.Event Gen.DigiDigi
HitHit MagneticFields
MagneticFields
TrackingTrackingDetectorDetector
PhysicsPhysics
TrackTrack
MaterialMaterial
ParticleDefinitionParticle
Definition
VisualizationVisualization
ODBMSODBMS
UtilitiesUtilities
GeometryGeometry CADInterface
CADInterface
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 11
In Context: Theoretical Hadronic Processes
Theoretical models, from low E to high E– Pre-Compound Model + Evaporation Phase– Cascades, CHIPS and QMD models– String models– Interface to event generator(s)
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 12
Pre-Compound Model + Evaporation Phase
Traditional pre-equilibrium model– as good as existing ones
Evaporation:– Weisskopf-Ewing model – Fermi breakup model– Model for fission– Multifragmentation model (Bondarf)– Photon Evaporation– only missing Internal Conversion
• (suppressed by more than 10^4)
Future: 2nd Pre-Compound, from HETC re-eng. (in 2000)
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 13
Other Hadronics
Data driven and parameterized approaches
Neutrons• from numerous evaluated data libraries• down to thermal energies, up to 20 MeV
– Isotope production (see earlier slide)– Induced Fission & Capture (H.Fesefeld)
• used above 20 MeV
Parameterized models– Gheisha + fixes– New parameterizations
15 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
Photon evaporation
Sn(120), 80 MeV Sn(112), 80 MeV
MeV MeV
Photon energy, discrete levels
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 16
Induced fission U235, 10 MeV neutrons
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 17
Isotope Production
Isotopes produced by neutrons on Lead 208
Small dots: evaluated data
Circles with error bars: Geant
– latest model– is in currrent
release: Geant4 1.0
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 18
Electro-Magnetic physics
Gammas: – Gamma-conversion, Compton scattering, Photo-electric effect
Leptons(e, mu) + charged particles(hadrons, ions): – Ionisation, Bremstrahlung, Energy loss, Multiple scattering, transition radiation,
Synchrotron radiation, PI model energy loss
Photons: – Cerenkov, Rayleigh, Reflection, Refraction, Absorption, Scintillation
High energy muons and lepton-hadron interactionsImplementation of EM physics to 250 eV
– using ICRU and Ziegler parameterisations for hadrons, ions – using evaluated data libraries (for e-, gammas)
19 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
Shower profile
1 GeV electron in H2O
G4
Data
G3
Very good agreement seen with the data
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 20
Sampling calorimeter
Sampling calorimeter
– visible energy
tests– all EM processes
for e-, e+ and photon
Data from Sicapo Col. NIM A332 (85-90) 1993
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 21
Multiple scattering
Example of comparisons:
– 6.56 MeV proton • on 93 microns Si
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 22
Results very stable with variation of cuts
– even track length
Shower profiles for different cuts stable
– between 10mm and 50 microns
Changing cuts
23 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
Electromagnetic processes
All processes at least at level of Geant-3
New process: Transition radiation
Multiple Scattering: new model– no path length restriction– added lateral displacement
• measured effect on result
Energy Loss: two types of approaches• two approaches: differential and integral• several alternatives: PAI model (thin),
Super E-loss
– Integration of cross section over Energy• hadronic resonances can be seen (future)
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 24
Low energy EM processes
Photons, electrons down to 250 eV
– Xsec from evaluations
– with thanks to J. Stepanek
CERN/ ESAA. FortiM.G. Pia
P. Nieminen
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 25
Software Engineering
Software Engineering in HEP
The software process: distributed development
ESA PSS-05: URD v.06
Object-Oriented methodology: Booch+unified
OOA&D: Rational Rose CASE
QA: Insure, Logiscope, code inspection, coding guidelines, scripts
Testing at class-level (ex: 375 test-cases for processes)
System integration testing
26 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
Geant4 geometries
EM Calorimeter, BaBar (SLAC)
Borexino Detector for Low Energy Solar NeutrinosURL: http://almime.mi.infn.it/
27 LHC Computing Review - Software Panel /CERN/IT/API/J.Knobloch
CMS Geometry in GEANT4April 1999
Beam Pipe Tracker
– Si Pixel Detectors• Barrel Si Pixel • Forward Si Pixel
– Si Strip Detectors • Barrel Si Strip
– MSGC • Barrel MSGC
Calorimeters – Electromagnetic Calorimeter
• Barrel ECAL – Hadron Calorimeter
• Barrel HCAL Muon System
– Barrel Muon
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 29
Milestones for Geant4 in 2000
Comparison projects with ‘experiments’• agree mini-projects, work & publish together
Create a training kit • and continually improve documentation, examples
Re-examine & improve the software process
Other goals– new development– publishing results
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 30
Milestone 1: comparison projects
Select key ‘detector’ setups– from experimental groups (eg test beams)
Perform in-depth simulations– with combined resources
Analyse results and publish together
Set of ‘detectors’ should be complementary– short projects (~ 6 months), few at a time (~3);– e.g. Calorimeters - Trackers - Radiation studies
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 31
Other Proposed Milestones
Create a training kit • to aid in teaching the capabilities of G4 toolkit• bringing together all GEANT 4 training experience
– Must address needs of user communities• type, length of training , proposed audiences
Software Process Improvement
– Identify areas needing new analysis/design cycle
– Create team for QA & optimisation monitoring– Fully integrate testing & QA in development cycle.
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 32
Libraries for HEP Computing (LHC++)
Slides extracted from CHEP2000 talk by
Andreas PfeifferCERN/IT/API
Name is being changed!
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 33
Strategy for AIDA, Lizard et al. (was LHC++)
Based on components
Defining abstract interfaces
Phasing out of Explorer
Reuse of components developed previously
C++ is still required
Two purposes:– “Out of the box” analysis– Toolset - Library
Mathematical functions provided by external source
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 34
LHC++
Aim: physics data analysis in OO-environment
Needs libraries and tools– functionally as complete as possible (full
replacement of CERNLIB)– making use of OO tools and methodologies– started in late 1995
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 35
LHC++: what it is (I)
Modular replacement of current CERNLIB for use in HEP experiments
– memory management (C++)– persistency (“I/O”)– mathematical library – foundation classes – random number generators – histogramming – fitting – simulation
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 36
LHC++: what it is (II)
Based on OO methodology– eases long term maintenance and extensibility– quick way to (basic) functionality– flexible wrt. changing user requirements
Initial choice: C++ – performance close to C/FORTRAN– Java for the future ?
In close collaboration with LHC (and other) experiments
“use standards as far as possible”
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 37
Present configuration
Object persistency– from RD45 collaboration (Objectivity/DB)
Foundation classes– HEP specific foundation classes (CLHEP)– Random number generators (CLHEP)
Mathematical library from NAG (NAG_C)– covers broad range of functionality– extensions required by CERN will be added in next
release (Mark 6)– quality assurance
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 38
Present configuration (cont.)
Histogramming: HTL
Fitting: Gemini, HepFitting packages– interface to any minimizer (at present: NAG, Minuit)
Event generators– Lund people started Pythia-7 (C++)– StdHep++ in process to become part of CLHEP
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 39
LHC++ packages and dependencies
Being
replaced
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 40
New Data Analysis Tool
First iteration on physics data anlysis tool (based on IRIS Explorer)
– data driven approach– GUI based, not command line driven– majority of users didn’t like it
Request made in September 1999 to create new physics analysis tool
– new requirements defined together with experiments– Try to maximise re-use of existing components
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 41
OO approach to create tool
Start with OO analysis– collection of user requirements
Study critical path(s)– user interface, distributed processing, ...
OO design phase– define categories and classes, find patterns
Create prototype– get feedback from users
Iterate
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 42
User requirements for a physics analysis tool
Easy to use for “end user”– "like PAW”
Foresee customization/integration wrt. existing frameworks of experiments
– e.g., use persistency/messaging/... from experiment– needs to be compatible with experiment’s framework
Plan for extensions
Maximize flexibility/interoperability– "plug-and-play-like" use of components from other
frameworks (shared libs using the same interfaces)
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 44
Designing the Abstract Interfaces
AIDA project started by HepVis’99 workgroup: Abstract Interfaces for Data Analysis
– http://wwwinfo.cern.ch/asd/lhc++/AIDA/index.html
In close collaboration with users and developers from experiments and providers of other packages
– Iguana, HippoDraw, JAS, OpenScientist
Starting with Histogram classes– presently in final iteration
Next items are Ntuples, Vectors and Fitting
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 45
User interface design
Create abstract interface for UserIF– decouple functionality from presentation– makes UI independent of implementation– using SWIG for flexibility wrt. scripting language
Starting with Python based UserIF– OO scripting language
Other implementations will come
Also GUI based user interfaces easy by design – "Controller" category defines interface to user
interface
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 46
Status
Libraries are ready– http://wwwinfo.cern.ch/asd/lhc++/index.html
Data analysis tool– first prototype ready– http://wwwinfo.cern.ch/asd/lhc++/Lizard/index.html
Work on defining interfaces– AIDA project (from HepVis’99 workgroup)– http://wwwinfo.cern.ch/asd/lhc++/AIDA/index.html
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 47
Plans
AIDA– Continue abstract interface definitions together with
users and other projects (JAS, OpenScientist,…)
Lizard– Use AIDA interfaces where available– First release in October 2000 (LINUX only)
• Histogramming, Ntuples, Vectors, Persistency• Functions, plotting, fitting• Scripting, GUI• Release on LINUX first
– Next release in April 2001
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 48
Lizard - October 2000 (1)
Histograms• create, destroy, manipulate, store/retrieve from DB, plotting
Ntuples• retrieve from DB, scan/project according to 'full' C++ cuts
Vectors• retrieve/store from ASCII file, creation from histogram,
plotting
Functions• definition of arbitrary functions, plotting• Fitter: chisquare/ML fitting of vectors/histograms with simple
models (Gaussian, Exponential, Polynomial and linear combinations).
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 49
Lizard - October 2000 (2)
Plotter• plotting on several zones, overlaying plots, colors,
titles/labels• 1D representations (histo,line,errors,markers) • 2D representations (Box, Contour)• Postscript• Locate feature.
Controller: PAW like interface
Analyzer• dynamic compiling and loading of experiment specific
code,creation of histograms which can then be accessed from Lizard
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 50
Lizard - April 2001
Ntuples: creation/storing in DB
Fitter• fitting with user defined functions, store/retrieve fit context
from DB.
Plotter• more graphics representations, more powerful locate
feature. Visual editor.
Lizard available on Linux, Solaris (and NT?).
Another script interface (Tcl, Perl, CINT?)
AIDA interfaces used where available.
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 51
WIRED
World-Wide Web Interactive Remote Event Display
Slides extracted from CHEP2000 talk by
Mark Dönszelmann, IT/API, CERN
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 52
Features
– Framework for building Event Displays• 10% of code is experiment specific
– Client Server Architecture• Client can be used as Application or Applet (browser mode)
– Written entirely in Java• Client in Java, Server can be in a different language
– Access to data• reads files or connects to Java, FORTRAN, C and C++
– Graphics Engine to do HEP specific projections• 3D as well as non-linear (fish-eye) projections
– ATLAS, BaBar, CHORUS, DELPHI and LHCb• Development infrastructure available for collaboration
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 54
WIRED as Applet in Netscape (BaBar)
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 55
GUI (Views and Controls)
ViewTop
ControlTop
ToolBar
MenuBar
Console
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 56
Infobus to connect Views to Controls
Info
Bus
Info Bus
Views
Comm. StatusEvent Selector Event Info
Command
Parts Select
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 57
Infrastructure for development
Unix and NT support for development
AFS to access repository --> migrate to CVS server
CVS repository and CVSWeb
GNUMake
Nightly build and test on different platforms
JIndent for coding convention
JInstall Toolkit to create distribution
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 58
WIRED - Summary
– Now• Graphics Engine, including picking, reasonably complete• Printing in PostScript• Easy event selection• Access to Data in XML, RMI and CORBA• Integration - Coupling to JAS
– Near Future• Further Infobus integration• Additional Controls• Addition of the scripting engine• Configuration saving and restoring (in XML)• HepRep implementation• Access to C++ objects directly (without CORBA)
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 59
Software Development Tools (SDT)
Providing expertise, making tools available at the server at CERN.
Tools in production:– Analysis and Design
• Rational Rose and StP - OMT/Booch/UML analysis and design tools with code generation
– Programming and Debugging• SNiFF+ : Integrated development environment for C, C++,
Java, Fortran and CORBA IDL
– Graphical User Interface Builder• X-Designer : Interactive X-platform GUI builder with
OSF/Motif toolkit widgets, C, JAVA and Windows codegeneration.
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 60
SDT (2)–Quality, Metrics and Testing
• Logiscope Tool Kit: Source code analyser and test coverage for C/C++/Fortran.
• Look!: C++ dynamic visualization system. • Insure++: Comprehensive tool for error detection in C/C++ programs
–Documentation• LIGHT: LIfecycle Global Hypertext • FrameMaker: Complete and integrated publishing tool. • IXgen: Indexing and marker management tool for FrameMaker. • SoDA : Software Documentation Automation. • VisualThought : Multipurpose drawing, diagramming and flowcharting
tool. • SDLT (API product) : Software Documentation Layout Templates, for
FrameMaker. • User Guide Templates (API product) : For FrameMaker.
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 61
SDT Statistics
Web accesses Licenses served
/CERN/IT/API/J.Knobloch LHC Computing Review - Software Panel 62
Conclusions
Geant 4• Comparisons with data very promising - continued• Training program being developed• Software process: monitoring, improvements
Analysis tools and libraries• C++ components reused in adapted environment• IRIS-Explorer will be replaced• Role of Java increasing• Java - C++ co-existence to be understood
Software Development Tools• Ensuring/enabling software quality
API - few dedicated people ready to help the experiments!