CMD-3 Detector Offline Software Development

CMD-3 DETECTOR OFFLINE SOFTWARE DEVELOPMENT

Alexander Zaytsev // [email protected] behalf of CMD-3 CollaborationCHEP2009 (Prague, Mar 2009)

CMD-3 Offline Software

2

OUTLINE CMD-3 Experiment Overview Basic Offline Software Requirements Generic Offline software Design Software Integration Solution Event Model and Data Persistency Status of the Implementation Summary

3/26/2009


3

NOVOSIBIRSK / BINP / CMD-3

3/26/2009

Prague

Novosibirsk


3/26/2009CMD-3 Offline Software

4

Novosibirsk



5

VEPP-2000

CMD-3, SND

0.6 km

BINP (founded in

1958)


HISTORICAL PERSPECTIVE

3/26/2009

CMD-1: 1979-1984Spark chamber, MWPCs(photo registration)

CMD-2: 1992-2000Drift chamber, crystalcalorimeters (dataarchival on tapes,Linux cluster foroffline processing)

CMD-3: many features are inherited

from CMD-2

CMD-3 DETECTOR SUBSYSTEMS Trackers:

Drift Chamber (DC)

Z-Chamber (ZC)

Calorimeters: BGO Endcap

Calorimeter CsI Barrel

Calorimeter

7

Calorimeters with tracking capabilities: Liquid Xe Calorimeter (LXe)

Particle ID: Muon TOF

Trigger: Low Level High Level

State-of-the-art sub-system most challenging from the reconstruction point of view

TDAQ rate: 1 kHzRAW event size: 3 KBRAW data rate: 3 MB/s

Total amount of RAW datastored for analysis: 10 TB

We are not limited by computing power: all the experimental data can be re-processed at once when

needed, factor of 10 more of reconstruction/simulation data is to be produced

CMD-3 GENERAL LAYOUT


UniversalCryogenic Magnetic Detector

DC/BGO

LXeCsI

VEPP-2000 COLLIDER COMPLEX


ILU3 MeVLinac

B-3M200 M eVsynchro-betatron

BEPe,e

booster–+

900 M eV SND

CMD-3

ee– +®convertor

2 m2 m

ILU3 MeV Linac

E 1 GeV (per beam)L 1×1032 cm-2 sec-1 (1×1 bunch,

round beams)

BEPe e booster

900 MeV+-B-3M

200 MeV synchro-betatron

VEPP-2000

Inherited from the previous VEPP-2M machine Built from the scratch

10

WHY ARE WE DOING IT?Mainstream task:extension of the workdone with CMD-2 withbroader energy rangeand much higherprecision

The most challenging taskfrom the point of view ofbuilding reconstructionalgorithms (yet to behandled)


11

BASIC SOFTWARE REQUIREMENTS Modularity: the reconstruction application might

consist of up to 100 components (modules) bonded in runtime or compilation time

Flexibility: changing the layout of module interaction without recompiling the code

Parallelization schema: splitting input data Number of threads: single thread mode Batch environment compatibility: extensive log

manage-ment support, switching off all the interactive features

Lifetime/Support: 10-15 years Persistent data lifetime: 15-20 years

3/26/2009

The main challenge is to build the system which is suitable for both development and production environment


12

RECONSTRUCTION SOFTWARE DESIGN Software integration

platform: custom Internal data exchange

mechanism: custom Configuration

representation: XML Generic containers/data

representation:STL & ROOT containers wrapped by custom high level experimental event container

Persistency platform: ROOT

Standard build/runtime environment : Platform: Scientific Linux

(SL, both x86 and x86_64) Main programming

language: C++ Compilers involved: GCC Build system: GNU/Make API documentation:

Doxygen, TWiki Use of code generation:

ROOT dictionaries Source code management:

CVS >> SVN Coding convention exists

from the early days of the project

3/26/2009


13

SOFTWARE INTEGRATION PLATFORM

External libraryExternal

libraryExternal libraryExternal

library

External libraryExternal

libraryExternal libraryExternal

library

Detector Subsystem Reconstruction Tools

(1)

Development (DSO based) environment

External software(ROOT, GSL, Boost, etc.)

Detector Subsystem Reconstruction Tools

(N)

…

Software developedby a particular sub-

detector team

Each block represent a partof subsystem specific

reconstruction algorithm

Offline module (1)

Offline module (M)

…

Wrapper software encapsulating the

reconstruction tools

Cmd3Fwk Software Integration Framework

Introduces the subdetector tools to the software

integration framework

No data exchange between the consequent partsof reconstruction occur here!

14

Module Module

Module

Data

Module

Data

SOFTWARE INTEGRATION PLATFORMDevelopment (DSO based) environmentCmd3Fwk Software Integration

Framework

Module

Module

Module

External datasource

Data

Data

Data

read

exports

Data

Data

marked asthe final module in the sequence

Reverse dependency lookup in the DAG implicitly specified in XML configurationExtern

al XML config

depends on

module parameters,dependency specs,external data I/O config

The same DAG based dependency resolution approach works fine for CMD-3 simulation (Geant4).

The 3rd level trigger will require predefined module sequence.

Data exchangeinterface provided

with cache functionality

15

SWITCHING TO PRODUCTION MODECmd3Fwk Software Integration

FrameworkNot yet available in current Cmd3Fwk version branch 1.x

ModuleModuleModuleModuleXML

config

Cmd3Fwk CLI Tools(DSO based modules)

XML config

Cmd3Fwk Static Snapshot (no

DSO modules)

ModuleModuleModuleModule

Development mode: short termOS environment compatibility

Production mode:1-3 years of OSenvironmentcompatibility

Building VM images of the production environment

…Production mode,

long term OS environment

compatibility, highreproducibility

vm vm vm


16

INTEGRATION PLATFORM IMPROVEMENTS A new version branch of Cmd3Fwk software integration product

(v2.0) which should replace current production version v1.3.x in the near future is being prepared for the release

The new version should provide the features missing which are required both for production and development environment, e.g.: Centralized log messages management Internal debugging mechanisms Extensible XML configuration support Configuration time validation mechanisms Multiple dependency resolution modes Revised module registration interface Much more advanced data access mechanism … and more

Migration to the new version will require changes onlyin a small part of the code written by users

3/26/2009


17

Generic reconstructio

n object container

type2

EXPERIMENTAL EVENT MODEL

3/26/2009

CmdEvent (high level container)CollectionCollection

Collection…

Generic reconstructio

n object container

type1…

Associations between

containers of type1 & type2

…

…

registeringcontainers

Example:type1: list<Track>

type2: list<ClbrHit>link(t1-t2): list< Track,

list<ClbrHit> >

Links of arbitrary complexity are

supported, though the use of multiple levels of nested containers must

be justified

Our users are strongly encouraged to not to put interdependency information

into reconstruction objects directly


18

DATA I/O MECHANISMS

3/26/2009

CmdEventCollectionCollection

Collection…

ROOT FileTTree

ROOT I/OModules

RAW Data File (Midas)

Midas Format

Read Module

Data

Data

Data

…

Cmd3Fwk

CmdEvent container normally isnot exposedto the user modules

Standalone user

analysis code (final

stages)

Both CMD-3 simulation and reconstruction data are stored in ROOT TTree representation


19

SOFTWARE BUILD SYSTEM

3/26/2009

Software sourcerepository (SVN)

Software buildrepository (NFS)

Configure

Generic offline tools, inc. Cmd3Fwk

Subdetector reconstruction toolsOffline Modules

Experimentalevent object

representations

make

ROOT CINT

createsource

snapshotgeneratingROOT dictionaries

make install


20

GENERIC RECONSTRUCTION LAYOUT

3/26/2009

Hit Hit StripHit

TowerHit Hit Hit

DC ZC LXe BGO CsI

ClusterTrack

StripHit

TowerHit

StripCluster

TowerCluster

Cluster

Cluster

TrackTrackClust

er

Vertex

Vertex

Vertex

ParticleCandidate

Subd

etec

tor R

econ

stru

ctio

nGl

obal

Rec

onst

ruct

ion

21

OFFLINE RECONSTRUCTION: DC

21

21

Garfield simulation of the driftlines in 1.5T magnetic field

Vertexfinding

OFFLINE RECONSTRUCTION: LXEProjections of the reconstructed cosmic track


23

OFFLINE RECONSTRUCTION: BGO

3/26/2009

Finding clustersin BGO calorimeter


24

OTHER OFFLINE SOFTWARE COMPONENTS

Detector geometry and material description(used also in Geant4 simulation)

Detector and experimental event visualization (custom solution)

Trigger simulation (required also for the offline processing) Offline calibration DBs and calibration procedures

for detector subsystems Reconstruction XML configuration DB and a web

interface on top of it Test production environment (SL5 x86_64):

host systems (Linux cluster) test XEN virtual machines

Tools for running production simulation/reconstruction jobs

3/26/2009


25

CONCLUSION CMD-3 detector commissioning is now being finalized Main reconstruction algorithms required by detector

subsystems individually are implemented, the work on their improvement and validation is in progress

Many auxiliary components are in production state and extensively used in test runs (calibration tools, event display, etc.)

The prototype of the offline data processing environment is implemented and being exploited on the data produced byGeant4 detector simulation and the first experimental dataobtained during the test runs

Next step is to put the whole system into production with a new version of the offline software integration framework

Global reconstruction algorithms are yet to be implemented

3/26/2009

CMD-3 Detector Offline Software Development

Documents

Transcript of CMD-3 Detector Offline Software Development