History, motivations

25
CMT 1 HEPVis Sep. 1999 C. Arnault LAL History, motivations Started in 1993 for providing support for horizontal software development at LAL After an evaluation of the autoconf world, decision was taken to create a new package basic requirements were : » emphasis on simplicity to use and to understand by non software experts » should be based on a real conceptual model study of works » SEI : definitions of Configuration Management (IEEE Std-729-1983) » Inspiration from the CMM (freely interpreted) Promote Human Oriented Software

description

History, motivations. Started in 1993 for providing support for horizontal software development at LAL After an evaluation of the autoconf world, decision was taken to create a new package basic requirements were : emphasis on simplicity to use and to understand by non software experts - PowerPoint PPT Presentation

Transcript of History, motivations

Page 1: History, motivations

CMT

1HEPVis Sep. 1999 C. Arnault LAL

History, motivations• Started in 1993 for providing support for horizontal

software development at LAL

• After an evaluation of the autoconf world, decision was taken to create a new package

– basic requirements were :» emphasis on simplicity to use and to understand by non

software experts

» should be based on a real conceptual model

– study of works» SEI : definitions of Configuration Management (IEEE Std-729-

1983)

» Inspiration from the CMM (freely interpreted)

• Promote Human Oriented Software

Page 2: History, motivations

CMT

2HEPVis Sep. 1999 C. Arnault LAL

The configuration management definition• IEEE Std-729-1983

“Configuration is the process of identifying and defining the items in the system, controlling the change of these items throughout their lifecycle, recording and reporting the status of items and change requests, and verifying the completeness and correctness of items”

• Identificationdescribes the system structure, the nature of its elements,

their identity, and gives access to each item version

• Controlorganises versions and changes to system items while

keeping coherency and consistency on the complete system.

Page 3: History, motivations

CMT

3HEPVis Sep. 1999 C. Arnault LAL

Principles of the package• Based on some scenarios

– Projects or sub-projects promote part of their software base as reusable

– Projects are divided into small groups of closely related developers

– Integration phases are iterative and must remain under the control of the package managers

– Configuration parameters should be defined/specified independently to their implementation

– Configuration parameters should be queried at any time outside of any effective software operation

– Maintain several concurrently visible versions of each package

– ...

Page 4: History, motivations

CMT

4HEPVis Sep. 1999 C. Arnault LAL

Principles of the package• …and on some definitions and conventions

– the package» the smallest autonomous entity in the software base

» support use relationships to other packages

» the physical organisation of packages is independent of their logical structure

– the configuration parameters» identify and describe the constituents of a package

• applications, libraries, …

» set up the environment needed to develop the package• environment variables, make macros, include paths, …

» set up the environment needed to operate the package• environment variables, aliases, setup scripts, …

Page 5: History, motivations

CMT

5HEPVis Sep. 1999 C. Arnault LAL

Principles of the package• …and on some definitions and conventions

– physical organisation

<root1>/<package>/<version tag>/src

/mgr

/<conf. tag1>

/<conf. tag2>

<root2> >/<package>/<version tag>/src

/mgr

/<conf. tag1>

/<conf. tag2>

/...

Page 6: History, motivations

CMT

6HEPVis Sep. 1999 C. Arnault LAL

Principles of the package• ...conventions

– configuration tag» reflects the machine type, the operating system, the context

(debug, gnu, etc…)

» may be either automatically from system features (uname, fs sysname, …) or freely defined

– Site tag» freely defined tag for site-dependent symbol values

Page 7: History, motivations

CMT

7HEPVis Sep. 1999 C. Arnault LAL

Principles of the package• Operations

– The configuration parameters are exploited by CMT to configure the various tools used during the software process activities, or to generate some documents:

» CVS• chain the used packages

» [g]make• transparent generation of all makefiles

» MS Developer Studio• generation of workspace and project files

» Unix shells• generation of environment variables, aliases, paths, etc...

» Web pages• generation of the software configuration documentation

Page 8: History, motivations

CMT

8HEPVis Sep. 1999 C. Arnault LAL

Conceptual and user requirements– Packages are autonomous entities

– Packages are related to each other by use relationships

– Package evolution and history is specific to each package

– Configuration parameters are any information required to» identify the constituents of the package

» set up the environment needed to develop and operate the package

» describe the use relationships between packages

– Version tag semantics reflects backward compatibility or incompatibility

– Configuration parameters are propagated through the chain of use relationships.

B CusesusesA

Page 9: History, motivations

CMT

9HEPVis Sep. 1999 C. Arnault LAL

Design• Based on the modelling of the configuration

management parameters– one object model (used to build tools)

– one syntax (used to make conf. Parameters persistent)

use

script

symbol_value

macro

alias

set

symbol

tag

setup_script

cleanup_script

library

applicationsource_file

constituent

n

n

n

n

n

1

document

generatorn

Page 10: History, motivations

CMT

10HEPVis Sep. 1999 C. Arnault LAL

Implementation• Parameters are stored and maintained within one

textual file per package named requirements

• Use a simple (easy to read) and homogeneous syntax

• One basic parser application implements the object model from the textual representation (cmt_parser.exe)

• One main user interface (cmt) to the parser provides the command oriented (à la CVS) interface used for

– querying the configuration parameters

– generate the effective environments» for package development

» for package usage

Page 11: History, motivations

CMT

11HEPVis Sep. 1999 C. Arnault LAL

Implementation• A Java browser

• and editor

The package search list

Packages

Versions for this package

The requirements file

The used packages

The constituents

The symbols defined in this package

Page 12: History, motivations

CMT

12HEPVis Sep. 1999 C. Arnault LAL

Some scenarios• Creating a simple test application, using some

existing packages.1> cd ...

2> edit requirements

use Atlas v1

use Mylib v2

application myapp A.cxx B.cxx C.cxx

3> cmt config # to be done only once in the package’s life

4> edit source files here

5> gmake

6> myapp.exe

7> back to 4

Page 13: History, motivations

CMT

13HEPVis Sep. 1999 C. Arnault LAL

Some scenarios• Creating a plain package

1> cmt config A v1 dev-area # to be done only once in the

package’s life

2> cd dev-area/A/v1/mgr

3> edit requirements

use Atlas v1

use Mylib v2

application myapp A.cxx B.cxx C.cxx

4> edit source files into ../src

5> gmake

6> ../${CMTCONFIG}/myapp.exe

7> back to 4 (or 3)

Page 14: History, motivations

CMT

14HEPVis Sep. 1999 C. Arnault LAL

...Some scenarios• Selecting cooperating projects

> setenv CMTPATH projectA:projectB:projectC

Page 15: History, motivations

CMT

15HEPVis Sep. 1999 C. Arnault LAL

...Some scenarios• Iterative integration

> edit use statements within the requirements

use Mylib v2r1

> source setup.csh

> gmake

> run

Page 16: History, motivations

CMT

16HEPVis Sep. 1999 C. Arnault LAL

...Some scenarios• Package evolution

> cd dev

> cmt checkout A

> cd A/v1/mgr

> source setup.csh

> gmake

> change source files

> test

> cvs commit

> cd ../

> cvs tag v1r1

> cvs release -d v1

Page 17: History, motivations

CMT

17HEPVis Sep. 1999 C. Arnault LAL

...Some scenarios• Building domain packages

– This is an interface package only containing a set of use statements towards a selection of versions for the packages belonging to a given conceptual domain

» simulation

» reconstruction

» visualisation

– A user of the simulation domain will simply use one given version of the simulation interface package, which automatically provides by transitivity (or inheritance) the appropriate selection of versions of all the simulation-related packages.

– All configuration parameters defined in these packages are therefore inherited through this domain package.

Page 18: History, motivations

CMT

18HEPVis Sep. 1999 C. Arnault LAL

Domain packages• The global “project” package

1> cmt config Atlas v1

2> cd public-area/Atlas/v1/mgr

3> edit requirements

use CxxFeatures v1r3

use CLHEP v1r4

use Simulation v1

use Reconstruction v2

use ...

user package:

use Atlas v1

Page 19: History, motivations

CMT

19HEPVis Sep. 1999 C. Arnault LAL

The services• Parameter monitoring

cmt show macro xxx show a particular macro

cmt show macros show all macros

cmt show constituents show all constituents

cmt show sets show all env. variables

• Environment generationcmt config install a package

cmt build msdev generate MSDev.Studio files

cmt build readme generate README.html

cmt checkout checkout a package from CVS

• othercmt broadcast cmd iterate a command over the

use chain.

Page 20: History, motivations

CMT

20HEPVis Sep. 1999 C. Arnault LAL

The requirements file• General syntax

package name

set name “default value” [ tag “value” ] …

alias name “default value” [ tag “value” ] …

macro name “default value” [ tag “value” ] …

application name source-file …

library name source-file …

document generator name source-file …

tag name tag-name …

include_dirs path …

setup_script name …

cleanup_script name …

etc...

Page 21: History, motivations

CMT

21HEPVis Sep. 1999 C. Arnault LAL

The requirements file, examples

package CLHEP

set CLHEPHOME "/afs/cern.ch/sw/lhcxx/specific/@sys/CLHEP/dev" \ Linux "/afs/cern.ch/atlas/project/graphics/@sys/CLHEP" \ hp_ux102 "/afs/cern.ch/atlas/project/graphics/@sys/CLHEP"

macro CLHEP_cppflags "-DCLHEP_MAX_MIN_DEFINED -I$(CLHEPHOME)/include"

macro CLHEP_linkopts "-L$(CLHEPHOME)/lib -lCLHEP -lm" \ hp_ux102 "-L$(CLHEPHOME)/lib -lCLHEP-aCC -lm"

path_append LD_LIBRARY_PATH "${CLHEPHOME}/lib"

Define an environment variable And variants forLinux or HP

Extends the C++ flags for all client packages

Will be linked by all client applications

Needed when shared libraries are used

Page 22: History, motivations

CMT

22HEPVis Sep. 1999 C. Arnault LAL

The requirements file, examples

package CxxFeatures

set CXXFEATURESHOME “${SRT_DIST}/${SRT_VERSION}/Utilities/CxxFeatures”

include_dirs ${CXXFEATURESHOME}/.srt/${SRT_TARGET} \ ${CXXFEATURESHOME}

use CLHEP v1r3use STL v1 Define additional include search paths

used in dependency buildingand in compilation

(for all client packages as well)

Transitive use statementsinherited by all client

packages

Page 23: History, motivations

CMT

23HEPVis Sep. 1999 C. Arnault LAL

The requirements file, examplespackage AgeToCxx

use CxxFeatures v0r3 Utilities

set AGETOCXXHOME “${SRT_DIST}/${SRT_VERSION}/Tools/AgeToCxx”

include_dirs ${AGETOCXXHOME}/.srt/${SRT_TARGET} ${AGETOCXXHOME}

make_fragment agetocxx_header

make_fragment agetocxx -suffix=cxx \ -dependencies \ -header=agetocxx_header

public

macro agetocxx \"${SRT_DIST}/${SRT_VERSION}/installed/${SRT_TARGET}/bin/agetocxx"

The CxxFeatures package is installed under the Utilities directory.

The agetocxx fragment will beused to generate C++ code

from Age files.The dependency builder

will be applied to age files.

Page 24: History, motivations

CMT

24HEPVis Sep. 1999 C. Arnault LAL

The requirements file, examplespackage Cm

use CSet v2r5

set CMDOMAIN "LAL” Virgo “Cascina”

alias cm "${CMROOT}/${CMTCONFIG}/cm.exe"

macro Cm_linkopts "-L$(CMROOT)/$(Cm_tag) -lCm -lm" \ LynxOS "$(CMROOT)/$(Cm_tag)/libCm.a -lnetinet -lrpc -lm"

macro_append cflags " -Dunix ”

# The Cm constituents.

library Cm -OS9 CmConnect.c CmMessage.c Cvt.c CmTransaction.c \ CmHandle.c

application NameServer -OS9 NameServer.capplication cm cm.c

The CMDOMAIN environment variablewill get different values

on different sites.

The constituents

Client packages will link to thelibrary on all supported plateforms.

Page 25: History, motivations

CMT

25HEPVis Sep. 1999 C. Arnault LAL

Status• Ported to

– all Unix flavours (Dec, AIX, SGI, SunOS, Linux, HP-UX)

– Windows 95/NT / MS Developer Studio

– LynxOS (Cetia, CES)

• Used by– the development team at LAL

– the Virgo experiment (http://www.virgo.infn.it) (Lapp Firenze Frascati IPN-Lyon Napoli LAL ESPCI Perugia Pisa Roma)

– the LHCb and Atlas experiments

– the NEMO experiment (http://www.lal.in2p3.fr/NEMO/nemo.html) (France, Russia, INEL MHC Finland Ukraine Praha)

– the AUGER experiment (http://wwwlpnhep.in2p3.fr/auger/welcome.html)