Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly...

18
Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb . It is optimised for the configuration and management of jobs that use the Gaudi / Athena Framework common to the two experiments. Developer’s page Savannah portal CVS Code repositories Tags form Release information Wiki Collaboration Page (user feedback, discussions) Technical notes Logbook Ganga usage statistics Dashboard Monitoring of ATLAS Ganga jobs User guides Installation & downloading Bugs Reporting a new bug Browse open , resolved or closed bugs. Project Members Meetings Presentations Documents Mailing lists Useful links

Transcript of Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly...

Page 1: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration and management of jobs that use the Gaudi / Athena Framework common to the two experiments.

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page (user feedback, discussions)Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

User guidesInstallation & downloadingBugsReporting a new bugBrowse open, resolved orclosed bugs.

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 2: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

GlobusEnabling Grids for E-Science and industry in Europe (EGEE)EU DataGrid Project (EDG)UK Grid for Particle Physics (GridPP)US Particle Physics Data Grid (PPDG) US Grid Physics Network (GriPhyN)ALICE Environment (AliEn)Grid Access Portal for Physics Applications (GRAPPA)Grid-enabled web environment for site-independent user job submission (GENIUS)

CERN LHC Computing Grid Project (LCG)LCG applications areaCore serveices and libraries (SEAL)Persistency framework: pool of persistent objects for LHC (POOL)Physicist interface (PI)Architectrue for Distributed Analysis (ARDA)LHC Experiment Dashboard

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 3: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Experiment’s homepageComputingWiki pagesUS physics, software and computingGrid activitiesUS Grid activitiesData ChallengesAthena FrameworkAthena Startup Kit (AthASK)Atlas Commander (AtCom)

Distributed interactive analysis of large datasets (DIAL)Package manager (Pacman)Production and Distributed Analysis (Panda)ATLAS distributed analysis (ADA)ADA meetingsDistributed Data ManagementVirtual Organisation Membership Service

Experiment’s homepageComputingWiki pagesGrid activitiesData ChallengesProduction overviewGaudi FrameworkSimulation for LHCb and its intergrated control environment (SLICE) (production system for 2002)Distributed Infrastructure with Remote Agent Control (DIRAC) (production system from 2003)

DIRAC agent installationDIRAC job submission to LCG: Quick Start GuideDIRAC APIDIRAC ServersDIRAC monitoring: Production ServerDIRAC monitoring: Analysis ServerDIRAC monitoring: Test ServerVirtual Organisation Membership ServiceGrid usage

Page 4: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

reStructured Text: markup syntax and parser component of DocutilsDocutils: documentation utilitiesEpydoc: tool for generating API documentation for Python modulesHappyDoc Documentation Extraction Tool

Core Infrastructure Centres (CIC) operation portalGrid Load (Grid monitoring system hosted by Imperial College)UK Virtual Organisation Member ServiceParticle DataGroup

Page 5: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Several Ganga-related mailing lists have been set up, as detailed below. Most of these are managed within the framework of the CERN Simba 2 ListBox Service, so that you can subscribe using the Simba 2 web interface, and to access the archives you may need to give your username and password for the CERN Mail Server. The ATLAS HyperNews list is an exception, and to be subscribe to this, or access the forum, you need to have registered using the ATLAS HyperNews New Membership pageGeneral information on Ganga project

[email protected] View list archive

List for Ganga developers project-ganga-

[email protected] list archive

ATLAS HyperNews forum for Ganga user support hn-atlas-

[email protected] View HyperNews forum

LHCb user support for Ganga and distributed analysis lhcb-distributed-

[email protected] list archive

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 6: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

• A. Maier, F. Brochu, U. Egede, J. Elmsheuser, B. Gaidioz, K. Harrison, B. Koblitz, H.C. Lee, D. Liko, J. Moscicki, A. Muraru, V. Romanovsky, A. Soroko, C.L. Tan Ganga - an optimiser and front-end for Grid job submission (abstract [html]) Contribution to Second EGEE User Forum, Manchester, 9-11 May 2007 • J. Moscicki The Ganga job-management system (abstract [html]) Contribution to XI International Workshop on Advanced Computing and Analysis Techniques in Physics Research (ACAT 2007), Amsterdam, 23-27 April 2007 • LHCb Software Course outshines Hogmanay celebrations (html) GridPP News, 25th January 2007

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick Ganga: a Grid user interface for distributed analysis (paper [pdf], doc])

Proc. Fifth UK e-Science All-Hands Meeting, Nottingham, UK, 18th-21st September, Ed. Simon J. Cox (National e-Science Centre, 2006) pages 518-525 Workshop proceedings (pdf)

2007, 2006, 2005, 2004, 2003, 2002, 2001

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 7: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick

Page 8: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

• ATLAS Software Workshop, CERN, 25th-29th June 2007 - Dietrich Liko: ATLAS distributed analysis (pdf, ppt) • LCG Applications Area meeting, CERN, 23rd May 2007 - Jakub Moscicki: The Ganga job management system (pdf) • 30th LHCb Software Week, CERN, 7th-11th May 2007 - Will Reece: Ganga status update (pdf, ppt) • ATLAS distributed analysis tutorial, Valencia, 3rd-4th May 2007 - Johannes Elmsheuser: Distributed analysis using Ganga (pdf)

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick Ganga: a Grid user interface for distributed analysis (paper [pdf], doc]) Proc. Fifth UK e-Science All-Hands Meeting, Nottingham, UK, 18th-21st September, Ed. Simon J. Cox (National e-Science Centre, 2006) pages 518-525 Workshop proceedings (pdf)

2007, 2006, 2005, 2004, 2003, 2002, 2001

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 9: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Links to the agendas, minutes and presentations for GANGA meetings are provided below. A separate page lists Ganga presentations at other meetings. Weekly telephone meetings are held Thursdays at 15:00 CERN time (14:00 UK, 09:00 NY), and make use of the CERN self-service teleconferencing. To join the meeting do one of the following: Call the CERN self-service teleconferencing at +41 22 767 6000 and enter the access code 0127472; Have the system call you by following the instructions at https://audioconf.cern.ch/call/0127472. If choosing the first option, callers from the UK can connect at local rates by using Telediscount. The access number for Switzerland is: 0844 462 96 96.

• K. Harrison, C.L. Tan, D. Liko, A. Maier, J. Moscicki, U. Egede, R.W.L. Jones, A. Soroko, G.N. Patrick Ganga: a Grid user interface for distributed analysis (paper [pdf], doc]) Proc. Fifth UK e-Science All-Hands Meeting, Nottingham, UK, 18th-21st September, Ed. Simon J. Cox (National e-Science Centre, 2006) pages 518-525 Workshop proceedings (pdf)

2005, 2004, 2003, 2002User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 10: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Check if the bug report contains a vaild request. If it is a non-issue then close the report immediately with a comment.

If you start fixing the bug then close the report ONLY AFTER: it has been confirmed to work by the original submitter or after it's been shown to work in the release at which the fixed was aimed (preferably with the test case)

Information for package developers:

Administrative information: CVS, AFSPackage Release and TaggingTesting PolicyTesting FrameworkInstall and Test Prerelease Version

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 11: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

./ganga-install --prerelease --extern=<packages> --prefix=<path> VERSION

Just before an official release is to be published, developers are encouraged to test the pre-release version To install locally the pre-release version use the --prerelease option of the installer option:

Packages in Ganga CVS which are maintained by independent developers have independent version numbers. The tagging information is stored in ganga/release/config/PACKAGES* directory in CVS. When you are ready with your changes, commit and tag your package in the CVS. Then use the tag form to update your tag and release notes for the next release(s). This information will be used to generate the PACKAGES-<version> and ReleaseNotes-<version> files. The good practice for package version numbers: GangaPackage-X-Y GangaPackage : it is a name which corresponds to the functionality of your package and its location in CVS X : a major version number increased with incompatible changes or changes required in other packages Y : a minor bugfix, no incompatible changes

Page 12: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

AtlasLHCb Gaudi LCG Physicist Interface (PI) ProjectLCG Services and Libraries (SEAL) Project LCG Persistency Framework (POOL)LCG Simulation ProjectCLHEP

Browse Ganga CVS online: CVS Web View CVSMore infromation

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 13: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

29 June 2005: Karl Harrison, Chun Lik Tan, Frederic Brochu, Jakub Moscicki and Dietrich Liko - Status of Ganga for Distributed Analysis in ATLAS (doc) 02 February 2005: Ganga 4 Reengineering Page19 January 2005: Jakub Moscicki - A CLIP implementation of LHCb use cases (with comments from Ulrik Egede) (pdf, sxw)6 January 2005: Ulrik Egede - LHCb Use cases

2005, 2004, 2003User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 14: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

It will take you 10 seconds to start Ganga at CERN and 30 seconds to download and install on a laptop. Please complain to us if it takes more time.Installation instructions

Ganga 4.3.6 released 05 July 2007

Access to release area for all available Ganga versions

Working with Ganga - read this tutorial firstGUI manual – read Working with Ganga document first

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 15: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Ganga section of ATLAS WorkBook - tips on setting up Ganga, and running simple jobsGanaga tutorial for ATLAS users – read in conjunction with Working with Ganga documentUsing Ganga to submit jobs to NorduGrid – assumes familiarity with Ganga tutorial for ATLAS usersUsing Ganga for physics analysis – recipe from B-Physics working groupWarning: still being testedUsing Ganga for small-scale production – recipe from B-Physics working groupATLAS reference manual for Ganga Public Interface (GPI) – documentation for Ganga classes, functions and objects available to ATLAS users

LHCb Ganga manual – read Working with Ganga document firstGanga tutorial for LHCb users – can be complemented by other tutorials form the 3rd LHCb-UK Software Course Edingbrough, 8th-10th January 2007LHCb referance manual for Ganga Public Interface (GPI) – documentation for Ganga classses, functions and objects available to LHCb users

Page 16: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

Developer’s Survival Guide

Old documentation for versions 4.0.1, 3,2,1

Migration from Previous ReleasesTo make sure you get the latest changes in the configuration file run ganga a single tiem with the –g option.

Page 17: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

GangaEnvgangaAll LHCb plugins will be automatically enabled

05 July 2007: Latest public release is 4.3.6

source /afs/cern.ch/sw/ganga/install/etc/setup-atlas.shgangaAll Atlas plugins will be automatically loaded

On lxplus no installation is needed, you may just start Ganga directly: /afs/cern.ch/sw/ganga/install/4.3.6/bin/ganga

For convinience you may set your PATH or make an alias. You will get the default set of plugins. You may enable new plugins in the ~/.gangarc config file.

User guidesInstallation & downloading

Developer’s pageSavannah portalCVS Code repositoriesTags form Release informationWiki Collaboration Page Technical notesLogbookGanga usage statisticsDashboard Monitoring of ATLAS Ganga jobs

Project MembersMeetingsPresentationsDocumentsMailing listsUseful links

Page 18: Ganga (Gaudi / Athena and Grid Alliance) is an interface to the grid that is being developed jointly by ATLAS and LHCb. It is optimised for the configuration.

First download the Ganga installation script: ganga-install A given version of Ganga for slc3_gcc32 can then be downloaded and installed using: python ganga-install options VERSION where you need python 2.2 or greater. You can optionally request installation of additional Ganga packages, and the external packages on which they depend. The following are strongly recommended:

•For Atlas: --extern=GangaAtlas,GangaNG,GangaCronus,GangaGUI,GangaPlotter •For LHCb: --extern=GangaLHCb,GangaGUI,GangaPlotter •GUI only: --extern=GangaGUI,GangaPlotter

For information on additional options, use: python ganga-install --help To enable the ATLAS runtime plugins, use:

•bash: export GANGA_CONFIG_PATH=GangaAtlas/Atlas.ini •tcsh: setenv GANGA_CONFIG_PATH GangaAtlas/Atlas.ini

In addition, to enable use of the Cronus and NorduGrid plugins, you should change RUNTIME_PATH in your GangaAtlas/Atlas.ini file to:

•RUNTIME_PATH = GangaAtlas:GangaCronus:GangaNG This is likely to become the default in a future Ganga release. To enable the LHCb runtime plugins, use: bash: export GANGA_CONFIG_PATH=GangaLHCb/LHCb.ini tcsh: setenv GANGA_CONFIG_PATH GangaLHCb/LHCb.ini Troubleshooting Failed to fetch http file: check your network connection and try if you can do wget http://cern.ch/ganga/index.html. If the network is OK then check if you do not have the web server proxy. Web server proxy may occasionally cause problems to download ganga via http. You can fix this problem by setting the environment variable. bash: export http_proxy=http://ip.or.name.of.the.proxy.server csh: setenv http_proxy http://ip.or.name.of.the.proxy.server