APEL Accounting Update Dave Kant CCLRC, e-Science Centre.
-
Upload
jordan-willis -
Category
Documents
-
view
217 -
download
0
description
Transcript of APEL Accounting Update Dave Kant CCLRC, e-Science Centre.
APELAccounting Update
Dave KantCCLRC, e-Science Centre
EGEE’03, April 2005 - 2
Overview
This is a summary of the status of Accounting & Reporting following its deployment in LCG2_4_0
1. Overview2. APEL Design3. What’s New?4. LCG Accounting (OSG , NorduGrid, EGEE)
EGEE’03, April 2005 - 7
72 Sites publishing data to GOC (May 17th 2005)
Over 1.7 Million Job records
~ 50K records per week
http://goc.grid-support.ac.uk//
EGEE’03, April 2005 - 15
Whats New?
Added LHC View to the reporting interface• Requirements driven by RRB / Kors Bos• Tier-1 and Country entry points• LHC VO only• All data normalised in units of 1000 . SI2000 . Hour• Tabular Summaries per Tier1/ Country
Breakdown of Usage per Country / Tier1
Tier-1 Resource Usage per LHC VO
Graphical Plot showing Usage per VO
Select Tier1 View / Country View using navigable tree
Select time frameDrill Down to view data belonging to individual sites within a Tier1/country
EGEE’03, April 2005 - 17
Batch System Support
• APEL supports PBS and LSF• Implementations are separate and independent of
one another. Currently LCG2_4_0 has PBS support only.
• What is the current status about LSF Support?• http://goc.grid-support.ac.uk/gridsite/accounting/lsf_dev.
html• On this page you find release candidate
• edg-rgma-lsf-1.0.0-4.noarch.rpm
• LSF currently comes in three flavours (4, 5 and 6), each has a different usage record format
• New RPM edg-rgma-apel-lsf has been released to CERN for testing.
• Expect both to be part of LCG 2_5_0
EGEE’03, April 2005 - 18
CERN Data
Meeting with Harry and Thorsten (17th May)• Run APEL-LSF over full 2005 dataset• Compare APEL numbers to internal LSF reporting
EGEE’03, April 2005 - 19
LSF Deployment at CERN
1. PBS and LSF implementations use the same APEL core. This has some consequences for the normalisation model.
1. We use a global parameter to normalise data belonging to each site which is published by the CE.
2. PBS uses internal normalisation model, lsf has no such model3. Two ways to perform nomalisation in lsf:
1. normalise record by record (exact but requires converting a cpu factor into an SI00 value) or
2. globally (approximate method) as is done with PBS.
2. CERN has a heterogeneous farm. They calculate a weighted mean on a daily basis and publish this value on the CE (via ldif : GlueHostBenchMark00)
3. Accuracy of Method1. We will compare numbers with CERN very shortly.
EGEE’03, April 2005 - 20
Accounting Dissemination
1. CERN Courier 2. LCG Computing
Newsletter (slightly more technical)
3. AHM 2005 (more technical still)
EGEE’03, April 2005 - 29
APEL and gLite
Is APEL integrated in g-Lite?• Work currently in progress.• We have ported the APEL code into the gLite CVS repository but need
to understand functional differences e.g. WMS and use of Condor• 3 Components: Core + PBS plugin + LSF plugin• Sent our requirements to Erin Laure….waiting for information.
What about its deployment plan? • As soon as possible• …but would also like to add some new features
– Global Job ID to link with L&B – dn to VO mapping
EGEE’03, April 2005 - 30
LCG Accounting
Project involves combining results from all three infrastructures and presenting an aggregated view
Peer Infrastructures in LCG• Open Science Grid (Ruth Pordes, Philippe Canal, Matteo Melani)• Nordugrid (Per Oster)• EGEE• Currently, LHCView filters LHC VO data from EGEE accounting data.
EGEE’03, April 2005 - 31
Requirements
Combine results from all three infrastructures …
Ideally: Distributed queries to multiple databases• Each peer manages an accounting database• LHC VO filtering provided through a web services interface
Initial Implementation: Centralised Collection• Peers publish data into a global database• WebServices or direct MySql inserts
Common Problem: Different Grid infrastructures may use different Schemas. GGF define a schema, but quite flexible.
May need “translators” to convert from one schema to another. (already exist)