Scaling and Validation Programme

David Groep & vle-pfour-teamVL-e SP Meeting 2004.12.07

NIKHEFSARALogicaCMGIBM

Aims of the Scaling & Validation (S&V)

• Build environments for experimentation different scales, with a variety of resources stable quasi-production Proof-of-Concept (PoC)

• Support applications stable, but also extend (and improve) continuously deploy VL software and components from elsewhere

• Disseminate knowledge about them tutorials, events, helpdesk

• Learn ourselves from the deployment close the cycle, improve with developers and users S&V needs applications for a realistic workload

Positioning of P4 in the Project

Application development

NL-Grid production clusterCentral mass-storage facilities

+SURFnetInitial compute

platform

Stable, reliable, testedCert. releases Grid MW & VL-

software

VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping Environment

DAS-2, local resources

VL-e Certification Environment

NL-Grid FabricResearch Cluster

Test & Cert.Grid MW & VL-software

Compatibility

Flexible, test environment

Environments

Usage

Characteristics

Virtual Lab.rapid prototyping

(interactive simulation)

Flexible, ‘unstable’

external developments

Food Informatics

Dutch Telescience

Medical Diagnosis &

Imaging

Bio-Informatics Data

Intensive Science/

Bio-Diversit

y

Three tracks

• Software engineering Rapid Prototyping Certification PoC

• Deploy a national-scale infrastructure central facilities (clusters, storage,DBs) access nodes on-site software installations

• Knowledge dissemination and import tutorials (and support for ~) pilots (think: SRB) developer & admin events (think: GANG)

PoC Software Release Methodology

Application development

NL-Grid production clusterCentral mass-storage facilities

+SURFnetInitial compute

platform

Stable, reliable, testedCert. releases Grid MW & VL-

software

VL-e Proof of Concept EnvironmentVL-e Rapid Prototyping Environment

DAS-2, local resources

VL-e Certification Environment

NL-Grid FabricResearch Cluster

Test & Cert.Grid MW & VL-software

Compatibility

Flexible, test environment

Environments

Usage

Characteristics

Virtual Lab.rapid prototyping

(interactive simulation)

Flexible, ‘unstable’

Download RepositoryPoC InstallerCluster Tools

Developer CVSNightly builds

Unit testsstable, tested

releases

Integration testsFunctionality testsAdventurous application people

PoC Release n Release Candidaten+1

Developers Heaven/Haven

Tagged Release Candidates

GT2 GT2

GT3.2

Software engineering process and tools

• Source code repository (CVS) also for general VL-e use

• For generally deployed VL-e components software engineering guidelines unit and integration test framework automatic builds

• Software download repository

Example: CruiseControl

Integrated results of unit tests

No tests yet

Artifacts produced for deployment

Deployment: VL-e Proof-of-Concept

• Today: Basis of the PoC is an enhanced

Globus Toolkit™ 2.x serviced by VDT VLAM-G installed **

o Matisse database for module definitionso s/w installed when cvs restructuring complete**

Tools from EDG and CrossGrid SRB (in pilot phase)

• your requirements feed into upgrades

Continuous updates

• Requirements from applications gathered in (bilateral) meetings

• Based on your use cases we will modify, upgrade, and update the PoC other workflow tool… other database server… … ‘time-boxed’ releases

• PoC should be unified generic infrastructure

Resources and systems

• Central resource sites today SARA: NCF NL-Grid Matrix cluster SARA: Storage (Teras), SRB Matrix NIKHEF: NDPF LCG2PROD + NCF GFRC all: services

• Access node install via PoC Installer ‘keep pressing enter and you get there…’

• Supported OS’s RedHat Enterprise 3 + clones (CentOS, SL)

(works on Fedora Core *) RedHat 7.3

Subject to change

coordinated with applications,

s/w, and resource providers

PoC InstallerEase installation of GT2 and VO support:

• a User Interface & job submission i/f

• GridFTP server

• Authentication and VO-LDAP

• Globus 2 Development environment

Dissemination & Support

• Core Grid and GT2 tutorials • Hands-on web services event

the Grid Admin Nerd Group (GANG)

• User help-desk for the PoC (020) 592 8008, grid.support@sara.nl

• Documentation

• Technology pilots: SRB, …

Example: the GANG

• expose people to the use of web/grid services December event based on GT3.2 future will use GT4, Axis+Apollo, …

• this is the likely future for new software leverage industry support and standards Web Service Resource Framework, Notification, …

now in Apache Incubator

OGSA to describe service architecture

• event will be repeated with new technology

organized by LogicaCMG, IBM, NIKHEF, SARA together with GridForum Nederland

Composition of P4 team

• NIKHEF, SARA, LogicaCMG, IBM

• will grow to ~8 FTE effective in 2005 still one position to be filled @NIKHEF

Summary

• Infrastructure: the PoC is there for you hardware, Grid software, DBs, VLAM-G s/w

• Software engineering support• Central services are available

SourceForge look-alike software distribution, web site, mailing lists

• Support and training

• Think about your requirements and wishes – and YOU tell us what you want! Type of support, tools, … We could take the PoC to your site (if you want) and we will visit you again…

Contacts

• Contact addresses davidg@nikhef.nl – services, PoC, web, PoC-installer ronalds@nikhef.nl – certification, testing, tooling, … maurice.bouwhuis@sara.nl – PoC@SARA, SRB pilot, … grid.support@{sara,nikhef}.nl – user questions

Or reach all of us atvle-pfour-team@sara.nl