Denis Caromel Scientific Coordinator [email protected] Beijing, October 2007

WP5 - GridSystems' Use Cases (May 2007)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 2006-2007 GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n034442)

Denis CaromelScientific Coordinator [email protected], October 2007
Overview of

Luxembourg - 6/7 February 2006 - TEL Projects meeting

GCM Partners


GCM: Grid Component ModelGCM Being defined in the NoE CoreGRID (42 institutions)Open Source ObjectWeb ProActive implements a preliminary version of GCMService Oriented: NESSI relation

GridCOMP takes:GCM as a first specification, ProActive as a starting point, and Open Source reference implementation.
The vision: GCM to be the GRID GSM


GSM and GCM Pictures
GSM cells:

Components:


GSM and GCM Pictures: Europe


GSM and GCM Pictures: China and Asia


ETSI Grid Plugtests, 04, 05, 06, 07
MelbourneNancyMetzNapoli
NancyNapoli NiceMetzParis
PiseRennesSantiago San DiegoBeijing
AmsterdamBelfastFribourgGrenobleLille
ManchesterMelbourneMeridaMetzBombay


Overview of Project

Interoperability Objectives:Interoperability with other standards: EGEE gLite, UNICORE, NorduGrid, Globus, Web Services, LSF, IBM LL, SGE, CGSP China Grid, etc.,A GCM ETSI Official Public Standard
Objectives:

GRID PROGRAMMING WITH COMPONENTS: AN ADVANCED COMPONENT PLATFORM FOR AN EFFECTIVE INVISIBLE GRID


WP 2:Component Framework Implementation



Scopes and Objectives:Grid Codes that Compose and DeployNo programming, No Scripting, No Pain

Innovation:Composite Components Multicast and GatherCast
GCM +
MultiCast
GatherCast

ProActive / GCM Environment



WP 3: Non Functional Component Features


WP3 - Non Functional Component Features
A framework for autonomic management of componentsDesign of the manager structureDefinition of proper implementation mechanismsCharacterisation of properties handled through autonomic managers

Preliminary experiments on parallel component compositionsAutonomic management of simple performance contracts Assessment of the autonomic framework features

Layered design of non functional component featuresIdentify proper mechanisms to handle dynamic adaptation of componentsDefine local strategies to achieve contractually specified QoS goalsDefine orchestration strategies preserving local to global effectInteraction with use cases to assess/improve NF framework



WP 4: Grid IDE for Programmers and Composers



WP4 - Grid IDE for Programmers and Composers
Provides an integrated programming and composing GUIOffers facilities to bind both normal code and legacy code into primitive componentsEnables assembly of Grid applications Provides launcher tools for the deployment of a component applicationOffers a steering tool for installing, removing, and re-installing new versions of component codeProvides a tool for the monitoring of resources


Component GUI under Dev. at Westminster Univ.



WP 5: Use cases


WP5 Use cases
Three main partners: IBM, Atos Origin, GridSystemsFour use cases:Biometric Identification System (IBM)Identify people solely on their biometric information (1:N match)Management Application Computing of DSO value (ATOS)Days Sales Outstanding is the mean time that clients delay to pay an invoice to an companyTelecom EDR Processor (GS)Aerospace Wing Design (GS)computes the aerodynamic wing performance for a given configuration, using a legacy fortran-77 app



Conclusion:
GCM Interoperability DeploymentDynamicity, Autonomicity, SOA integrationFull-fledged Environment:




ETSI GCM TC Grid Standard
GCM Interoperability Deployment

GCM Application Description

GCM Fractal ADL (Architecture Description Language)

GCM Management (Java, C, WSDL API)
Work Item No 1


GSM cells:

Components:






GridCOMP and SOA

Components: The binding is achieved at deployment, statically. Services: a new binding can occur at each invocation, depending of SLA, conditions, etc. What we aime at in GridCOMP:A Cp system with dynamic reconfiguration including new bindingsDynamic Dispatch Multicast, Autonomicity, Reconfiguration

Other relations to SOA:Register and LookupESBCode provisioning



Conclusion:
GCM Interoperability DeploymentDynamicity, Autonomicity, SOA integrationFull-fledged Environment:



Components and SOA
Definition:Components: Binding at configuration and deployment Services and SCA: Binding potentially at each callIn GridCOMP:Autonomicity, ReconfigurationLookup, Dynamic Bindings, etc.


ProActive/GCM Component Definition
A component:Is formed from one (or several) Active ObjectIs executed on one (or several) JVMProvides a set of server ports: Java InterfacesUses a set of client ports: Java AttributesUses Point-to-point or Group communication between componentsHierarchical:Primitive component: define with Java code and a descriptorComposite component: calls in compositesDescriptor: composition of primitive + composite Parallel component: multicast of XML definition of primitive and composite (ADL)Virtual nodes capture the deployment capacities and needsVirtual Node is a very important abstraction for GRID components


GCM Architecture


GCM Technical Structure
Component Specification as an XML schema or DTD

Run-Time API defined in several languagesC, Java

Description and Information for Deployment(XML DD, Virtual Nodes, File Transfer, )

Packaging described as an XML schema


Main Technical Activities --- Objectives
WP 2, Task 2.1: Primitive Component Programming WP 2, Task 2.2: Legacy Code Wrapping, Interoperability

WP 2, Task 2.3: Composition, Composite, and Deployment

For memory:Autonomic features (WP 3)IDE for GCM, Composition GUI (WP 4)


Collective Interfaces


Collective Interfaces
Simplify the design and configuration of component systemsExpose the collective nature of interfacesCardinality attributeMulticast, Gathercast, gather-multicast

The framework handles collective behaviour at the level of the interfaceBased on Fractal API : Dedicated controller Interface typing Verifications


Multicast interfaces
Transform a single invocation into a list of invocationsMultiple invocationsParallelismAsynchronismDispatch Data redistribution (invocation parameters)Parameterisable: Distribution FunctionBroadcast, scatteringDynamic redistribution: Dynamic dispatchResult = list of results




Results as lists of results
Invocation parameters may also be distributed from lists


Gathercast interfaces
Transform: a list of invocations a single invocationSynchronization of incoming invocations~ join invocationsTimeout / Drop policyBidirectional Bindings (callers callee)Data gatheringAggregation of parameters into lists
Result: Redistribution of resultsRedistribution function


Status of GCM in ProActive
Partial implementation: ADL schema, API, Multicast, Gathercast, Component GUI (prototype)

Distributed components for various applications: Numerical, Legacy,

Achieved experiments:A component application on up to 300+ CPUs



IC2D and Generic Environment

Eclipse GUI


Prototype : GUI for Components




TimIt Automatic Timers in IC2D


ProActive / GCM Environment


Current GCM experiments in ProActive
JEM3D: 3D electromagnetic application: a single Cp on 300+ CPUs on Grid

Vibro-Acoustic application with EADS (legacy MPI coupling)


Scheduler: User Interface


Scheduler: Resource Manager Interface


XML Deployment Descriptors


Component GUI under Dev. at Westminster Univ.



Activity Summary


First Year: Main Dates, Official Meetings
Project kick-off meeting: Sophia Antipolis, at INRIAs facility, on June 1st -2nd 2006.

Second meeting, Steering Committee,Brussels, September 19, 2006 Third meeting Sophia Antipolis, GRIDs@work, ETSI, November, 2006. - ProActive GCM user group Nov. 28th, 2006 (40 p.) - ProActive GCM tutorial Nov. 29th, 2006 (20 p.) WP4 technical meeting March 2007, London, with Technical presentations and architecture decisions Fourth meeting, GridSystems facility, Palma on 22-23 May 2007.


First Year: Main Technical Dates
15 September 2006: Release of an improved GCM prototype within the ProActive library, available on svn GForge

January 2007 : SR ProActive 3.2, stable core GCM features:Collective Interfaces : allows handling of multiple bindings between components through a single interface. Conformance tests : partial support of the Fractal conformance tests Pi Tutorial using components: a step by step GCM tutorial

April 2007 : ProActive 3.2.1Components : Bug in multicast interfaces with round-robin mode. Timit : capacity to time executions and produce charts. Deployment : Improved correctness and understandability of deployment descriptors examples.

Palma 22-23 May: First Successful Demos of AutonomicityFirst Successful Demos of Use CasesFirst Successful Demos of IDE First contribution from Outside of EU


First Year: Main Dates, GCM Public Presentations
GridCOMP website was launched in July 2006. At GRID@Asia meeting on 11-12 December 2006 in Seoul, a presentation on GridCOMP and GCM was given by Yu Feng (INRIA). Two GCM presentations in Australia (meeting in Melbourne 29th January, and an invited talk at the conference, February 1st, ACSW 2007 in AusGrid), D. Caromel. INRIA co-organized a ProActive/GCM tutorial in Beijing, China, on 26-27 April 2007 with the EchoGRID project meeting, on 24-27 April 2007 in Beijing China: 90 registrations!GCM-ProActive Tutorial at CCGrid 2007, M. Morel et. al, May 14th 2007, Rio, Brazil AFTER THE REVIEW PERIOD:CoreGRID Workshop, in June 12-13, 2007, Heraklion - Crete, GreeceCoreGrid, CoreGRID Symposium, August 27-28, 2007, Rennes, FranceUNIPI course on Autonomic Component Management (WP3), PhD course named Advanced parallel programming, June - July 2007, Pisa.CoreGrid Summer School Research Europe, 3-7 Sept. 2007, Budapest, Hungary

UPCOMING :HPC-GECO/CompFrame 2007, 21-22 October 2007, Montreal, CanadaProActive/GCM workshop and tutorial at GRIDs@Work, ETSI: IV Grid PlugTests, 31 Oct.-1 Nov. 2007, Beijing


First Year: STANDARDIZATION MEETINGS
ETSI TC GRID Standardization Meeting, Sophia Antipolis, 1st Dec. 2006 Second TC Grid meeting held in Sophia Antipolis in February 2007

OGF 20 meeting, 7-11 May 07, Manchester, UK, two presentations: Vladimir Getov (UoW) gave an invited talk on Grid Component Model and platform: An Overview at the Workshop on Evolutions of GRIDs towards SOKUs, 8 May 07. Denis Caromel (INRIA) formally propose GCM to ETSI TC Grid, 11 May 07 : Formal acceptation of GCM Deployment as TC Grid Work Item with formal support from IBM, Huawei, Orange, Alcatel-Lucent,

AFTER THE REVIEW PERIOD: 20th Sept. 07, @ ETSI, 4th TC Grid Meeting, D. Caromel GCM update

ETSI TC Grid Standardization Meeting 1st November 2007, Beijing


Activity Summary
06, June 1st: Kickoff06, September: Release of an improved GCM prototype

07, January: release of stable core GCM features

07, April : New GCM implementation release

07, May, OGF week, ETSI TC GRID: Acceptation of GCM Deployment as Work Item: Std

07, May: First Successful Demos of Autonomic Features IDE, Use CasesFirst contributions from Outside of EU




Exploitation Plan


Academic exploitation : INRIA
Industrial feedbackvalidation of the GCM programming modelnew development, feature requestAcademic feedbackvalidation of the programming modellaunch new collaboration and research (Pisa, )Already 3 thesis (not funded by GridCOMP) using the prototype implementation:Component reconfiguration (Marcela Rivera), GCM extensions for autonomic applications (Paul Naoumenko), Specification Languages and Model-Checking (Antonio Cansado)Use in other project (EU and not)


Industrial exploitation : ActiveEon start-up
Support and Services around ProActive+GCMUse standard GCM deployment for all developmentsUse standard GCM programming model to develop components, and build component applications Collaboration with GridSystems


ActiveEon


ActiveEon


ActiveEon


ActiveEon



Standardization and WP 7Collaboration

Denis [email protected]


Collaboration WP 7
GridCOMP project participated to the EC workshop on Collaboration activities and Concertation meeting preparation

March 27 2007 in Brussels. Many Grid project representatives were gathered and had the opportunity to interact with each other.


Task 3: Co-ordination of standardisation efforts
Philipp Wieder, Research Centre Juelich: in charge of this task to discuss GridCOMP contribution. in the first step the person responsible for standardisation collaboration in GridCOMP, Denis Caromel becomes the contact point for GridCOMP at the Grid Standards Coordination Group. Denis Caromel has thus been registered into the [email protected] mailing list and got a BSCW account.Philipp Wieder will circulate information about plans and achievements within the Grid Standards Coordination Group.

In addition, GridCOMP via INRIA and others has already collaborated with several EU projects such as g-Eclipse, AKOGRIMO, QosCosGrid, NESSI-Grid, EchoGrid, Bridge, EGEE to initiate Grid discussions. These contacts will be carried on further


Just coming back from FMCO07, Amsterdam
Concertation meeting with:GridCOMP + CoreGRID

SENSORIA Soft. Eng. For Service Oriented overlay computerCREDO Modeling and analysis of evolutionary structure for dist. servicesSelfMan Self management of large-scale distributed systems, P2P + ComponentARTIST Design process and archi for Real-time embedded soft.MOBIUS Proof carrying code


ETSI GCM TC Grid Standard
GCM Interoperability Deployment

GCM Application Description

GCM Fractal ADL (Architecture Description Language)

GCM Management (Java, C, WSDL API)
Work Item No 1


Form of GCM Interoperability Deployment
Just an XML Schema: Specifies the deployment of the application Virtual Nodes onto the infrastructure (machine, OS, protocols, schedulers, etc.)Example: EGEE gLite schema:



More to come on Friday:ETSI TC Grid Meeting


Summary and Conclusion
Technical work: Well under wayEvaluation: Excellent by 3 experts

Concertation: Strong, with many EU projectsGrid Plugtests (IV): GridCOMP is Key to itStandardization: officially under way with ETSIStartup: a company as an offspring



Apologies for being tough with refusal of August 30th Evaluation Date
Samuel, August 30th 2007, 19H07, 3.7 Kg


Example


Task 4: Technical Inventory and Information Service

Technical InventoryThe coordination has recently moved from NEXTGRID to BEinGRID projectit is now Kostas Kavoussanakis from the Edinburgh Parallel Computing Centre (EPCC), who is now responsible of this activity.

BEinGRID project plans to send a policy for inclusion of other projects output in the BEinGRID repository. Once the policy will be circulated, GridCOMP will discuss with BEinGRID its potential contribution.


Denis Caromel Scientific Coordinator [email protected] Beijing, October 2007

Documents

Transcript of Denis Caromel Scientific Coordinator [email protected] Beijing, October 2007