Denis Caromel GridCOMP Scientific Coordinator [email protected] Beijing, October 2007

WP5 - GridSystems' Use Cases (May 2007)

Grid programming with components: an advanced COMPonent platform for an effective invisible grid 2006-2007 GridCOMP Grids Programming with components. An advanced component platform for an effective invisible grid is a Specific Targeted Research Project supported by the IST programme of the European Commission (DG Information Society and Media, project n034442)

Denis CaromelGridCOMP Scientific Coordinator [email protected], October 2007
Middleware Comparison
/ GCM

Luxembourg - 6/7 February 2006 - TEL Projects meeting


ProActive / GCM Overview


ProActive Parallel Suite (1)





Integrated development environments (IDE)


Rich GUI for Grid / Parallel Programming


Component GUI under Dev. at Westminster Univ.



Programming Models




Programming models
Basicprogramming models supportedby the middleware



Scopes and Objectives:Grid Codes that Compose and DeployNo programming, No Scripting, No Pain

Innovation:Composite Components Multicast and GatherCast
GCM +
MultiCast
GatherCast


Data Management


Data management: File transfer
Currently ProActive provides support for the following type of transfers:To a remote node (Push)From a remote node (Pull)It could be used at:Deployment time (XML)Retrieval time (XML)Execution time (Objects API)



Infrastructure: Resource management & SLA


Scheduler: User Interface


Scheduler: Resource Manager Interface


Infrastructure management
Virtualisation of multi-core, multi-processors, servers, clusters: YesProActive/GCM makes it possible to:Virtualize the infrastructure using host and VM capacityMap the application onto the resulting virtual infrastructureMeta scheduler, Broker: YesProActive Scheduler and Infrastructure Manager, two graphical application, can talk with primitive schedulers. Scheduler: YesProActive Scheduler can deploy directly a list of jobs over various machines.Peer to peer support for auto-organization : YesDesktop P2P can be used to build desktop grid. Windows and Unix for heterogeneous grid. No need to change code of ProActive/GCM applications.Hardware resources handled by the middleware: computers: YesNetworks, storage devices, scientific instruments: NoSecurity:AIC: Authentification, Integrity, confidentiality : Yesboth middleware and application communications are handled by mechanismRole management: PartiallyFull support for construction & administration of VO: No


Orthogonal questions
Globus based or notNo

Approach to SOAYes, both component and Active Object can be exported as Web Service. An application can itself registered into a registry and lookup other services for SOA integration. Dynamic reconfiguration of Components

Main languages: (C++, Java, C#, Fortran...)Java as principal language, and support wrapping of native legacy applicationsInteroperability with other Grid middleware: Yes !!SSH, RSH, OARSH, ARC NorduGrid, CGSP China Grid, EGEE gLite, Fura/InnerGrid, GLITE, GLOBUS, GridBus, IBM Load Leveler, LSF, MPI, OAR, PBS, PRUN, SGE, Sun Grid Engine, UNICORE



Applications


Current GCM experiments in ProActive
JEM3D: 3D electromagnetic application: a single Cp on 300+ CPUs on Grid

Vibro-Acoustic application with EADS (legacy MPI coupling)


Applications: successfully executed

GridCOMP use casesBiometric Identification SystemDays Sales OutstandingThe mean time that clients delay to pay an invoiceEDR processingExtended Data Record, extract, transform and Load in database large amount of dataWingDesignComputes the aerodynamic wing performance for a given configuration


Other ProActive/GCM applications
Scilab Parallel execution of Scilab Scripts on clusters and desktop grids.MathlabBIDistribution of databases for Business IntelligenceFlowShop A distributed approach for solving the FlowShop problem.Nbody The nbody problem (SPMD)NQueen An example of the N Queen problem distributed JEM3D An electromagnetism application C3DA distributed and collaborative 3D renderer.Monte carloStock pricing (EU and USA options)ProPotts MonteCarlo simulation for NLPOPSSA On-line power systems security analysisSalishan A resource sharing and simulation problemSAT solverDistributed SAT solvers in Java based on the SAT4J libraryProFractal Parallel fractal image generationGeB Grid-enabled BLASTCruise Control A real-time java controller to maintain a cruise speedNASNASA benchmark suite (5 cores/5, 1 application/7)

e-science, e-business, e-engineering


Distribution model
Open SourceYes, the source code is under GPL version 2 licence.Close SourceNoCommercial supportYes, the ActiveEon company provides commercial support for the ProActive/GCM middleware.


ActiveEon


Summary-Perspective:
Full-fletched Upper Ware
Active ObjectsSPMD





Form of GCM Interoperability Deployment
Just an XML Schema: Specifies the deployment of the application Virtual Nodes onto the infrastructure (machine, OS, protocols, schedulers, etc.)Example: EGEE gLite schema:


XML Deployment Descriptors


Components and SOA
From the review:

Definition:Components: Services and SCA:In GridCOMP:Autonomicity, ReconfigurationLookup, Dynamic Bindings, etc.


ProActive/GCM Component Definition
A component:Is formed from one (or several) Active ObjectIs executed on one (or several) JVMProvides a set of server ports: Java InterfacesUses a set of client ports: Java AttributesUses Point-to-point or Group communication between componentsHierarchical:Primitive component: define with Java code and a descriptorComposite component: calls in compositesDescriptor: composition of primitive + composite Parallel component: multicast of XML definition of primitive and composite (ADL)Virtual nodes capture the deployment capacities and needsVirtual Node is a very important abstraction for GRID components


GCM Architecture


GCM Technical Structure
Component Specification as an XML schema or DTD

Run-Time API defined in several languagesC, Java

Description and Information for Deployment(XML DD, Virtual Nodes, File Transfer, )

Packaging described as an XML schema


Main Technical Activities --- Objectives
WP 2, Task 2.1: Primitive Component Programming WP 2, Task 2.2: Legacy Code Wrapping, Interoperability

WP 2, Task 2.3: Composition, Composite, and Deployment

For memory:Autonomic features (WP 3)IDE for GCM, Composition GUI (WP 4)


Collective Interfaces


Collective Interfaces
Simplify the design and configuration of component systemsExpose the collective nature of interfacesCardinality attributeMulticast, Gathercast, gather-multicast

The framework handles collective behaviour at the level of the interfaceBased on Fractal API : Dedicated controller Interface typing Verifications


Multicast interfaces
Transform a single invocation into a list of invocationsMultiple invocationsParallelismAsynchronismDispatch Data redistribution (invocation parameters)Parameterisable: Distribution FunctionBroadcast, scatteringDynamic redistribution: Dynamic dispatchResult = list of results




Results as lists of results
Invocation parameters may also be distributed from lists


Gathercast interfaces
Transform: a list of invocations a single invocationSynchronization of incoming invocations~ join invocationsTimeout / Drop policyBidirectional Bindings (callers callee)Data gatheringAggregation of parameters into lists
Result: Redistribution of resultsRedistribution function


Industrial exploitation : ActiveEon start-up
Support and Services around ProActive+GCMUse standard GCM deployment for all developmentsUse standard GCM programming model to develop components, and build component applications Collaboration with GridSystems


ActiveEon


ActiveEon


ActiveEon



More to come on Friday:ETSI TC Grid Meeting


Example


Apologies for being tough with refusal of August 30th Evaluation Date
Samuel, August 30th 2007, 19H07, 3.7 Kg


Denis Caromel GridCOMP Scientific Coordinator [email protected] Beijing, October 2007

Documents

Transcript of Denis Caromel GridCOMP Scientific Coordinator [email protected] Beijing, October 2007