Holding slide prior to starting show. Grid Projects at WeSC: Synergies and Opportunities David W....

Holding slide prior to starting show

Grid Projects at WeSC: Synergies and Opportunities

David W. Walker

School of Computer Science

Cardiff University

http://www.cs.cf.ac.uk/User/David.W.Walker/

26 November 2003 WeSC Seminar 3

Overview of Activities• Software development of middleware, tools, and

problem-solving environments.– Triana – UDDIe – JACAW– SWFL/JISGA – G-QoSM – MEDLI

• Funded research projects (RC&EU):– GSiB – WOSE – BD-World– GridLab – PASOA – e-HPTX– GridOneD – GENSS

• Five collaborative industrial projects (DTI).• Also work in patterns and operators, performance and

evaluation, semantic web technologies, and Grid economies.


Collaborative Industrial Projects

• Constraint-Oriented Negotiation in an Open Information Services Environment (CONOISE-G). Alex Gray

• Collaborative Virtual Teams (COVITE). Alex Gray and John Miles

• Environment for Industrial Design Optimisation (DIPSO). Omer Rana

• Grid-Enables Computational Electromagnetics (GECEM). David Walker

• Resource-Aware Visualisation Environment (RAVE). David Walker


Staff Researchers in COMSC

• Nick Avis: Medical imaging, collaborative visualisation.• Alex Gray, Andrew Jones, Jianhua Shao: Bio-

Informatics, information/knowledge management• Yan Huang: Jini-based Grid middleware, workflow

description, composition, deployment and enactment• Omer Rana: QoS frameworks, provenance and

metadata issues, agent technologies• Ian Taylor: APIs for Grid computing, workflow,

composition.• David Walker: PSEs/portals, workflow, visualisation

and Grid applications.


Other Cardiff Researchers

• Sathyaprakash (PHYSX)

• Peter Kille (BIOSC)

• John Miles (ENGIN)

• Also interest in PSYCH,ENCAP, ARCHI, EARTH, and UWCM.


Grid Middleware

• Lightweight service-oriented architecture for grids based on Jini and P2P technologies.

• Workflow tools and description languages• Grid execution environments• Quality of service frameworks• Provenance and other metadata issues• Collaborative visualisation and collaborative

working


Web Services

• Everything is a (Web/Grid) service.

• This includes:– Computation routines– Access to files and databases– Components of the Grid infrastructure,

such as workflow enactment engines, resource monitors, etc.


A Common Approach to Workflow

• Visual service composition.

• Service interfaces and other metadata expressed in an XML-based service description document.

• Services registered with, and discovered through, a registry.


Workflow Within an SOA• Workflow description

– SWFL

• Visual composition of services– Triana

• Aggregation– Higher level services and applications

• PSE (or portal) for deploying. Managing,and monitoring services, applications and grid resources.– GSiB


JISGA

• JISGA consists of two main parts:– A WorkflowEngine service– A JobProcessor service

• Grid application is submitted to a WorkflowEngine service as SWFL.

• Sequential jobs are handled directly by the WorkflowEngine service.

• Parallel jobs involve multiple JobProcessor services.


What the Workflow Engine Does

• Converts a SWFL description of a composite service-based job into an executable Java code, and executes it.

Java Executable Code

<?xml…><JFlowModel……………………….……….….

SWFL description

IntermediateFlowModel

object

SWFL2Graph Graph2Java

SWFL2Java

To be continued…


WOSE

• Workflow Optimisation Services for e-Science Applications.

• Middleware Open Call.• Collaboration with Imperial College and

Daresbury Lab.• £400k, 2 years, one postdoc at each site• Status:

– Advertising for postdocs– 1 Dec 2003 start date


WOSE Overview

• Draws together JISGA and Triana work at CU, with ICENI at IC, and portal expertise at DL.

• Topics addressed– Service aggregation and deployment– Runtime discovery and late binding of services– Service discovery and selection from multiple

semantically equivalent services


Quality of Service Framework

• Service discovery using QoS properties• Guarantees QoS at the application, middlware and

resource levels (similar to DiffServ), and establishes Service Level Agreements

• Support for QoS adaptation• Implementation using GARA/DSRT, NRM/Diffserv

BB, and UDDIe• UDDIe supports the description of a service through

service properties, and service discovery based on these properties.


Provenance and Metadata

• Important in many middleware and application projects.

• Two main middleware projects– PASOA: Provenance-Aware Service-Oriented

Architecture– GENSS: Grid-Enabled Numerical and Symbolic

Services

• Also key in BD-World application project.


PASOA• Provenance-Aware Service-Oriented

Architecture.• Fundamental Computer Science for e-Science

call.• Collaboration with Southampton Univ.• £443k, 3 years, one postdoc and student each.• Status:

– Recently funded– Aiming for 1 Feb 2004 start date.


PASOA outline• Execution and service provenance in relation

to workflow enactment. • Algorithms to reason over provenance data,to

help scientists to achieve better utilisation of Grid resources for their specific tasks.

• Generating provenance data in workflow enactment.

• Properties that can be deduced from provenance-based data.

• Prototype that supports provenance generation and reasoning in Grid environments.


Collaborative Visualisation

• Central to two joint industrial projects– RAVE: Resource-Aware Visualisation

Environment.– GECEM: Grid-Enabled Computational

Electromagnetics.


RAVE Project

• Resource-Aware Visualisation Environment• Status:

– Started 1 April 2003. – Collaboration agreement in place.

• Partners: SGI and ORNL• Duration: 3 years• Partner contribution: £150,000 (SGI)• EPSRC/DTI contribution: £186,534• Staff: Dr Ian Grimstead hired as postdoc.


RAVE Overview

• Aims to develop a collaborative visualization environment that scales across a wide range of network-enabled devices.

• Will respond to changes in network bandwidth and capabilities of the target display device.

• Will start by examining VizServer and COVISE systems


GECEM Project• Grid-Enabled Computational Electromagnetics • Status:

– Start date 1 May 2003. – Collaboration agreement not in place yet

• Partners: Swansea University, BAE Systems, Hewlett-Packard, and Singapore Institute of High Performance Computing

• Duration: 2 years• Partner net contributions: £113,750 (BAE Systems),

£113,750 Hewlett-Packard• EPSRC/DTI contribution: £227,500 • Staff: Postdoc in place at Swansea; waiting for work

permit for CU postdoc


GECEM Overview

• Aims to use and develop Grid technology as an enabler of large-scale and globally-distributed scientific and engineering research.

• The focus of the project will be collaborative numerical simulation and visualisation between the UK and Singapore.


Two Hard Problems

• Semantic specification of applications• Scheduling of workflow nodes on distributed

resources.– Early binding model: bind to specific service/platform

at composition time (“validation”).– Intermediate binding model: bind at “compile” time

(when converting from XML to executable form).– Late binding model: bind dynamically at runtime.

• Later binding allows the use of more up-to-date information to make scheduling decisions.


Key Research Problems

• Semantic specification of applications• Scheduling of workflow nodes on distributed resources.

– Early binding model: bind to specific service/platform at composition time (“validation”).

– Intermediate binding model: bind at “compile” time (when converting from XML to executable form).

– Late binding model: bind dynamically at runtime.• Later binding allows the use of more up-to-date information to

make scheduling decisions.• How to deal with “volatile” services• Also need to discuss limitations of workflow approach.


A WeSC Service Repository

• We need to create a service repository for the publishing and discovering services created in WeSC projects.

• Initially based on UDDI 3.0.

• Will allow projects to make use of and experiment with services developed by other projects.


Summary of Activities

• Lightweight Grids.• Visual Service Composition Environment for

creating services and applications based on workflow

• Workflow enactment and execution environments.

• Collaborative visualisation.• Quality-of-Service.• Provenance and metadata.

Holding slide prior to starting show. Grid Projects at WeSC: Synergies and Opportunities David W....

Documents

Transcript of Holding slide prior to starting show. Grid Projects at WeSC: Synergies and Opportunities David W....