Holding slide prior to starting show

Grid Projects at WeSC: Synergies and Opportunities

David W. Walker

School of Computer Science

Cardiff University

http://www.cs.cf.ac.uk/User/David.W.Walker/

26 November 2003 WeSC Seminar 3

Overview of Activities• Software development of middleware, tools, and

problem-solving environments.– Triana – UDDIe – JACAW– SWFL/JISGA – G-QoSM – MEDLI

• Funded research projects (RC&EU):– GSiB – WOSE – BD-World– GridLab – PASOA – e-HPTX– GridOneD – GENSS

• Five collaborative industrial projects (DTI).• Also work in patterns and operators, performance and

evaluation, semantic web technologies, and Grid economies.

26 November 2003 WeSC Seminar 4

Collaborative Industrial Projects

• Constraint-Oriented Negotiation in an Open Information Services Environment (CONOISE-G). Alex Gray

• Collaborative Virtual Teams (COVITE). Alex Gray and John Miles

• Environment for Industrial Design Optimisation (DIPSO). Omer Rana

• Grid-Enables Computational Electromagnetics (GECEM). David Walker

• Resource-Aware Visualisation Environment (RAVE). David Walker

26 November 2003 WeSC Seminar 5

Staff Researchers in COMSC

• Nick Avis: Medical imaging, collaborative visualisation.• Alex Gray, Andrew Jones, Jianhua Shao: Bio-

Informatics, information/knowledge management• Yan Huang: Jini-based Grid middleware, workflow

description, composition, deployment and enactment• Omer Rana: QoS frameworks, provenance and

metadata issues, agent technologies• Ian Taylor: APIs for Grid computing, workflow,

composition.• David Walker: PSEs/portals, workflow, visualisation

and Grid applications.

26 November 2003 WeSC Seminar 6

Other Cardiff Researchers

• Sathyaprakash (PHYSX)

• Peter Kille (BIOSC)

• John Miles (ENGIN)

• Also interest in PSYCH,ENCAP, ARCHI, EARTH, and UWCM.

26 November 2003 WeSC Seminar 7

Grid Middleware

• Lightweight service-oriented architecture for grids based on Jini and P2P technologies.

• Workflow tools and description languages• Grid execution environments• Quality of service frameworks• Provenance and other metadata issues• Collaborative visualisation and collaborative

working

26 November 2003 WeSC Seminar 8

Web Services

• Everything is a (Web/Grid) service.

• This includes:– Computation routines– Access to files and databases– Components of the Grid infrastructure,

such as workflow enactment engines, resource monitors, etc.

26 November 2003 WeSC Seminar 9

A Common Approach to Workflow

• Visual service composition.

• Service interfaces and other metadata expressed in an XML-based service description document.

• Services registered with, and discovered through, a registry.

26 November 2003 WeSC Seminar 10

Workflow Within an SOA• Workflow description

– SWFL

• Visual composition of services– Triana

• Aggregation– Higher level services and applications

• PSE (or portal) for deploying. Managing,and monitoring services, applications and grid resources.– GSiB

26 November 2003 WeSC Seminar 13

JISGA

• JISGA consists of two main parts:– A WorkflowEngine service– A JobProcessor service

• Grid application is submitted to a WorkflowEngine service as SWFL.

• Sequential jobs are handled directly by the WorkflowEngine service.

• Parallel jobs involve multiple JobProcessor services.

26 November 2003 WeSC Seminar 15

What the Workflow Engine Does

• Converts a SWFL description of a composite service-based job into an executable Java code, and executes it.

Java Executable Code

<?xml…><JFlowModel……………………….……….….

SWFL description

IntermediateFlowModel

object

SWFL2Graph Graph2Java

SWFL2Java

To be continued…

26 November 2003 WeSC Seminar 16

WOSE

• Workflow Optimisation Services for e-Science Applications.

• Middleware Open Call.• Collaboration with Imperial College and

Daresbury Lab.• £400k, 2 years, one postdoc at each site• Status:

– Advertising for postdocs– 1 Dec 2003 start date

26 November 2003 WeSC Seminar 17

WOSE Overview

• Draws together JISGA and Triana work at CU, with ICENI at IC, and portal expertise at DL.

• Topics addressed– Service aggregation and deployment– Runtime discovery and late binding of services– Service discovery and selection from multiple

semantically equivalent services

26 November 2003 WeSC Seminar 18

Quality of Service Framework

• Service discovery using QoS properties• Guarantees QoS at the application, middlware and

resource levels (similar to DiffServ), and establishes Service Level Agreements

• Support for QoS adaptation• Implementation using GARA/DSRT, NRM/Diffserv

BB, and UDDIe• UDDIe supports the description of a service through

service properties, and service discovery based on these properties.

26 November 2003 WeSC Seminar 19

Provenance and Metadata

• Important in many middleware and application projects.

• Two main middleware projects– PASOA: Provenance-Aware Service-Oriented

Architecture– GENSS: Grid-Enabled Numerical and Symbolic

Services

• Also key in BD-World application project.

26 November 2003 WeSC Seminar 20

PASOA• Provenance-Aware Service-Oriented

Architecture.• Fundamental Computer Science for e-Science

call.• Collaboration with Southampton Univ.• £443k, 3 years, one postdoc and student each.• Status:

– Recently funded– Aiming for 1 Feb 2004 start date.

26 November 2003 WeSC Seminar 21

PASOA outline• Execution and service provenance in relation

to workflow enactment. • Algorithms to reason over provenance data,to

help scientists to achieve better utilisation of Grid resources for their specific tasks.

• Generating provenance data in workflow enactment.

• Properties that can be deduced from provenance-based data.

• Prototype that supports provenance generation and reasoning in Grid environments.

26 November 2003 WeSC Seminar 22

Collaborative Visualisation

• Central to two joint industrial projects– RAVE: Resource-Aware Visualisation

Environment.– GECEM: Grid-Enabled Computational

Electromagnetics.

26 November 2003 WeSC Seminar 23

RAVE Project

• Resource-Aware Visualisation Environment• Status:

– Started 1 April 2003. – Collaboration agreement in place.

• Partners: SGI and ORNL• Duration: 3 years• Partner contribution: £150,000 (SGI)• EPSRC/DTI contribution: £186,534• Staff: Dr Ian Grimstead hired as postdoc.

26 November 2003 WeSC Seminar 24

RAVE Overview

• Aims to develop a collaborative visualization environment that scales across a wide range of network-enabled devices.

• Will respond to changes in network bandwidth and capabilities of the target display device.

• Will start by examining VizServer and COVISE systems

26 November 2003 WeSC Seminar 25

GECEM Project• Grid-Enabled Computational Electromagnetics • Status:

– Start date 1 May 2003. – Collaboration agreement not in place yet

• Partners: Swansea University, BAE Systems, Hewlett-Packard, and Singapore Institute of High Performance Computing

• Duration: 2 years• Partner net contributions: £113,750 (BAE Systems),

£113,750 Hewlett-Packard• EPSRC/DTI contribution: £227,500 • Staff: Postdoc in place at Swansea; waiting for work

permit for CU postdoc

26 November 2003 WeSC Seminar 26

GECEM Overview

• Aims to use and develop Grid technology as an enabler of large-scale and globally-distributed scientific and engineering research.

• The focus of the project will be collaborative numerical simulation and visualisation between the UK and Singapore.

26 November 2003 WeSC Seminar 27

Two Hard Problems

• Semantic specification of applications• Scheduling of workflow nodes on distributed

resources.– Early binding model: bind to specific service/platform

at composition time (“validation”).– Intermediate binding model: bind at “compile” time

(when converting from XML to executable form).– Late binding model: bind dynamically at runtime.

• Later binding allows the use of more up-to-date information to make scheduling decisions.

26 November 2003 WeSC Seminar 28

Key Research Problems

• Semantic specification of applications• Scheduling of workflow nodes on distributed resources.

– Early binding model: bind to specific service/platform at composition time (“validation”).

– Intermediate binding model: bind at “compile” time (when converting from XML to executable form).

– Late binding model: bind dynamically at runtime.• Later binding allows the use of more up-to-date information to

make scheduling decisions.• How to deal with “volatile” services• Also need to discuss limitations of workflow approach.

26 November 2003 WeSC Seminar 29

A WeSC Service Repository

• We need to create a service repository for the publishing and discovering services created in WeSC projects.

• Initially based on UDDI 3.0.

• Will allow projects to make use of and experiment with services developed by other projects.

26 November 2003 WeSC Seminar 30

Summary of Activities

• Lightweight Grids.• Visual Service Composition Environment for

creating services and applications based on workflow

• Workflow enactment and execution environments.

• Collaborative visualisation.• Quality-of-Service.• Provenance and metadata.

26 November 2003 WeSC Seminar 31