2 nd December 2002James Magowan - Surrey e-Science Day1 James Magowan IT Specialist Dynamic...

2nd December 2002 James Magowan - Surrey e-Science Day 1

James MagowanIT SpecialistDynamic e-business, IBM UK, Hursley Lab


IBM Hursley - Background

3000 Employees

Largest Software Development Lab (of any organisation) outside of the USA

IBM Software Group – Middleware

• CICS• MQ Series• Java Technology Centre• Advanced Technology & eSolutions


Computing:The Grid

Distributed

Networking:TCP/IP

Information:World Wide Web

Communications:e-mail

Internet Evolution


Virtual, collaborative organizations sharing applications and data in an open heterogeneous environment

A vast aggregation of geographically dispersed computing resources

Virtual Servers, Storage and Instruments

Grid Middleware

Distributed Physical Servers and Storage

Grid ComputingDistributed Computing Over the Internet Using Open Standards


StorageData

Applications

Processing I/O Operating System

Microcosm – Pre-Internet “System”Grid Computing

2nd December 2002 6

Grid Computing

....a single unified image

StorageData

Applications

Processing I/O Operating System

Macrocosm – Distributed Resources and Applications


Motivations for Grid Computing

Increase Capacity

• Exploit distributed resources to provide capacity for high-demand applications

– Existing applications that cannot be run effectively on a single processor

– New large scale application that provide strategic business advantages



Increase Capacity

• Exploit distributed resources to provide capacity for high-demand applications

Improve Efficiency / Reduce Costs

• Reduce infrastructure cost associated with over-provisioned resources

• Reduce the cost of manpower to manage and configure resources



Reduce “Time to Results”

• Exploit opportunities for parallel computing to allow business critical computation to be completed in a timely fashion

• Gain competitive advantage by allowing computation to be executed morefrequently and on customer demand

112

2

3

4567

8

9

1011

March

29March

28March

27

Serial Execution

Parallel Execution



Reduce “Time to Results”• Exploit opportunities for parallel computing to

allow business critical computation to be completed in a timely fashion

• Gain competitive advantage by allowing computation to be executed morefrequently and on customer demand

Enable Collaborations• Enable collaboration across

applications to integrate results • Support large multi-disciplinary

collaborations• Both within a single organization

and between partners

Simulation

PricingDesign

Design

Design Analytics


IBMIBMIBM

Provide Reliability / Availability

• Use distributed resources • Monitor work progress• Restart failed jobs


112234

567891011

JobScheduler

TIMEOUT !

JOB 1JOB 1 JOB 2JOB 2 JOB 3JOB 3JOB 1JOB 1Recovery / Restart


Provide Reliability / Availability

• Use distributed resources • Monitor work progress• Restart failed jobs

Support Heterogeneous systems

• Different hardware, system platforms, and available middleware

• Specialized equipment


Linux / Z-OS

IBM

IBM

AIX / Linux

IBM

IBM

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6

serverpSeries

IBM

H C R U6


Uses of Grid Technology4 Models, Unique Value Propositions

Increased

Processing

“Aggregate processing power from a distributed collection of heterogeneous systems”

Customer Values:

ProductivityFlexibilityResource useReliability/ Availability

ComplexityTotal cost of ownership

Decreased


Increased

Processing


Data

“Secure access and sharing of distributed data & information ina collaborative fashion”

Customer Values:



Decreased



Increased

Processing


Resiliency“Improve the quality of service of distributed systems, despite unplanned events”

Data


Customer Values:



Decreased



Increased

Processing


Resiliency“Improve the quality of service of distributed systems, despite unplanned events”

Data


On Demand“Access data & processing capabilities in a utility-like fashion…….. Make vs. Buy”

Customer Values:


Decreased




1. Intra-Grids

Grid

NAS/SAN

Grid

NAS/SAN

Grid Deployment OptionsA Function of Business Need, Technology and Organizational Flexibility


1. Intra-Grids

2. Extra-Grids

GridGrid

NAS/SANNAS/SAN

Grid

NAS/SAN

VPN



1. Intra-Grids

2. Extra-Grids

3. Inter-Grids

GridGrid

NAS/SANNAS/SAN

Grid

NAS/SAN

VPN

Cactus

NTG

(SF) Express Project

MFG

Fin.Services



Grid & Autonomic Computing

Self-healing Discover, diagnose, and react to disruptions

Self-healing Discover, diagnose, and react to disruptions

Self-optimizingMonitor and tune resources automatically

Self-optimizingMonitor and tune resources automatically

Self-protecting Anticipate, detect, identify, and protect against attacks from anywhere

Self-protecting Anticipate, detect, identify, and protect against attacks from anywhere

Self-configuringAdapt automatically to the dynamically changing

environments

Self-configuringAdapt automatically to the dynamically changing

environmentsSelf-

ConfiguringSelf-

Configuring

Self-Healing

Self-Healing

Self-Optimizing

Self-Optimizing

Self-Protecting

Self-Protecting


IBM and Grid

GGF - Global Grid Forum

• Working Groups• Approves GRID Open Standards

OGSA - Open Grid Services Architecture

• Standard proposed by Globus and IBM• Reference Implementation (Globus-3 Toolkit)

Open Source Software


Applications

Middleware

Systems Management and Automation

Workload / Performance Management

Security

Availability / Service Management

Logical Resource Management

Clustering Services

Connectivity Management

Physical Resource Management

OSOSOS + + +

Open Grid Services Architecture


Pro

fessio

nal S

erv

ices

Network

OGSA Enabled

Storage

OGSA Enabled

Servers

OGSA Enabled

Messaging

OGSA Enabled

Directory

OGSA EnabledFile

Systems

OGSA Enabled

Database

OGSA EnabledWorkflo

w

OGSA Enabled

Security

OGSA Enabled

General Middleware

Web Services

OGSI – Open Grid Services Infrastructure

Grid ServicesSystem Management Sevices

Open Grid Services Architecture (OGSA)

Applications

Au

ton

om

ic C

ap

ab

ilit

ies

Architecture FrameworkOGSA Structure


OGSA Structure – OGSI

Architecture Framework

Web Services

HandleMapNotificationFactoryManagementRegistryLifecycleDiscovery


Grid ServicesSystem Management Services

Exploits existing web services properties

• Interface abstraction (WSDL)• Protocol, language, hosting platform independence

Enhancement to web services

• State Management• Event Notification• Referenceable Handles• Lifecycle Management• Service Data Extension



Web Services




the globus project tm

www.globus.org

• Open source• Reference

implementation

Hosting platform

(Java)

Other Possible Hosting

Platforms(environments)

Microsoft (C#)Globus (C/C++)

(Python)

OGSA Structure – Open Hosting

http://www.bea.com/

http://www.oracle.com/index.html


Web ServicesWebsphere Application Server




Websphere

OGSI

Service Collections

JobScheduling

FileTransfer

DataReplicationProvisioningLogging

ProblemDetermination

ResourceManagement

ClusterManagement

Policy


• Value-added Grid Middleware and management applications

the globus project tm

www.globus.orgTivoli

OGSA Structure – Software Evolution

• Open architecture for “Grid Services”• Some “open source” reference

implementations• Vendor provider “value added”

implementationsthe globus project tm

www.globus.org

WebSphere


European DataGrid

9 TB/day

Large Hadron Collider


European DataGrid Geographically dispersed people and resources Petabytes of data per year (100 PB by 2010) Challenge:

• Coordinated Use of Distributed computing resources • Remote software development and physics analysis• Communication and collaboration at a distance

1800 Physicists150 Institutes32 Countries


OGSA-DAI

Open Grid Services Architecture Database Access and Integration

Provide access to large scientific databases via the Grid.

Open Source and Standards


OGSA-DAI

Data Access and Integration Services

• XML and Relational Database Management Systems• Design for any JDBC-enabled RDBMs• Test with Oracle, DB2, MySQL

Languages

• Java Handle Large Datasets –Optimised Performance

• BLOB’s and/or rows• Multiple Protocols for Data Transport (SOAP/HTTP and

GridFTP) Globus

• Globus-2 (GSI & GridFTP) and Globus-3


e-Diamond - UK Digital Mammography Archive

Support for Breast Cancer Diagnosis, Treatment, Teaching and Epidemiological Studies

National eScience Centre Initiative


e-Diamond - UK Digital Mammography Archive

“One of the pilot e-Science projects is to develop a digital mammography archive, together with an intelligent medical decision support system for breast cancer diagnosis and treatment. An individual hospital will not have supercomputing facilities, but through the Grid it could buy the time it needs. So the surgeon in the operating theatre will be able to pull up a high-resolution mammogram to identify exactly where the tumour can be found”

Tony Blair (speech to the Royal Society – 23 May 2002)


James MagowanIT SpecialistDynamic e-business, IBM UK, Hursley Lab

2 nd December 2002James Magowan - Surrey e-Science Day1 James Magowan IT Specialist Dynamic...

Documents

Transcript of 2 nd December 2002James Magowan - Surrey e-Science Day1 James Magowan IT Specialist Dynamic...