33_onwards_3rd PM IBM KeyNote Cloud Computing Garuda-Sampath

7/29/2019 33_onwards_3rd PM IBM KeyNote Cloud Computing Garuda-Sampath

1/41

2005 IBM Corporation

IBM India

From Grid Computing to Cloud Computing The IBM ApproachGaruda Partner Meet ,4th March 2008,Bangalore,India

P. Sambath Narayanan Ph.DIndia Systems & Technology LabIBM


2/41

2

Agenda

I. Important Challenges

III. Important Technologies, Trends and Standards

V. Case Studies

VII.From GRID Computing to Cloud Computing


3/41

IBM India


Terminology

Virtualization Service Orientation Data management information Data Service Policy Management Interoperable Automation

Lifecycle


4/41

IBM India


I. Some Important Challenges

Data Management Right Time & Right Data

Network Bandwidth and Latency

Security

Software and Standards

Need for many Grid based Scientific and CommercialApplications

Enable smooth scaling in many dimensions

Integration with the physical world


5/41

IBM India



6/41

IBM India



7/41

IBM India


Few Important Challenges(contnd.)

Data Management is a Challenge

Diverse usage scenarios

Volume of data - TBs

Right data at right time

Format of data

Heterogenity of systems at all level

Bandwidth, transfer, manipulation and analysis oflarge volume of data


8/41

IBM India


Few Important Challenges (contnd.)

Network Bandwidth

Large volume of data needs to be transferred across

the network

Ensuring right data to be available at the right time Latency, Bandwidth, transfer, manipulation and

analysis of large volume of data

Cost of Bandwidth


9/41

IBM India


Grid Security

Intrusion

Detection

Anti Virus

Management

Policy

Management(Auth,

Privacy,

federation

User

Managmnt.

Key

Managmnt

TrustModel

Securelogging

Policy Expression and exchange

Bindings security(transport, protocol,message

security

Secure

conversations

Credential & Id

Translation

Access control

enforcementAudit

Service/end-point

policy

Mapping

rules

Authorize

Policy

Privacy

Policy


10/41

IBM India


II. Key Technologies

Virtualization

Storage / Data Management

Grid Security

Grid Software


11/41

IBM India


Keywords

Virtualization Service SOA Data management information

Data Service Policy Management Interoperable

Automated Lifecycle


12/41

IBM India


Grid Technology Evolution

OGSA

Managed,Shared

Virtual System

Globus Toolkit

Standards,GT2

Many Deployments

Many DeploymentsScientific Applications

1990 1995 2000 2005 2007

GridAdoption&

Acceptan

ce


13/41

IBM India


Grid Open Standards

OASIS (organization for the advancement of structuredinformation standard) WS Resource Framework

WS Notification

Open Grid Services Architecture-GGF OGSA Basic Profile

OGSA Security Profile

Basic Execution Services (OGSA-BES)

Job Submission Description Language (JSDL) Data Access and Integration Services (DAIS)

Configuration Description, Deployment, and LifecycleManagement (CDDLM)

OGSA Byte I/O (Byte IO)


14/41

IBM India


OGSA Design Principles

Service Orientation to virtualize resources Everything is a service

From Web service

Standard interface service mechanisms, multiple protocolsbindings, local/remote transparency

From Grids Service semantics, reliability and security models

Life cycle management, discovery and other services

Multiple hosting environments C,J2EE,.NET


15/41

IBM India


Technology Classification & Trends

Serial Applications

Parallel Applications

Multi-threaded

MPI

open

Client Server

CORBA

COM/DCOM

.NET, J2EE

Home grown work

distribution

P2P

App Integration

Reliable Messaging

Reliable execution

Service virtualization

Web services

Service registration,

Discovery, invocation

Location independent

Lift App off the servers

Mainframes

Storage

DAS

Open Systems

Unix, Linux,Windows

Storage

DAS

Clusters

DRM

Storage

DAS

Infra. Virtualization

Grid

OGSA

Data Grid

Service provisioning

Infrastructure

Application technologies

Monolithic Open Distributed Virtualized


16/4116

Virtual I/O paths

Virtualization-Single system & partitioning

Features

Micro-partitioningShare processors acrossmultiple partitionsMinimum Partition: 1/10

processorAIX 5L V5.3 or Linux*

Virtual I/O ServerShared EthernetShared SCSI & Fiber ChannelInt Virtualization Manager

AIX 5L V5.3 & Linux partitions

Partition LoadManagerAIX 5L V5.2 & V5.3 supportedBalances Processor & memory

request

Partition Mobility

* = SLES 9 or RedHat v3 with update 3

POWER Hypervisor

LinuxAIX5L

V5.2

Dynamically Resizable

3Cores

2

Cores

ManagerServer

LPAR 2

AIX 5L V5.3

LPAR 1

AIX 5L V5.2

LPAR 3

Linux

PLM PartitionsUnmanagedPartitions

POWER Hypervisor

PLM Agent PLM Agent

AIX5L

V5.3

6Cores

6

Cores

Linux

AIX

5L

V5.3

AIX

5L

V5.3

AIX

5L

V5.3

AIX

5L

V5.3

AIX

5L

V5.3

Linux

Micro-partitioning

1Cores

Linux Linux

3Cores

AIX5L

V5.3

3Cores

EthernetSharing

Virtual I/OServer

Partition

StorageSharing

Int VirtManager


17/41

IBM India


Virtualization Information/StorageTechnology

Helps in addressing Data Management Challenges

Integrated view of storage, fs and DB driven by

standard

Data transformation, security and replication


18/41

IBM India


Virtualization - WorkloadTechnology

Workload Management Challenge

Single logical view of workload scheduling

Different type of scheduling environments and domains

Workload virtualization strategy is to create a single, logicalview of workload scheduling.

This will enable users to accelerate performance of multiple largeapplication workloads across their organization, leveraging andorchestrating IT resources in a flexible and dynamic fashion.


19/41

IBM India


Virtualize

Unlike Resources

Virtualize

the

Enterprise

Virtualize

Outside the Enterprise

Virtualize

Like Resources

Single Systems& Partitioning

Cluster

Simple(2-4)

Sophisticated(4+)

Workload Management

SAN Volume Controller

Information Integrator

GPFS

NFS V4

SAN FS

IntegratedCluster Environment

XDExtended Deployment

LoadLeveler

LoadLevelerMultiCluster

CSMCluster Systems Manager

Intelligent

Orchestrator

Systems Edition

Enterprise Edition

ProvisioningManager

EnterpriseWorkloadManager

Information

IBM Grid Toolbox

LSF

GridServer

Symphony

MultiCluster

MP Enterprise

Who does virtualization

di


20/41

IBM India


Overcoming Network Challenges

Through efficient utilization of the Network.

IBM Download Grid example. Explained in later slides.

Integrating with Global Research Networks

National Research & Education Networks.

Supporting Research and Education communities

Specialized ISP

Lambda Grid

IBM I di


21/41

IBM India


IBM I di


22/41

IBM India


Overcoming Grid Security Challenges

Three key attributes of Grid Security Model Enables integration and interoperability

Creation and management of dynamic trust domains

Supports dynamic creation of services

OGSA Security Web services security standard

Grid Security Infrastructure (GSI) Portion of the Globus tool kit that implements security function

IBM I di


23/41

IBM India


Virtualize Unlike

Resources

Virtualize the

Enterprise

Virtualize Outside the

Enterprise

Virtualize Like

Resources

Single Systems

& Partitioning

Cluster

Simple(2-4)

Sophisticated(4+)

China Grid

Ministry of EducationPeoples Republic of China

National DigitalMammography Archive

IBM

Across the Spectrum:Real Life References

IBM India


24/41

IBM India


Earth System Grid(ESG) - Case StudyOvercomingData Management Challenges

Service = RepositoryStorage Repository for Model generated atmospheric data3200 Users91,000 filesMore than 150 TB of data downloadedMore than 300 research papers

0

100

200

300

400

500

600

11/1/04

12/1/04

1/1/05

2/1/

05

3/1/

05

4/1/

05

5/1/

05

6/1/

05

7/1/05

8/1/

05

9/1/

05

10/1/05

11/1/05

12/1/05

1/1/

06

2/1/

06

3/1/06

4/1/

06

5/1/06

6/1/

06

7/1/

06

8/1/

06

9/1/06

10/1/06

GB/day

Daily 7-Day Average

GB/day

600

IBM India


25/41

IBM India


ESG Architecture & Technologies

Climate data

Metadata catalog NcML (metadata schema)

OPenDAP-G

(aggregation and subsetting)

Data management

Data Mover Lite

Storage Resource Manager

Globus toolkit

Globus Security Infrastructure

GridFTP

Monitoring and Discovery

Services

Replica Location Service

Security

Access control

MyProxy

User registration

DataSubsetting

AccessControl

UserRegistration

OPeNDAP-GMyProxy SRM DISKCache

ESG Web Portal

NCAR

Cache

NCARMSS

RLS SRM

ORNLHPSS

RLS SRM

RLS

SRM

RLS LANLCache

searchbrowse

download

WebBrowser

DMLDataUser

publishWebBrowser

DataProvider

MonitoringServices

DataPublishing

ClimateMetadata

CatalogsBrowsing

UsageMetrics

DataDownload

DataSearch

NERSC

MSS, HPSS: Tertiarydata storage systems

IBM India


26/41

IBM India


Energy Exploration Case Study

Service = Seismic Computing 3-D Seismic imaging is the most resource intensive

Grid Enabled system for Seismic ImagingGulf of Mexico 3-D Marine Surveys Estimated run times on a cluster(128 cpu,2.4 GHz,Pentium)

Compute intensive wave equation provides better accuracy

Slides are based on the work done by 3DGeo Team. See Reference Material

IBM India


27/41

IBM India


Parallelization of PSDM on Multiple Clusters

Clusters from 3DGeo

processing centres and

Clusters from SDSC MPICH-G2 / MPICH-

GP(Kum Rye Park) Globus Tool kit DCs

SantaClara,Houston,SanD

iego SupercomputingCentre

This slide is based on the work done by 3DGeo Team. See Reference Material

IBM India


28/41

IBM India


Medical Education Over Access Grid

Work Done by J.Silverstein, U. Chicago

IBM India


29/41

IBM India


National Digital Mammography ArchiveElectronic Medical Record data grid and repository

Motivation To help doctors and medical students learn more about breast

cancer and related diseases

Challenges

Managing and storing of huge files for fast retrival Annual NDMA volume could exceed 5.6 peta bytes per year

Image size 160 MB per study

Minimum daily traffic estimated 28 TB

NETwork bandwidth and response Encryption of patient data and transmission across public

networks

IBM India


30/41

IBM India


caBIG: sharing of infrastructure, applications, and data.

Data

Integration!

Service Oriented Science Cancer & Biology

IBM India
http://images.google.com/imgres?imgurl=www.nature.com/nsu/010215/images/mouse.jpg&imgrefurl=http://www.nature.com/nsu/010215/010215-1.html&h=152&w=159&prev=/images%3Fq%3Dmouse%26svnum%3D10%26hl%3Den%26lr%3D%26ie%3DUTF-8%26sa%3DNhttp://www3.cancer.gov/cancercenters/centerslist.htmlhttp://cagraidsvr06.cs.tcd.ie/index.htmlhttp://www.grid.it/


31/41

IBM India


IBM India
http://www.gridpp.ac.uk/http://grid.infn.it/index.php?infngridhttp://www.westgrid.ca/home.htmlhttp://www.grangenet.net/http://www.gridlab.org/http://www.ivdgl.org/grid2003/index.phphttp://apgrid.gridforumkorea.org/board/list.php?table=newshttp://cagraidsvr06.cs.tcd.ie/index.htmlhttp://www.grid.it/


32/41

IBM India


Technology Evolution

OGSA

Managed,Shared

Virtual System

Globus Toolkit

Standards,GT2

Many Deployments

Many Deployments

Scientific Applications

1990 1995 2000 2005 2007

GridAdoption&

Acceptance

IBM India


33/41

IBM India


Business Challenges

With demand for IT resources hard to predict, serviceproviders usually over-provision resources in order to support

peak demands and ensure continuous service availability and

quality, while other systems run at lower capacity,

IBM India


34/41

IBM India


Cloud Computing Defined

Large pools of systems are linked together to provide IT

services Service-based online economy

resources and services are transparently provisionedand managed.

IBM India


35/41


The Need - Cloud Computing

Dramatic growth in connected devices

Real-time data streams

the adoption of service oriented architectures

Web 2.0 applications

Open collaboration, social networking and mobilecommerce.

Massive increase in the scale of IT environments

driving the need to manage them as a unified cloud.

IBM India


36/41


Business SolutionAfter Cloud Computing

Cloud-computing-based technologies that will enable the

borderless delivery of IT services based on actual

demands to keep costs competitive.

Seamless delivery of services to consumers regardless

of demand or available computing resources

Virtualization and Grid Technologies

IBM India


37/41


Cloud Computing Example

Delivery of online entertainment. Distribution of television shows, movies and other videosare moving to the Web

the cloud computing technologies would enable anetwork of service providers to host the different media.Using cloud computing technology, the broadcasters can

join forces to reach a service cooperation contract thatenables them to tap into advanced services includingcontent distribution, load balancing, and overlaynetworking across different platforms in differentcountries.

If there is large demand for a show hosted by a particularsite, it can dynamically 'hire' additional servers andservices from other sites that are not being used.

IBM India


38/41


Blue Cloud

Series of cloud computing offerings

Allow corporate data centers to operate more like the

Internet

Enable computing across a distributed, globally

accessible fabric of resources, rather than on local

machines or remote server farms.

IBM India


39/41


Reference Material

A Virtualization Experience: IBM Worldwide GridImplementation, Moon Kim et al., IBM Red Books, IBM Grid 2, Edited by Ian Foster & Carl Kesselman,Elsevier,2004 Grid computing for energy exploration,

D.Beve,S.E.Zarantonello,N.Kaushik,I.Musat

IBM India


40/41


Summary

1) Storage, Security, Network and Application availabilty aremajor Grid challenges

2) Virtualization is an important technology for the Grid3) Many large Grid projects have been working successfully4) Think of Grid for variety of services, not just for computing

alone

5) Grid, virtualization and service orientation have many things incommon


41/41


IBM India

From Grid Computing to Cloud Computing The IBM ApproachGaruda Partner Meet ,4th March 2008,Bangalore,India

P. Sambath Narayanan Ph.D

[email protected]

33_onwards_3rd PM IBM KeyNote Cloud Computing Garuda-Sampath

Documents

Transcript of 33_onwards_3rd PM IBM KeyNote Cloud Computing Garuda-Sampath