Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging...
-
date post
19-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging...
![Page 1: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/1.jpg)
Ian FosterArgonne National Laboratory
University of Chicago
Univa Corporation
Globus®
Bridging the Gap Steve Tuecke
CEO, Univa Corporation
![Page 2: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/2.jpg)
2
Overview
Why Globus New partners What’s next The larger context The Globus community
![Page 3: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/3.jpg)
3
Overview
Why Globus Motivation Uses
New partners What’s next The larger context The Globus community
![Page 4: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/4.jpg)
4
The Application-Infrastructure Gap
Dynamicand/or
DistributedApplications
A
1
B
1
99
Shared Distributed Infrastructure
![Page 5: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/5.jpg)
5
Bridging the Gap:Service-Oriented Infrastructure
Service-oriented applications Wrap applications as
services Compose applications
into workflows Service-oriented
infrastructure Provision physical
resources to support application workloads
ApplnService
ApplnService
Users
Workflows
Composition
Invocation
Provisioning
![Page 6: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/6.jpg)
6
Globus is Service-Oriented Infrastructure Technology
Software for service-oriented infrastructure Service enable new & existing resources E.g., GRAM on computer, GridFTP on storage system,
custom application service Uniform abstractions & mechanisms
Tools to build applications that exploit service-oriented infrastructure Registries, security, data management, …
Open source & open standards Each empowers the other
Enabler of a rich tool & service ecosystem
![Page 7: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/7.jpg)
7
Globus asService-Oriented Infrastructure
IBM
IBM
Uniform interfaces,security mechanisms,Web service transport,
monitoring
Computers StorageSpecialized resource
UserApplication
UserApplication
UserApplication
IBM
IBM
GRAM GridFTPHost EnvUser Svc
DAIS
Database
ToolTool Reliable
FileTransfer
MyProxy
Host EnvUser Svc
MDS-Index
![Page 8: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/8.jpg)
A Typical eScience Use of Globus:Network for Earthquake Eng. Simulation
Links instruments, data, computers, people
![Page 9: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/9.jpg)
An eBusiness Use of Globus:SAP Demonstration @ GlobusWorld
3 Globus-enabled applns: CRM: Internet Pricing Configurator (IPC) CRM: Workforce
Management (WFM) SCM: Advanced Planner
& Optimizer (APO) Applications modified to:
Adjust to varying demand & resources
Use Globus to discover & provision resources
IPCDispatcher
IPCServerRequest:
Price Query
Delegation ofRequest
Response: PricelistDepending on: - Time - Discount - Number of Items - …
Web Browsers / Batch Processes(typically several thousand requests)
IPCServer
1
2
2
3
SAP AG R/3 Internet Pricing & Configurator (IPC)
![Page 10: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/10.jpg)
10
A
1
B
1
9
A
1 2
B
1 2
1
7
Strategy A:Reduce TCO
18 Servers 11 Servers
A
1 2
B
1 2
1
14
Strategy B:Improve
Performance
9 Servers Each 16 Servers Each
9
Cheaper and/or Faster ...
![Page 11: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/11.jpg)
11
Overview
Why Globus New partners
Globus Alliance Globus Consortium Univa
What’s next The larger context The Globus community
![Page 12: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/12.jpg)
12
How Globus Works
Globus is a distributed open source community with many contributors & users CVS, documentation, bugzilla, email lists Modular structure allows many to contribute
Globus Alliance Board provides governance when needed Meritocracy: individuals who demonstrate ongoing
contributions & commitment Primarily: what to include, when to release
Globus Alliance is an informal partnership of organizations led by Board members
![Page 13: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/13.jpg)
13
Evolution of the Globus Alliance
Argonne/U.Chicago (Childers, Foster): 1995 USC/ISI (Kesselman): 1995 Edinburgh (Atkinson, Parsons): 2003 Swedish PDC (Johnsson, Mulmo): 2003 NCSA (Welch): 2004 Univa (Czajkowski, Tuecke): 2004
Other contributors will surely be added
![Page 14: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/14.jpg)
14
From eScience to eBusiness
Since ~2001, growing interest in Globus for commercial use Enterprises, IT vendors, ISVs asking Globus
leaders to address commercial needs But hard to do in a research laboratory
In response, we have created two new organizations Globus Consortium Univa
![Page 15: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/15.jpg)
15
Globus Consortium(www.globusconsortium.com)
Nonprofit organization funded by companies to advance Globus Toolkit for enterprise use Initial sponsor members: HP, IBM, Intel, Sun Initial contributors: Nortel, Univa
First two projects already identified Member-driven software quality improvements Contributions to job submission standards
Other projects to be defined, e.g. Develop new features key to enterprise use Education & outreach
![Page 16: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/16.jpg)
16
Provider of commercial support, services, & products around open source Globus Commercial distribution of GT4 & beyond Integration with enterprise systems Committed to open source & open standards
Founded by Tuecke, Foster, Kesselman Tuecke left Argonne to be CEO Foster, Kesselman remain at Argonne, ISI
Experienced management team Rich Miller, Vas Vasiliadis, Paul Davé,
Bob Mandel
![Page 17: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/17.jpg)
17
Overview
Why Globus New partners What’s next
Globus Toolkit v4 The larger context The Globus community
![Page 18: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/18.jpg)
18
Globus asService-Oriented Infrastructure
IBM
IBM
GRAM GridFTPHost EnvUser Svc
ReliableFile
Transfer
MyProxy
Uniform interfaces,security mechanisms,Web service transport,
monitoring
Host EnvUser Svc
Computers StorageSpecialized resource
UserApplication
UserApplication
UserApplication
IBM
IBM
DAIS
Database
MDS-Index
ToolTool
![Page 19: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/19.jpg)
Data Management
SecurityCommonRuntime
Execution Management
Information Services
Web Services
Components
Non-WS
Components
Pre-WSAuthenticationAuthorization
GridFTP
GridResource
Allocation Mgmt(Pre-WS GRAM)
Monitoring& Discovery
System(MDS2)
C CommonLibraries
GT2
WSAuthenticationAuthorization
ReliableFile
Transfer
OGSA-DAI[Tech Preview]
GridResource
Allocation Mgmt(WS GRAM)
Monitoring& Discovery
System(MDS4)
Java WS Core
CommunityAuthorization
ServiceGT3
ReplicaLocationService
XIO
GT3
CredentialManagement
GT4
Python WS Core[contribution]
C WS Core
CommunitySchedulerFramework
[contribution]
DelegationService
GT4
Globus Open Source Grid Software
![Page 20: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/20.jpg)
21
Java Services in Apache AxisPlus GT Libraries and Handlers
YourJava
Service
YourPythonService
YourJava
Service RF
T
GR
AM
Del
egat
ion
Inde
x
Trig
ger
Arc
hive
r
pyGlobusWS Core
YourC
Service
C WS Core
RLS
Pre
-WS
MD
S
CA
S
Pre
-WS
GR
AM
Sim
pleC
A
MyP
roxy
OG
SA
-DA
I
GT
CP
Grid
FT
P
C Services using GT Libraries and Handlers
SERVER
CLIENT
InteroperableWS-I-compliant
SOAP messaging
YourJavaClient
YourC
Client
YourPythonClient
YourJavaClient
YourC
Client
YourPythonClient
YourJavaClient
YourC
Client
YourPythonClient
YourJavaClient
YourC
Client
YourPythonClient
X.509 credentials =common authentication
Python hosting, GT Libraries
GT4 Components
![Page 21: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/21.jpg)
22
Our Goals for GT4
Usability, reliability, scalability, … Web service components have quality equal
or superior to pre-WS components Documentation at acceptable quality level
Consistency with latest standards (WS-*, WSRF, WS-N, etc.) and Apache platform WS-I Basic (Security) Profile compliant
New components, platforms, languages And links to larger Globus ecosystem
![Page 22: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/22.jpg)
23
GT4 Web Services Core
Supports both Globus services (GRAM, RFT, Delegation, etc.) & user-developed services
Redesign to enhance scalability, modularity, performance, usability
Leverages existing WS standards WS-I Basic Profile: WSDL, SOAP, etc. WS-Security, WS-Addressing
Adds support for emerging WS standards WS-Resource Framework, WS-Notification
Java, Python, & C hosting environments
![Page 23: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/23.jpg)
24
CustomWeb
ServicesWS-Addressing, WSRF,
WS-Notification
CustomWSRF Web
Services
GT4WSRF Web
Services
WSDL, SOAP, WS-Security
User Applications
Reg
istr
yA
dmin
istr
atio
n
GT
4 C
onta
iner
GT4 Web Services Core
![Page 24: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/24.jpg)
26
GT4 Security
VO
RightsUsers
Rights’
ComputeCenter
Access
Services (runningon user’s behalf)
Rights
Local Policyon VO identityor attributeauthority
CAS or VOMSissuing SAMLor X.509 ACs
SSL/WS-Securitywith ProxyCertificates
Authz Callout
KCA
MyProxy
![Page 25: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/25.jpg)
27
GT4 Security Extensible authorization framework based on
Web services standards SAML-based authorization callout
As specified in GGF OGSA-Authz WG
Integrated policy decision engine XACML policy language, per-operation policies, pluggable
Credential management services MyProxy (One time password support) Community Authorization Service
Standalone delegation service Implements IETF RFC
![Page 26: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/26.jpg)
28
GT4’s Use of Security Standards
![Page 27: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/27.jpg)
29
GT4 Data Management
Stage large data to/from nodes Replicate data for performance & reliability Locate data of interest Provide access to diverse data sources
File systems, parallel file systems, hierarchical storage (GridFTP)
Databases (OGSA DAI)
![Page 28: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/28.jpg)
30
GT4 Data Functions
Find your data: Replica Location Service Managing ~40M files in production settings
Move/access your data: GridFTP, RFT High-performance striped data movement
27 Gbit/s memory-to-memory on a 30 Gbit/s link (90% utilization) with 32 IBM TeraGrid nodes.
17.5 Gbit/s disk-to-disk limited by the storage system Reliable movement of 120,000 files (so far)
Couple data & execution management GRAM uses GridFTP and RFT for staging
![Page 29: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/29.jpg)
31
GridFTP in GT4
100% Globus code No licensing issues Stable, extensible
IPv6 Support XIO for different transports Striping multi-Gb/sec wide area transport Pluggable
Front-end: e.g., future WS control channel Back-end: e.g., HPSS, cluster file systems Transfer: e.g., UDP, NetBLT transport
Bandwidth Vs Striping
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
20000
0 10 20 30 40 50 60 70
Degree of Striping
Ba
nd
wid
th (
Mb
ps
)
# Stream = 1 # Stream = 2 # Stream = 4
# Stream = 8 # Stream = 16 # Stream = 32
Disk-to-disk onTeraGrid
![Page 30: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/30.jpg)
32Reliable File Transfer:Third Party Transfer
RFT Service
RFT Client
SOAP Messages
Notifications(Optional)
DataChannel
Protocol Interpreter
MasterDSI
DataChannel
SlaveDSI
IPCReceiver
IPC Link
MasterDSI
Protocol Interpreter
Data Channel
IPCReceiver
SlaveDSI
Data Channel
IPC Link
GridFTP Server GridFTP Server
Fire-and-forget transfer Web services interface Many files & directories Integrated failure recovery
![Page 31: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/31.jpg)
33
Replica Location Service
Identify location of files via logical to physical name map
Distributed indexing of names, fault tolerant update protocols
GT4 version scalable & stable
Managing ~40 million files across ~10 sites
IndexIndex
Local DB
Update send (secs)
Bloom filter
(secs)
Bloom filter (bits)
10K <1 2 1 M
1 M 2 24 10 M
5 M 7 175 50 M
![Page 32: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/32.jpg)
34
Execution Management (GRAM)
Common WS interface to schedulers Unix, Condor, LSF, PBS, SGE, …
More generally: interface for process execution management Lay down execution environment Stage data Monitor & manage lifecycle Kill it, clean up
A basis for application-driven provisioning
![Page 33: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/33.jpg)
35
GT4 GRAM
2nd-generation WS implementation optimized for performance, flexibility, stability, scalability
Streamlined critical path Use only what you need
Flexible credential management Credential cache & delegation service
GridFTP & RFT used for data operations Data staging & streaming output Eliminates redundant GASS code
![Page 34: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/34.jpg)
36
GRAMservices
GT4 Java Container
GRAMservices
Delegation
RFT FileTransfer
Transferrequest
GridFTPRemote storage element(s)
Localscheduler
Userjob
Compute element
GridFTP
sudo
GRAMadapter
FTPcontrol
Local jobcontrol
Delegate
FTP data
Cli
ent
Job
functions
Delegate
Service host(s) and compute element(s)
GT4 GRAM Structure:WSRF/WSN Poster Child
![Page 35: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/35.jpg)
37
Monitoring and Discovery “Every service should be monitorable and
discoverable using common mechanisms” WSRF/WSN provides those mechanisms
A common aggregator framework for collecting information from services, thus: MDS-Index: Xpath queries, with caching MDS-Trigger: perform action on condition MDS-Archiver: Xpath on historical data
Deep integration with Globus containers & services: every GT4 service is discoverable GRAM, RFT, GridFTP, CAS, …
![Page 36: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/36.jpg)
38
GT4 Container
GT4 Monitoring & Discovery
GRAM User
MDS-Index
GT4 Cont.
RFT
MDS-Index
GT4 Container
MDS-Index
GridFTP
adapter
Registration &WSRF/WSN Access
Custom protocolsfor non-WSRF entities
Clients(e.g., WebMDS)
Automatedregistrationin container
WS-ServiceGroup
![Page 37: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/37.jpg)
GT4 Documentation
is Much Improved!
![Page 38: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/38.jpg)
40
Working with GT4
Download and use the software, and provide feedback Join [email protected] mail list
Review, critique, add to documentation Globus Doc Project: http://gdp.globus.org
Tell us about your GT4-related tool, service, or application Email [email protected]
![Page 39: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/39.jpg)
41
Overview
Why Globus New partners What’s next The larger context
Web services, utility computing, virtualization, …
The Globus community
![Page 40: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/40.jpg)
42
Infrastructure Complexity:What’s the Answer?
Web services Utility computing Virtualization Data center automation Adaptive enterprise …
Recall:
Service-oriented applications
demand
service-oriented infrastructure
![Page 41: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/41.jpg)
43
Overview
Why Globus New partners What’s next The larger context The Globus community
Who’s using Globus? What are they using it for? The Globus ecosystem Growing the Globus community
![Page 42: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/42.jpg)
26776 U.S.
2753 China
1318 Japan
1017 India
750 U.K.
495 Italy
488 Germany
391 Brazil
328 S. Korea
306 Taiwan
268 France
241 Canada
211 Viet Nam
211 Spain
202 Russia
187 Pakistan
159 Australia
142 Singapore
131 Greece
119 Colombia
111 Poland
109 Netherlands
107 Thailand
94 Switzerland
77 Chile
74 Sweden
68 Belgium
66 Venezuela
66 Romania
64 Indonesia
62 Mexico
61 Turkey
60 Malaysia
58 Portugal
57 Austria
54 Ireland
44 Hong Kong
40 Hungary
38 Egypt
38 Argentina
34 Uruguay
31 Ukraine
29 Slovakia
25 Israel
23 Yugoslavia
23 Iran
22 Bulgaria
22 Uzbekistan
22 Czech Rep.
22 N. Korea
21 Lithuania
21 Croatia
20 Finland
20 New Zealand
17 Nigeria
17 South Africa
16 Jordan
16 Slovenia
16 Afghanistan
15 Denmark
15 Philippines
14 Vanuatu
14 Luxembourg
14 Tunisia
12 Virgin Is. (U.K.)
12 Peru
12 Yemen
11 Norway
11 Algeria
11 Armenia
10 Iceland
9 Zambia
9 Virgin Is. (U.S.)
9 Uganda
9 Bosnia & Herz.
8 Kenya
7 Zimbabwe
7 Saudi Arabia
7 Ecuador
7 Macedonia
6 Bolivia
6 Comoros
6 Zaire
6 Lebanon
5 Puerto Rico
5 Namibia
5 Togo
5 Tajikistan
5 Paraguay
5 Albania
5 Sudan
4 Estonia
4 Camaroon
4 Ghana
4 Tuvalu
4 Costa Rica
4 Cuba
4 UAE
4 Tonga
4 W. Samoa
4 Tanzania
3 Syria
3 Bahamas
3 Ethiopia
3 Mongolia
3 Sri Lanka
3 Wallis & Futuna Is.
3 Belarus
3 Bangladesh
2 Falkland Islands
2 Kuwait
2 Sierra Leone
2 Trinidad & Tobago
2 Guyana
2 American Samoa
2 Andorra
2 Georgia
2 Cook Islands
2 Turkmenistan
2 Gabon
2 The Gambia
2 Kazakhstan
2 Macau
2 Malta
2 Jamaica
2 Latvia
2 Turks & Caicos
1 Bhutan
1 Ascension Island
1 Cyprus
1 Mozambique
1 Tokelau
1 Greenland
1 Nepal
1 Swaziland
1 Iraq
1 Serbia
1 Barbados
1 Cambodia
1 Qatar
1 Saint Vincent
1 Laos
1 San Marino
1 Libya
1 Benin
1 Angola
1 Chad
1 Gibraltar
1 Haiti
1 Guatemala
1 Malawi
1 Equatorial Guinea
1 Palau
1 Bermuda
1 Botswana
1 Suriname
386
69 d
ow
nlo
ad
s in
20
04
fro
m g
lobu
s.org
![Page 43: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/43.jpg)
26776 U.S.
2753 China
1318 Japan
1017 India
750 U.K.
495 Italy
488 Germany
391 Brazil
328 S. Korea
306 Taiwan
268 France
241 Canada
211 Viet Nam
211 Spain
202 Russia
187 Pakistan
159 Australia
142 Singapore
131 Greece
119 Colombia
111 Poland
109 Netherlands
107 Thailand
94 Switzerland
77 Chile
74 Sweden
68 Belgium
66 Venezuela
66 Romania
64 Indonesia
62 Mexico
61 Turkey
60 Malaysia
58 Portugal
57 Austria
54 Ireland
44 Hong Kong
40 Hungary
38 Egypt
38 Argentina
34 Uruguay
31 Ukraine
29 Slovakia
25 Israel
23 Yugoslavia
23 Iran
22 Bulgaria
22 Uzbekistan
22 Czech Rep.
22 N. Korea
21 Lithuania
21 Croatia
20 Finland
20 New Zealand
17 Nigeria
17 South Africa
16 Jordan
16 Slovenia
16 Afghanistan
15 Denmark
15 Philippines
14 Vanuatu
14 Luxembourg
14 Tunisia
12 Virgin Is. (U.K.)
12 Peru
12 Yemen
11 Norway
11 Algeria
11 Armenia
10 Iceland
9 Zambia
9 Virgin Is. (U.S.)
9 Uganda
9 Bosnia & Herz.
8 Kenya
7 Zimbabwe
7 Saudi Arabia
7 Ecuador
7 Macedonia
6 Bolivia
6 Comoros
6 Zaire
6 Lebanon
5 Puerto Rico
5 Namibia
5 Togo
5 Tajikistan
5 Paraguay
5 Albania
5 Sudan
4 Estonia
4 Camaroon
4 Ghana
4 Tuvalu
4 Costa Rica
4 Cuba
4 UAE
4 Tonga
4 W. Samoa
4 Tanzania
3 Syria
3 Bahamas
3 Ethiopia
3 Mongolia
3 Sri Lanka
3 Wallis & Futuna Is.
3 Belarus
3 Bangladesh
2 Falkland Islands
2 Kuwait
2 Sierra Leone
2 Trinidad & Tobago
2 Guyana
2 American Samoa
2 Andorra
2 Georgia
2 Cook Islands
2 Turkmenistan
2 Gabon
2 The Gambia
2 Kazakhstan
2 Macau
2 Malta
2 Jamaica
2 Latvia
2 Turks & Caicos
1 Bhutan
1 Ascension Island
1 Cyprus
1 Mozambique
1 Tokelau
1 Greenland
1 Nepal
1 Swaziland
1 Iraq
1 Serbia
1 Barbados
1 Cambodia
1 Qatar
1 Saint Vincent
1 Laos
1 San Marino
1 Libya
1 Benin
1 Angola
1 Chad
1 Gibraltar
1 Haiti
1 Guatemala
1 Malawi
1 Equatorial Guinea
1 Palau
1 Bermuda
1 Botswana
1 Suriname
386
69 d
ow
nlo
ad
s in
20
04
fro
m g
lobu
s.org
26776 U.S. 2753 China 1318 Japan 1017 India 750 U.K. 495 Italy 488 Germany 391 Brazil 328 S. Korea 306 Taiwan
Top 10
![Page 44: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/44.jpg)
GlobalCommunity
![Page 45: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/45.jpg)
47
Grids for Physics: LHC Computing Grid
![Page 46: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/46.jpg)
48
“Paradigm Shift: Transition to Grid”
Jun: 80%:20%
25% of DC’04
Aug: 27%:73%
42% of DC’04
May: 89%:11%
11% of DC’04
Jul: 77%:23%
22% of DC’04
424 CPU · Years
Slide: Tony Doyle, University of Glasgow
![Page 47: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/47.jpg)
49Grid2003 Open Science Grid 30 sites (2100-2800 CPUs) & growing 400-1300 concurrent jobs 8 substantial applications + CS experiments Running since October 2003
Korea
http://www.ivdgl.org/grid2003
![Page 48: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/48.jpg)
50
ExampleOSG
Workflows
Genome sequence analysis
Physicsdata
analysis
Sloan digital sky
survey
![Page 49: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/49.jpg)
51
Further Application Examples
Earth System Grid:O(100TB) online data
STAR: 5 TB transfer(SRM, GridFTP)
NASA/NVO: Mosaicsfrom multiple sources
Fusion Grid:1000s of jobs
![Page 50: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/50.jpg)
系统架构 : DartGrid 的分层模型
“We’ve used GT3 to build the largest database grid system for Traditional Chinese Medicine, integrating
about 50 TCM-relevant databases.” (Zhejiang U.)
VO的地址栏
语义浏览面板
本体论树
DartGri d 虚拟组织资源浏览面板
Q3语义查询显示面板
语义注册面板
![Page 51: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/51.jpg)
53
Nimrod-G, Australia
Astrophysics Air PollutionAntenna Design
Airfoil Design
Circuit Design Monte Carlo
Computational Chemistry
Public Health Policy
CFD
Cardiac Modelling
Climate
Earth Sciences
![Page 52: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/52.jpg)
54
The Globus Ecosystem
Globus components address core issues relating to resource access, monitoring, discovery, security, data movement, etc. GT4 being the latest version
A larger Globus ecosystem of open source and proprietary components provide complementary components A growing list of components
These components can be combined to produce solutions to Grid problems We’re building a list of such solutions
![Page 53: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/53.jpg)
55
Many Tools Build on, or Can Contribute to, GT4-Based Grids
Condor-G, DAGman MPICH-G2 GRMS Nimrod-G Ninf-G Open Grid Computing Env. Commodity Grid Toolkit GriPhyN Virtual Data System Virtual Data Toolkit GridXpert Synergy Platform Globus Toolkit
VOMS PERMIS GT4IDE Sun Grid Engine PBS scheduler LSF scheduler GridBus TeraGrid CTSS NEES IBM Grid Toolbox …
![Page 54: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/54.jpg)
56Documenting
The Grid Ecosystem
The Grid Ecosystem: Software Components for Grid SystemsAnd Applications
www.grids-center.org
![Page 55: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/55.jpg)
57
Example Solutions
Portal-based User Reg. System (PURSE) VO Management Registration Service Service Monitoring Service TeraGrid TGCP Tool Lightweight Data Replicator GriPhyN Virtual Data System
![Page 56: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/56.jpg)
58
The Globus Developer Community
We want to facilitate community contributions to the Globus software Increasingly open governance & mechanisms Documentation of larger ecosystem What else can we do?
Tell us how you think we should evolve E.g., adopt Apache processes? Contribute more of Globus to Apache?
We’ve already made major contributions to Apache Axis, WS-Addressing, and WS-Security
Also WSRF/WSN Apache (Apollo, Hermes)
![Page 57: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/57.jpg)
59
Globus and its User Community
How can “we” best support “you”? We try to provide the best software we can We use bugzilla & other community tools We work to grow the set of contributors
How can “you” best support “us”? Become a contributor: of software, bug fixes,
answers to questions, documentation Provide us with success stories that can justify
continued Globus development Promote Globus within your communities
Contact me at [email protected]
![Page 58: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/58.jpg)
60
2005 and Beyond
We have a solid Web services base We now want to build, on that base, a open
source service-oriented infrastructure Virtualization New services for provisioning, data
management, security, VO management End-user tools for application development Etc., etc.
See Carl Kesselman’s talk for more details!
![Page 59: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/59.jpg)
61
Summary:Globus is Bridging the Gap
Application to infrastructure Science to commercial Web services to Grid Data to computation Silo to silo Organization to organization Country to country Developer to developer Developer to user
![Page 60: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/60.jpg)
Acknowledgments Carl Kesselman, our long-time Globus co-conspirators Gregor von Laszewski, Kate Keahey, Jennifer Schopf, Mike
Wilde, other Argonne colleagues Globus Alliance members at Argonne, U.Chicago, USC/ISI,
Edinburgh, PDC Miron Livny, U.Wisconsin Condor project, Rick Stevens,
Argonne & U.Chicago Other partners in Grid technology, application, &
infrastructure projects DOE SciDAC program, NSF Middleware Initiative, NASA,
IBM for generous support
![Page 61: Ian Foster Argonne National Laboratory University of Chicago Univa Corporation Globus ® Bridging the Gap Steve Tuecke CEO, Univa Corporation.](https://reader038.fdocuments.us/reader038/viewer/2022103005/56649d2b5503460f949fffa6/html5/thumbnails/61.jpg)
63
For More Information
Globus Alliance www.globus.org
Globus Consortium www.globusconsortium.com
Global Grid Forum www.ggf.org
Background information www.mcs.anl.gov/~foster
2nd Editionwww.mkp.com/grid2