PDC Enabling Science Globus – status & perspective Olle Mulmo.
-
date post
18-Dec-2015 -
Category
Documents
-
view
216 -
download
3
Transcript of PDC Enabling Science Globus – status & perspective Olle Mulmo.
![Page 2: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/2.jpg)
PDCEnabling Science
What this talk will cover
• Background: Why we are Where we are today• Where we are today• Where we will go tomorrow
[Insert proper acknowledgement to everyoneI have stolen slides from here]
![Page 3: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/3.jpg)
PDCEnabling Science
Metacomputing...
• A concept conceived of in the (late?) ’80s• 1990’s: Dramatic increase in network bandwitdh
• Opens up for entire new problem areas• Solving ”larger-than-a-supercomputer” problems• Remote visualization• Remote instrumentation• Collaboration (VR)
• 1995:• I-WAY, proof-of-concept demo/project
• 1996:• The Globus Project sees the day
![Page 4: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/4.jpg)
PDCEnabling Science
The Globus approach
• Middleware approach• A set of orthogonal APIs
that applications can use• Command-line tools that
use the APIs (for scripting)• IP hourglass model
• Portability requirements• No perl on the Cray at the time• Poor C++ support• -> Implementation using bourne shell
(sh) and ANSI C only
Diverse global services
Coreservices
Local OS
A p p l i c a t i o n s
![Page 5: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/5.jpg)
PDCEnabling Science
Hot technologies half a decade ago
• Network communication• Raw ATM vs TCP/IP?• Use Nexus to avoid technology lock-in• Already ported to several UNIX flavors• Automatic data conversion• Optimization for local communication (AAL5,TCP,MPI,SHMEM)
• Information Discovery• Queryable, Extensible (”give me a computer with 128M RAM”)• Use LDAP (Lightweight Directory Access Protocol)• Add TTL on all published data• Add heart beats to know the test bed status
![Page 6: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/6.jpg)
PDCEnabling Science
Hot technologies half a decade ago (cont.)
• Security• Hot potato: Local control a must• -> mapfiles (what users running on what accounts)• PKI evolving, big push from SSL• -> let’s use it
• Resource Allocation• No standard: myriads of different local schedulers• -> RSL (Resource Specification Language) + wrappers
![Page 7: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/7.jpg)
PDCEnabling Science
First round: the GUSTO testbed
• 1997: The Globus Metacomputing Toolkit• MDS (Metacomputing Directory Service)• GRAM (Globus Resource Allocation Manager)• DUROC (multi-site coordinated startup)
• No security (plain-text passwords)• No file transfers
![Page 8: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/8.jpg)
PDCEnabling Science
The GUSTO testbed (cont.)
• 1998: Globus release 1.0• Security added: GSI (Globus Security Infrastructure)
• PKI (or Kerberos) below GSS-API• SSL handshakes• Proxy certificate delegation• No encryption due to export laws
• File transfer added:• GASS (fetch files over https:// or file://)
![Page 9: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/9.jpg)
PDCEnabling Science
The GUSTO testbed (cont.)
• ... not really that heavily used, though...
![Page 10: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/10.jpg)
PDCEnabling Science
Lessons learned from Globus v 1.x
• Severe performance problems with MDS• A centralized LDAP server didn’t scale very well• -> MDS-2 framework (GRIS and GIIS)
• Complaints on configuration and build times• You got everything and all the ”flavors”• debug,nodebug, nothreads,pthreads,native threads, with MPI,
without MPI, ...• -> Packaging technology
• One of the multi-site jobs often got stuck in a queue / poor network performance• -> GARA (Advanced reservation)
![Page 11: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/11.jpg)
PDCEnabling Science
Lessons learned from Globus v 1.x
• Users don’t want to hack their favorite applications• Easy access
• They have NT on their desktops• Mass storage accessibility is important • -> Data and replica management work begins
![Page 12: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/12.jpg)
PDCEnabling Science
1999: The year of the Grid
• The Concept of a Grid evolves• Resource sharing & coordinated problem solving in dynamic,
multi-institutional virtual organizations
![Page 13: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/13.jpg)
PDCEnabling Science
1999-now: The Grid boom...
• The Grid book covers a lot of groundin the community
• Lots of grid projects spring to life• Increase in funding• Commercial interests (Microsoft,Sun,IBM,...)
• Grid Forum is founded• ”The place to standardize Grid-related stuff”• Many of the protocols and approaches tried
out by Globus are adopted as a first shot
![Page 14: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/14.jpg)
PDCEnabling Science
Some renaming is required...
• GSI -> Grid Security Infrastructure• GRAM -> Grid Resource Allocation Management• MDS -> Monitoring and Discovery Service
• Never say ”Metacomputing” again...
![Page 15: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/15.jpg)
PDCEnabling Science
Back to the drawing board: Globus v 2.0
• New packaging technology (not RPM)• Precompiled binaries• GridFTP• MDS-2: decentralized LDAP framework• Security on all communications channels
• Repackaged with add-ons by several DataGrid projects• GriPhyN, EDG, NMI, ...
![Page 16: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/16.jpg)
PDCEnabling Science
Where we are today
• The Globus Toolkit™ centers around four key protocols• Connectivity layer:
• Security: Grid Security Infrastructure (GSI)
• Resource layer:• Resource Management: Grid Resource Allocation Management
(GRAM)
• Information Services: Grid Resource Information Protocol (GRIP) and Index Information Protocol (GIIP)
• Data Transfer: Grid File Transfer Protocol (GridFTP)
![Page 17: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/17.jpg)
PDCEnabling Science
A second look
• Many different protocols and methods are used• LDAP, HTTP, FTP, ...
• Missing or lack of support for• Databases• Notification systems• Error propagation
• The security part needs more work• Dynamic, Multi-VO Authorization schemes• Audit• Proxy delegation (IETF draft not likely to become an RFC)
![Page 18: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/18.jpg)
PDCEnabling Science
Let’s go shopping: Hot technologies of today
• XML & Web services• Generic and extensible• Many tools available• Backed up by industry
• Attribute Certificates• Give a certificate holder certain privilegies (access to files,
resources,...)• NMI is pursuing this path• -> Globus ships CAS (similar concept)
![Page 19: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/19.jpg)
PDCEnabling Science
Web services means generic encapsulation
• ”Corba rewritten in XML” (and then some)• WSDL: Web Services Definition Language
• Defines a service and what methods that can be invoked• Stub compilers exist for several programming languages• -> Easy to wrap your old application
• SOAP: Simple Object Access Protocol• Invocation protocol, operating on an instance of a WSDL target
• WS-Inspection and UDDI• Conventions (naming etc.) for querying and locating WSDL
documents with certain properties
![Page 20: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/20.jpg)
PDCEnabling Science
Open Grid Services Architecture (OGSA)
• Web Services for the Grid• WSDLs for some key components
• Factory, Registry, Grid Service• Dynamic properties (lifetime, version compatibility) are added• Open-source implementation in Java, .NET & C
• Guidelines, conventions and best practices• Global naming• Lifetime managment• Notification framework• ...
![Page 21: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/21.jpg)
PDCEnabling Science
OGSA
• Still a draft (continously revised)• Alpha release of Core implementation (Java)
• The Factory and generic Grid Service part
• Base implementation next• Current Globus functionality refactored
as Grid Services• Alpha at the end of this summer
• Globus 3.0 (Q1 ’03?)• Old APIs will still work (or so they say)• ...but the wire protocols are new GT3 Core
GT3 Base Services
Other GridServicesGT3
DataServices
![Page 22: PDC Enabling Science Globus – status & perspective Olle Mulmo.](https://reader030.fdocuments.us/reader030/viewer/2022032703/56649d235503460f949f9230/html5/thumbnails/22.jpg)
PDCEnabling Science
Next steps: your guess is as good as mine...
• Experiences learned with v2 and v3• Access to tertiary storage and databases• How much can be gotten ”for free” from other WS
initiatives?• Other forms of security infrastructures?
• Passport / Liberty Alliance technologies• ”MyProxy” effort
• Continued work on authorization & delegation