Globus
description
Transcript of Globus
![Page 1: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/1.jpg)
![Page 2: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/2.jpg)
Globus
Grid Middleware Toolkit
Otto Sievert
CSE 225
8 June, 2000
![Page 3: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/3.jpg)
Globe
and other European Grid Activities
Otto Sievert
CSE 225
8 June, 2000
![Page 4: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/4.jpg)
EGRID
• European Grid Community
• Collaborative community, not a standards group
• Commercial and Academic Interests
• www.egrid.org
![Page 5: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/5.jpg)
European Tour
• Netherlands• Germany• Poland• Italy• Sweden
![Page 6: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/6.jpg)
![Page 7: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/7.jpg)
![Page 8: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/8.jpg)
![Page 9: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/9.jpg)
![Page 10: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/10.jpg)
Grid Theme 1
• Be very (very) careful when choosing a project name.
![Page 11: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/11.jpg)
Amsterdam: Globe
• Vrije Universiteit– Maarten van Steen– Andrew Tanenbaum
• “Middleware to facilitate
large-scale distributed applications”– Web focus– object-based coherency
![Page 12: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/12.jpg)
Globe Uniqueness
• Too much data, too little resources (bandwidth, etc.)
• Caching helps
• Data Coherency integral to Cache Policy
• Release constraint of a single replication/ distribution policy for all objects– example: web pages
![Page 13: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/13.jpg)
IMAGES
COUNTS
HTML
Globe Object
• Physically Distributed• Replicated
• Distribution Policy
![Page 14: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/14.jpg)
Globe Local Object
• 4 subobjects (minimum)
• Modularity
IMAGES
SEMANTICS
REPLICATION
CONTROL
COMMUNICATION
![Page 15: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/15.jpg)
Globe Binding
1. Name server
2. Object handle
3. Location service
4. Contact points
5. Choose point
6. Repository
7. Protocol
8. Bind!
NAMINGSERVICE
LOCATIONSERVICE
IMPLEMENTATIONREPOSITORY
CLIENTPROCESS
1
2
3
4
8
6
7
DISTRIBUTEDSHAREDOBJECT
![Page 16: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/16.jpg)
Legion Binding
• Two-stage name resolution
• Binding agent• No implementation
repository
BINDINGAGENT
CLIENTOBJECT
1
2
3
4
5
SERVEROBJECT
![Page 17: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/17.jpg)
Autopilot Binding
1. Sensor registers with the sole manager
2. AP client requests sensors from the manager
3. Manager returns available sensors
4. Client and sensor communicate directly
MANAGERCLIENT
1
2
3
4 SENSOR
![Page 18: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/18.jpg)
Globe Claim: Remote Object Model Lacks Replication
• Globe– objects can be
replicated
– still maintain one state
– allows complex coherency policies
• Legion– in theory, supports
replication
– replicated state
– allows some but not all coherency policies
– in practice is not allowed
![Page 19: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/19.jpg)
NAMINGSERVICE
LOCATIONSERVICE
IMPLEMENTATIONREPOSITORY
CLIENTPROCESS
1
2
3
4
8
6
7
DISTRIBUTEDSHAREDOBJECT
Globe Architecture
• Why all these servers?– separate naming from
locating– allow extensible
binding protocols (?)
![Page 20: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/20.jpg)
Grid Theme 2: Communication
Centralized– NWS
– Globus MDS*
• Simple Management• Single Point of Failure
Distributed– NetSolve
– Fran’s Sensor Net
• Complex Management
• Scalable
Mixed– Legion
– Globe
– Autopilot
*
***
*** * *
*
*
***
*
![Page 21: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/21.jpg)
Globe Implementation Example
• Set of HTML/image/ Java files
• One semantics subobject• browsers aren’t that
extensible, so …use gateway
SEMANTICS
REPLICATION
CONTROL
COMMUNICATION
browser
gateway
http://globe.foo.edu:8989/dir/file.html
![Page 22: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/22.jpg)
Globe Example (cont’d)
• Replication Policies– Object-based
• “Permanent store”
• “Object-initiated store”
– Client-based• “Client-initiated store”
• How is this any better than what we have now?
![Page 23: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/23.jpg)
Globe Live Demo ...
![Page 24: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/24.jpg)
Globe Location Service
• Scalability Questions
• Tree Heirarchy– again Legion-like in its worst-case behavior
• Typical sol’n assumes mobile client
• Globe sol’n assumes mobile software
![Page 25: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/25.jpg)
Does This Work?
• Single experiment - 5 wk. web server trace
• compare– no caching– various complex replication/coherency policies– automatic adaptive policy
• Results– (essentially) any replication scheme wins big– individual object adaptivity didn’t perform well
![Page 26: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/26.jpg)
Globe: Conclusion
• Explicit coherency is clearly a Good Thing
• Security?
• Representative implementation?
![Page 27: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/27.jpg)
Germany: Cactus
• Albert Einstein Institute, Potsdam– Thomas Radke
– Ed Seidel
• Distributed Astrophysics• Software Engineering• NCSA “hot code”
![Page 28: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/28.jpg)
Cactus
• Separate CS from disciplinary science– Flesh = CS architecture
– Thorns = science modules
• 2-stage compilation– encapsulation
– modularity
– reuse
![Page 29: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/29.jpg)
Cactus Compilation
• Two stage– Permanently bind thorns [Perl]– Compile binary [C++/F77]
• Efficient– Don’t carry unneeded thorn info
![Page 30: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/30.jpg)
Grid Theme 3: Applications
• Numerical, or Non– Computation vs. Specialization
• Performance Measures:– Execution Time– Scale– Efficiency– Distribution
![Page 31: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/31.jpg)
Grid Theme 4: Transparency
• Ease of Use vs. High Performance– As system becomes opaque, EoU increases– As system becomes opaque, Perf decreases– Where is the balance?
![Page 32: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/32.jpg)
Germany: Gridware
• 1999 San Jose-based merger of two companies: Genias GmbH and Chord (U.S.)
• CoDINE– Compute farm load balancing system– Recently adopted by Sun™
• PaTENT– WinNT MPI
![Page 33: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/33.jpg)
Grid Theme 5: Commoditization
• Reuse is strong in the Grid– Resources (Beowulf)– Middleware (Globus, PaTENT)– Applications (Cactus)
• Industry is influential– Largest grid apps in use today are commercial– Grid-ernet is profitable
![Page 34: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/34.jpg)
To This Point ...
Resource
Middleware X X
Application X X
Commercial X
![Page 35: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/35.jpg)
Germany: Unicore
• UNIform Computer Resources (German SC access)• Goal is to provide uniform access to high performance
computers - painful to learn– OS details
– Data storage conventions
– Administration policies
• 3 phase project:I self-contained jobs
II remote data access
III concurrent remote execution
![Page 36: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/36.jpg)
Unicore (cont’d)
• How is this done?– Web (Java) user interface– X.509 authentication– Network Job Supervisor
• interprets Abstract Job Objects
• manages jobs and data
• interfaces with local batch systems (like LoadLeveler and CoDINE)
• vs. Globus?
![Page 37: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/37.jpg)
Poland: POL-34
• National Grid• Very like the system used
by Unicore, a collection of widely-distributed parallel computers
• Tree-connected ATM network
![Page 38: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/38.jpg)
POL-34
• Yellow = 2 Mb/s• Red = 34 Mb/s• Cyan = 155 Mb/s• Single administrative
domain via Virtual Users (skirting the grid issue)
• Use Load Sharing Facility (LSF)
![Page 39: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/39.jpg)
Italy: SARA
• University of Lecce, Italy Giovanni Aloisio (with Roy Williams of Caltech)
• Synthetic Aperature Radar Atlas– Distributed data-intensive app
– Alan Su and the UCSD AppLeS group is involved
![Page 40: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/40.jpg)
SARA Architecture
• The goal: easy, fast, efficient retrieval and processing of SAR image data
• Issues– data is distributed, stored
in tracks
– complex hierarchical system
• Prototypical grid app
![Page 41: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/41.jpg)
Sweden: Computational Steering
• Parallelldatorcentrum Royal Institute of Technology Per Oster
• Using Globus and the Visualization Toolkit (VTK) to steer a single CFD code.
• Little data available• Eclipsed by Autopilot
![Page 42: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/42.jpg)
Grid Theme 6: Heterogeneity
• Some attempt to hide it– Globus, CORBA, Java/Jini
• Some take advantage of it– Netsolve, Ninf
• Some characterize and manage it– AppLeS (SARA), NWS
![Page 43: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/43.jpg)
To This Point ...
Resource X
Middleware X X X X
Application X X X X
Commercial X
![Page 44: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/44.jpg)
Conclusion
• Explored Globe, Cactus, and other EuroGrid favorites in the context of– Communication architectures– Grid application characteristics– Grid transparency– Commodity computing influence– Grid heterogeneity
![Page 45: Globus](https://reader035.fdocuments.us/reader035/viewer/2022070413/56814cfc550346895dba198a/html5/thumbnails/45.jpg)
Network Weather Service (NWS)
• Rich Wolski, U. Tenn.• Monitors and Predicts
Grid Resources– network latency,
bandwidth
– CPU load, avail. Mem.
• Central NWS data server
• nws.npaci.edu/NWS