Prabal Dutta , Stephen Dawson-Haggerty, Yin Chen, Chieh -Jan (Mike) Liang, and Andreas Terzis
1 Monitoring Grid Services Yin Chen [email protected] June 2003.
-
Upload
karen-walters -
Category
Documents
-
view
217 -
download
0
Transcript of 1 Monitoring Grid Services Yin Chen [email protected] June 2003.
![Page 2: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/2.jpg)
2
Contents
Issues of MonitoringProject Proposal
![Page 3: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/3.jpg)
3
Issues of Monitoring
What the goals of Grid monitoringWhat's the characteristics of Grid
systemWhat may need to be MonitoredWhat’s the characteristics of Monitoring
DataRelated Work
![Page 4: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/4.jpg)
4
What the goals of Grid monitoring
The question is
Propagate errors to users/management
Performance monitoring to tune the application use the Grid more efficiently
Not how to measure resources But how to deliver information to end-users and system/Grid
![Page 5: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/5.jpg)
5
What's the characteristics of Grid system
Complex distributed system =>often observe unexpectedly low performance
Where is the bottleneck? - application - operating system - disks - network adapters on either the sending or the receiving host - network switches, routers
Experience of the Netlogger group - 40% network, 40% application, 20% host problems - application: 50% client, 50% server process problems
![Page 6: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/6.jpg)
6
What's the characteristics of Grid system (cont..)
Dynamic environment World-wide distributed environment with
- high latency- frequent faults- very heterogeneous resources
![Page 7: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/7.jpg)
7
What may need to be Monitored
Disk space, speed of processor, network bandwidth, CPU load, memory load, network load, network communication time, number of parallel streams, stripes TCP/IP buffer size, disk access time that includes time to copy data to or from the local hard disk on the server.[2][3]
Some of this information are relative static information while others are run-time dynamic information.
![Page 8: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/8.jpg)
8
What’s the characteristics of Monitoring Data
Run-time monitoring data goes "Old" quickly
Producer should near the entities. Rapidly and efficiently transport from producer to consumer. Information should be explicate, e.g. by timestamps
Updates are frequent
Performance information is often stochastic
![Page 9: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/9.jpg)
9
Related Work
Monitoring and Discovery Service (MDS) Grid Monitoring Architecture (GMA) Relational Grid Monitoring Architecture
(R-GMA) HawkeyeGlobus Heartbeat Monitor (HBM) Network Weather Service (NWS) GridRM
![Page 10: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/10.jpg)
10
MDS Architecture
![Page 11: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/11.jpg)
11
GMA Architecture
![Page 12: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/12.jpg)
12
R-GMA Architecture
![Page 13: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/13.jpg)
13
Hawkeye Architecture
![Page 14: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/14.jpg)
14
HBM Architecture
![Page 15: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/15.jpg)
15
NWS Architecture
![Page 16: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/16.jpg)
16
The Global Layer of GridRM
![Page 17: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/17.jpg)
17
The Local GridRM Layer
![Page 18: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/18.jpg)
18
Summary and Conclusion
Varieties of different systems exist for monitoring
Each system has its own strengths and weaknesses
Tend to use standard and open components
GGF advocated architecture GMA
![Page 19: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/19.jpg)
19
Summary and Conclusion (cont.)
The similarities in architecture At the lowest level, have a sensor or other program that generates a piece of data. Some systems allow data to be aggregated from a set of resources At the resource level, gather together the data from several information collectors into one component Directory component Decentralised hierarchy structure, which have higher ability in fault tolerance Differences in using push or pull mechanism
![Page 20: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/20.jpg)
20
Project Proposal
GoalRequirementArchitecture -- Pull ModelSpecificationImplementationTestingSchedule
![Page 21: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/21.jpg)
21
Goal
RealisationLightweight & Simple designReliability & Robustness
![Page 22: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/22.jpg)
22
Architecture
What is Pull model The monitor sends requests to the service for information. This implies repeated
queries of resource attributes over some time period at a specific frequency
On the other hand in a Push model the service sends out notifications to a subscribed sink.
![Page 23: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/23.jpg)
23
Benefits of Pull
Less network traffic: collections initiated only from top Has no time synchronisation problem: collect data
from resources at the same time. The server can determine the size of the file, select
the appropriate alternate server, and passively control the bandwidth and storage space.
According to Globus, "push" model "generates a large amount of data and results in constant updates to the MDS.
Standard LDAP databases are not designed to handle frequent updates.
![Page 24: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/24.jpg)
24
Benefits of Pull (Cont.)
The Pull model is based on distributed intelligence to the asset site - it becomes automated. Using machine-to-machine communications with connected sensors and autonomic computing the asset does self-diagnostics, self maintain and repair, re-routes energy flows, schedules non-routine
maintenance and reports on any out of the ordinary activity that poses a security threat. IBM calls it autonomic computing where machine to machine communications take place to optimise the performance of computing and network resources.
![Page 25: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/25.jpg)
25
Problems of Pull
must gathering current measurements from all resources.
if the data volume is large in real-time may cause bottleneck problem.
may be not useful in fault detection -- heartbeat events are valid only for a short time interval and should be delivered in this time constraint.
may be not useful in dynamic sensor management. The push model is the most efficient in terms of
bandwidth as requests are not sent, just responses from the service.
![Page 26: 1 Monitoring Grid Services Yin Chen s0231189@sms.ed.ac.uk June 2003.](https://reader035.fdocuments.us/reader035/viewer/2022062407/56649dde5503460f94ad651a/html5/thumbnails/26.jpg)
26
Monitoring Grid Services
Thanks