Post on 18-Dec-2015
Resource Management of Grid Computing
-- Juan Chen 陈娟 Ying Tao 陶莺
Overview
Grid Computing Resource Management
Local Resource Management Global Resource Management Scheduling
Instance --Nimrod/G
Grid Computing (1)
A Grid is a very large-scale, generalized distributed Network Computing system that can scale to Internet size environments with machines distributed across multiple organizations and administrative domains.
There are four main aspects characterize a grid Multiple Administrative Domains and Autonomy Heterogeneity Scalability Dynamicity or Adaptability
Grid Computing (2) Grid computing is concerned with coordinated resource
sharing and problem solving in dynamic, multi-institutional virtual organizations.
The key concept is the ability to negotiate resource-sharing arrangements among a set of participating parties and then to use the resulting resource pool for some purpose. So we can see that resource management system is the central component of grid computing systems.
Designing a Grid architecture is challenging due to:
supporting adaptability, extensibility, and scalability;
allowing systems with different administrative policies to inter-operate while preserving site autonomy;
co-allocating resources; supporting quality of service; economy of computations
A layered Grid architecture and components
Globus Architecture
Basic library and supported softwares
GIISs
GRAM, GRIS, GSS
GIS, GAA
Low-level (Fabric)
Globus
Applications
local scheduler, PBS, Condor, SQMS
mpich, PSEs
collective
resource
connectivity
interface
Traditional Resource Management
Designed and operated under the assumption that: – They have complete control over a resource – They can implement the mechanisms and policies
needed for effective use of that resource in isolation
This is not the case for Grid Resource management – Separate administrative domains – Resource Heterogeneity – Lack of control of different policies
What is Grid Resource Management?
Identifying application requirements, resource specification
Matching resources to applications Allocating/scheduling and monitoring tho
se resources and applications over time in order to run as effectively as possible.
Resource Management System
RMS system abstract structure
Grid Resource Management System consists of :
Local resource management system (Resource Layer) Basic resource management unit Provide a standard interface for using remote resources e.g. GRAM, etc.
Global resource management system (Collective Layer)
Coordinate all Local resource management system within multiple or distributed Virtual Organizations (VOs)
Provide high-level functionalities to efficiently use all of resources Job Submission Resource Discovery and Selection Scheduling Co-allocation Job Monitoring, etc.
e.g. Meta-scheduler, Resource Broker, etc.
Local Resource Management Globus Resource Allocation Manager (GRAM) is r
esponsible for 1. processing RSL specifications representing resource
requests, by either denying the request or by creating one or more processes (a \job") that satisfy that request;
2. enabling remote monitoring and management of jobs created in response to a resource request
3. periodically updating the MDS information service with information about the current availability and capabilities of the resources that it manages.
Major components of the GRAM implementation
Resource co-allocator it is often the case that a metacomputing
application requires that several resources be allocated simultaneously. In these cases, a resource broker produces a multirequest and co-allocation is required.
the role of a co-allocator is to split a request into its constituent components, submit each component to the appropriate resource manager, and then provide a means for manipulating the resulting set of resources as a whole
Types of co-allocation
a range of different co-allocation service can be constructed. require all resources to be available before the
job is allowed to proceed, and fail globally if failure occurs at any resource;
allocate at least N out of M requested resources then return
return immediately, but gradually return more resources as they become available.
Scheduling Scheduling is the matching of application requirements and available resources.
System-level schedulers---focus on throughput and generally do not consider application requirements in scheduling decisions.
Application-specific schedulers---have been very successful for individual applications, but are not easily applied to new applications.
Grid Application Development Software Project (GrADS) Scheduling
Launch-time scheduling is the pre-execution determination of an initial matching of application requirements and available resources.
Rescheduling involves making modifications to that initial matching in response to dynamic system or application changes.
Meta-scheduling involves the coordination of schedules for multiple applications running on the same Grid at once.
Grid Application Development Software Architecture
Launch-time scheduling
The launch-time scheduler is called just before application launch to determine how the current application execution should be mapped to available Grid resources.
The resulting schedule specifies the list of target machines, the mapping of virtual application processes to those machines, and the mapping of application data to processes.
GrADS launch-time scheduling architeture
The drawbacks of launch-time scheduling
When two applications are submitted to GrADS at the same time, scheduling decisions will be made for each application ignoring the presence of the other.
If the launch-time scheduler determines there are not enough resources for the application, it can not make further progress.
A long running job in the system can severely impact the performance of numerous new jobs entering the system
The root cause of these and other problems is the absence of a metascheduler
Rescheduling Rescheduling, can include changing the machines
on which the application is executing or changing the mapping of data and/or processes to those machines according to the change of load or application requirements.
Rescheduling can be implemented via two ways: Application Migration Process Swapping
GrADS rescheduling architecture
Metascheduling The goal of metascheduling is to investigate scheduling pol
icies that take into account both the needs of the application and the overall performance of the system.
The metascheduler possesses global knowledge of all applications in the system and tries to balance the needs of the applications.
The metascheduler is implemented by the addition of four components, namely database manager, permission service, contract negotiator and rescheduler
Metascheduler and interactions
Introduction of Nimrod A large-scale parameter study of a simulation is
well suited to high-throughput computing. It involves the execution of a large number of tasks (task farms) over a range of parameters.
The Nimrod system is designed to address the complexities associated with parametric computing on clusters of distributed systems.
However, Nimrod is unsuitable as implemented in the large-scale dynamic context of computational grids, where resources are scattered across several administrative domains, each with their own user policies, employing their own queuing system, varying access cost and computational power.
Nimrod/G
Shortcomings of Nimrod are addressed by a new system called Nimrod/G
It uses the Globus middleware services for dynamic resource discovery and dispatching jobs over computational grids.
The architecture of Nimrod/G and its key components are shown as follows:
The architecture of Nimrod/G
Scheduling and Computational Economy of Nimrod/G
Nimrod/G system has integrated computational economy as part of a scheduling system, It can be handled in two ways: systems can work on the user’s behalf and try to
complete the assigned work within a given deadline and cost. (the early prototype of Nimrod/G)
the user can enter into a contract with the system and pose requests. The advantage of this approach is that the user knows before the experiment is started whether the system can deliver the results and what the cost will be. (rather complex and need grid middleware services for resource reservation, broker services for negotiating cost )
The important parameters of computational
economy that can influence the way resource
scheduling is done are: Resource Cost (set by its owner) Price (that the user is willing to pay) Deadline (the period by which an application execution
need to completed) The scheduler can use all sorts of information
gathered by a resource discoverer and also
negotiate with resource owners to get the best “value
for money”.
Thank you