Resource Management of Grid Computing

-- Juan Chen 陈娟 Ying Tao 陶莺

Overview

Grid Computing Resource Management

Local Resource Management Global Resource Management Scheduling

Instance --Nimrod/G

Grid Computing (1)

A Grid is a very large-scale, generalized distributed Network Computing system that can scale to Internet size environments with machines distributed across multiple organizations and administrative domains.

There are four main aspects characterize a grid Multiple Administrative Domains and Autonomy Heterogeneity Scalability Dynamicity or Adaptability

Grid Computing (2) Grid computing is concerned with coordinated resource

sharing and problem solving in dynamic, multi-institutional virtual organizations.

The key concept is the ability to negotiate resource-sharing arrangements among a set of participating parties and then to use the resulting resource pool for some purpose. So we can see that resource management system is the central component of grid computing systems.

Designing a Grid architecture is challenging due to:

supporting adaptability, extensibility, and scalability;

allowing systems with different administrative policies to inter-operate while preserving site autonomy;

co-allocating resources; supporting quality of service; economy of computations

A layered Grid architecture and components

Globus Architecture

Basic library and supported softwares

GIISs

GRAM, GRIS, GSS

GIS, GAA

Low-level (Fabric)

Globus

Applications

local scheduler, PBS, Condor, SQMS

mpich, PSEs

collective

resource

connectivity

interface

Traditional Resource Management

Designed and operated under the assumption that: – They have complete control over a resource – They can implement the mechanisms and policies

needed for effective use of that resource in isolation

This is not the case for Grid Resource management – Separate administrative domains – Resource Heterogeneity – Lack of control of different policies

What is Grid Resource Management?

Identifying application requirements, resource specification

Matching resources to applications Allocating/scheduling and monitoring tho

se resources and applications over time in order to run as effectively as possible.

Resource Management System

RMS system abstract structure

Grid Resource Management System consists of :

Local resource management system (Resource Layer) Basic resource management unit Provide a standard interface for using remote resources e.g. GRAM, etc.

Global resource management system (Collective Layer)

Coordinate all Local resource management system within multiple or distributed Virtual Organizations (VOs)

Provide high-level functionalities to efficiently use all of resources Job Submission Resource Discovery and Selection Scheduling Co-allocation Job Monitoring, etc.

e.g. Meta-scheduler, Resource Broker, etc.

Local Resource Management Globus Resource Allocation Manager (GRAM) is r

esponsible for 1. processing RSL specifications representing resource

requests, by either denying the request or by creating one or more processes (a \job") that satisfy that request;

2. enabling remote monitoring and management of jobs created in response to a resource request

3. periodically updating the MDS information service with information about the current availability and capabilities of the resources that it manages.

Major components of the GRAM implementation

Resource co-allocator it is often the case that a metacomputing

application requires that several resources be allocated simultaneously. In these cases, a resource broker produces a multirequest and co-allocation is required.

the role of a co-allocator is to split a request into its constituent components, submit each component to the appropriate resource manager, and then provide a means for manipulating the resulting set of resources as a whole

Types of co-allocation

a range of different co-allocation service can be constructed. require all resources to be available before the

job is allowed to proceed, and fail globally if failure occurs at any resource;

allocate at least N out of M requested resources then return

return immediately, but gradually return more resources as they become available.

Scheduling Scheduling is the matching of application requirements and available resources.

System-level schedulers---focus on throughput and generally do not consider application requirements in scheduling decisions.

Application-specific schedulers---have been very successful for individual applications, but are not easily applied to new applications.

Grid Application Development Software Project (GrADS) Scheduling

Launch-time scheduling is the pre-execution determination of an initial matching of application requirements and available resources.

Rescheduling involves making modifications to that initial matching in response to dynamic system or application changes.

Meta-scheduling involves the coordination of schedules for multiple applications running on the same Grid at once.

Grid Application Development Software Architecture

Launch-time scheduling

The launch-time scheduler is called just before application launch to determine how the current application execution should be mapped to available Grid resources.

The resulting schedule specifies the list of target machines, the mapping of virtual application processes to those machines, and the mapping of application data to processes.

GrADS launch-time scheduling architeture

The drawbacks of launch-time scheduling

When two applications are submitted to GrADS at the same time, scheduling decisions will be made for each application ignoring the presence of the other.

If the launch-time scheduler determines there are not enough resources for the application, it can not make further progress.

A long running job in the system can severely impact the performance of numerous new jobs entering the system

The root cause of these and other problems is the absence of a metascheduler

Rescheduling Rescheduling, can include changing the machines

on which the application is executing or changing the mapping of data and/or processes to those machines according to the change of load or application requirements.

Rescheduling can be implemented via two ways: Application Migration Process Swapping

GrADS rescheduling architecture

Metascheduling The goal of metascheduling is to investigate scheduling pol

icies that take into account both the needs of the application and the overall performance of the system.

The metascheduler possesses global knowledge of all applications in the system and tries to balance the needs of the applications.

The metascheduler is implemented by the addition of four components, namely database manager, permission service, contract negotiator and rescheduler

Metascheduler and interactions

Introduction of Nimrod A large-scale parameter study of a simulation is

well suited to high-throughput computing. It involves the execution of a large number of tasks (task farms) over a range of parameters.

The Nimrod system is designed to address the complexities associated with parametric computing on clusters of distributed systems.

However, Nimrod is unsuitable as implemented in the large-scale dynamic context of computational grids, where resources are scattered across several administrative domains, each with their own user policies, employing their own queuing system, varying access cost and computational power.

Nimrod/G

Shortcomings of Nimrod are addressed by a new system called Nimrod/G

It uses the Globus middleware services for dynamic resource discovery and dispatching jobs over computational grids.

The architecture of Nimrod/G and its key components are shown as follows:

The architecture of Nimrod/G

Scheduling and Computational Economy of Nimrod/G

Nimrod/G system has integrated computational economy as part of a scheduling system, It can be handled in two ways: systems can work on the user’s behalf and try to

complete the assigned work within a given deadline and cost. (the early prototype of Nimrod/G)

the user can enter into a contract with the system and pose requests. The advantage of this approach is that the user knows before the experiment is started whether the system can deliver the results and what the cost will be. (rather complex and need grid middleware services for resource reservation, broker services for negotiating cost )

The important parameters of computational

economy that can influence the way resource

scheduling is done are: Resource Cost (set by its owner) Price (that the user is willing to pay) Deadline (the period by which an application execution

need to completed) The scheduler can use all sorts of information

gathered by a resource discoverer and also

negotiate with resource owners to get the best “value

for money”.

Thank you