NeSC Workshop Open Issues in Grid Scheduling Ali Anjomshoaa EPCC, University of Edinburgh
-
Upload
garth-robinson -
Category
Documents
-
view
24 -
download
0
description
Transcript of NeSC Workshop Open Issues in Grid Scheduling Ali Anjomshoaa EPCC, University of Edinburgh
|epcc||epcc|
NeSC Workshop
Open Issues in Grid Scheduling
Ali Anjomshoaa
EPCC, University of Edinburgh
Tuesday, 21 October 2003
Overview of a Grid Scheduling Paradigm
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
22
Grid Scheduling
Why is scheduling so important for Grids?
In order to make coherent and coordinated use of ubiquitous and heterogeneous Grids, Grid management is essential.
Grid management is not an option.
“Scheduling is all about management.”
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
33
Scheduling is about Management
Grids are complex multivariate environments, which are made up of numerous Grid entities that need to be managed.
These Grid entities fulfill various roles, which places them within the remit of different management environments.
Grid entities and their roles are interdependent on each other and their environments.
The management of these Grid entities is, for the most part, the undertaking of Grid scheduling.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
44
Have introduced the concepts of:
Grid entities Grid entity roles Grid management environments
There is a mapping:
entity roles ↔ management environments
In addition, there are interdependencies:
Between the Grid entities Between the Grid entity roles Between the Grid management environments
Scheduling is about Management
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
55
Grid Entities
So what are these Grid entities?
A Grid entity is defined by its roles.
Grid entity roles can include:
A Grid entity can have one or more roles.
Resource Account Contract User
Job Policy Workflow …
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
66
Grid Entity Roles
A Grid entity can fulfill many roles. For example:
it can have both a Job role and a Resource role, it can have both a Job role and a Workflow role, or indeed any other combination of roles.
A Grid entity role is defined by a set of role specific attributes.
A Grid entity should be managed within different management environments, depending on the various roles that it fulfills.
Could say that the interest of Grid scheduling is in the role management of the various roles of Grid entities.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
77
Grid Entity Definition
A Grid entity is defined by its roles.
A Grid entity role is defined by a set of role specific attributes.
“A Grid entity is defined by the sets of role specific attributes of its roles.”
The definition of a Grid entity could be captured in a description document as sets of role attributes.
The description document will then allow the Grid entity to be managed according to its various roles, within a Grid.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
88
Grid Entity Definition
GridEntity
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
99
Grid Entity Definition
ResourceRole
JobRole
GridEntity
…
…
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1010
Grid Entity Definition
GridEntityJob
AttributeSet
ResourceAttribute
Set
…
…
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1111
Grid Entity Definition
GridEntityJob
AttributeSet
ResourceAttribute
Set
DescriptionDocument …
…
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1212
Summary of Concepts
Previously introduced the concepts of:
Grid entities Grid entity roles Grid management environments
Have now introduced two new concepts of:
Grid entity role specific attributes Grid entity description documents
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1313
Grid Entity Roles and Management
Different roles require different management environments.
“The management environment within which a Grid entity is being managed, will depend on the particular role of that Grid entity which is being managed.”
A role type can not be managed in more than one management environment.
A management environment can manage more than one type of role.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1414
Grid Entity Roles and Management
Role interdependencies need to be managed also.
This will mean that the boundaries of management environments will need to be crossed.
Common factorisations of methods and protocols, from various management environments, need to be made, in order to allow the management of role interdependencies.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1515
Grid Management Environments
Resource Management Environment Computing Resource Management Environment Database Resource Management Environment Network Resource Management Environment
Job Management Environment
User Management Environment
Policy Management Environment
Account/Finance Management Environment
Security Management Environment
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1616
Grid Management Environments
A Grid management environment sets the rules for the management of a set of Grid entity role types.
The types of Grid entity roles managed in a particular management environment, can not be managed by any other.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1717
Grid Entity Roles and Management
GridEntity
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1818
Grid Entity Roles and Management
JobResourceResource
RoleJobRole
GridEntity
Job Management Environment
Resource Management Environment
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
1919
Grid Entity Roles and Management
GridEntity
Job Management Environment
Resource Management Environment
JobAttribute
Set
ResourceAttribute
Set
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2020
Grid Management Environments
Different management environments should allow for custom management models, methods, protocols, languages, and semantics.
However, due to the interdependencies between these management environments, they must also share a common set of languages, semantics, and ontologies in an interoperability layer to allow them to interoperate.
Without this common set of interoperability features, it would not be possible to manage role and environment interdependencies.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2121
Grid Management Environments
Account
Security
Policy
UserJob
Resource
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2222
InteroperabilityLayer
Grid Management Environments
Account
Security
Policy
UserJob
Resource
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2323
The Result?
What is the advantage of these concepts and abstractions?
Discrete roles and management environments allows them to be developed in a highly independent way.
This framework is easily extendable to include new roles and management environments.
This framework allows role definition and attribute sets to be extended or updated, without breaking the framework.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2424
Meta-Scheduling
Scheduling should allow the management of roles within their respective management environments.
Meta-scheduling should handle the interdependencies between roles and their environments, both within Grid entities, and between Grid entities.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2525
Summary
Have introduced the concepts of:
Grid entities Grid entity roles
Grid entity role specific attributes Grid entity description documents Grid management environments
Roles are managed within their respective management environments.
Role interdependencies must be managed.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2626
Conclusions
Managing discrete Grid entity roles in respective management environments allows for the development of highly independent pieces of the Grid management puzzle.
This framework enables manageable extensions and upgrades of the various pieces of the Grid management puzzle.
This framework gives flexibility in defining Grid entities through the attribution of an extensible set of roles.
This framework allows independent management of discrete roles, while enabling management of role interdependencies through the interoperability layer.
This framework should allow the consolidation of disparate scheduling efforts in various collaborations.
|epcc|
Tuesday, 21 October 2003Tuesday, 21 October 2003 NeSC Workshop - Open Issues in Grid SchedulingNeSC Workshop - Open Issues in Grid Scheduling
2727
The Holy Grail!
Interoperable Grid architectures based on heterogeneous infrastructure layers, managing heterogeneous computing environments.
Need common interoperability layers: protocols, languages, semantics, ontologies