Superscheduling and Resource Brokering Sven Groot (0024821)
-
Upload
august-marshall -
Category
Documents
-
view
218 -
download
0
Transcript of Superscheduling and Resource Brokering Sven Groot (0024821)
Superscheduling and Resource Brokering
Sven Groot (0024821)
Grid Information Service• Not all information available• Grid Information System
– Globus Monitoring and Discovery Service (MDS2)– Grid Monitoring Architecture (GMA)
• Common features– Organise sensors– Static vs. Dynamic data– Extensible– Agreed upon schema
Stages of Grid Scheduling• Phase 1: Resource Discovery
– Authorization filtering– Application Requirement Definition– Minimal requirement filtering
Stages of Grid Scheduling (2)• Phase 2: System Selection
– Dynamic information gathering– System Selection
Stages of Grid Scheduling (3)• Phase 3: Job Execution
– Advance Reservation (optional)– Job Submission– Preparation Tasks– Monitoring Progress– Job Completion– Cleanup Tasks
Application requirements
Application Requirements (2)• General requirements
– Compute-related requirements– Data-related requirements– Network-related requirements
Application Requirements (3)• Challenges
– Application deployment– Metacomputing– Predicting performance
• Theoretical prediction• History based prediction• Testcase-based prediction
– Adaptive brokering
Application Requirements (4)• Related issues
– Application frameworks– Virtual Organizations– Security requirements– Accounting policies– User preferences
Scheduling in GrADS• Scheduling phases
– Launch-time scheduling– Rescheduling– Meta-scheduling
GrADS
GrADS (2)• Focus applications
– ScaLAPACK– Cactus– FASTA– Iterative applications
• Jacobi method• Game of Life• Fish
GrADS: Launch-time scheduling
GrADS: Launch-time scheduling (2)
• Configurable Object Program – Application requirements definition
• AART• ClassAds• Redline
ClassAds sample
GrADS: Launch-time scheduling (3)
• Performance model– General method
• develop an analytic model for well-understood aspects of applicatio or system performance
• test the analytic model against achieved application performance
• develop empirical models for poorly-understood aspects of application or system behavior
– Some application specific methods– Implemented as shared libraries
GrADS: Launch-time scheduling (4)
• Mapper– Maps data and/or tasks to resources– Different mapping methods
• Equal allocation• Time balancing• Data locality
GrADS: Launch-time scheduling (5)
• Search procedure– General steps
• identify a large number of sets of resources that may be good platforms for the application
• use the application-specific mapper and performance model to generate a data map and predicted execution time for those resource sets
• select the resource set that results in the lowest predicted execution time
GrADS: Launch-time scheduling (6)
• Resource-aware search
GrADS: Launch-time scheduling (6)
• Simulated Annealing
GrADS: Rescheduling• Additional complexities
– Lack of built-in mechanisms– Need to distinguish processors that are
running/not running the current process– Overheads can be high
GrADS: Rescheduling (2)
GrADS: Rescheduling (2)• Rescheduling methods
– Application migration– Process swapping
GrADS: Metascheduling
Grid Service Level Agreements
• Contract– Provide some capability– Perform some task
• Types of SLAs– Resource Service Level Agreements– Task Service Level Agreements– Binding Service Level Agreements
Grid SLAs (2)
Grid SLAs (3)• Motivating scenarios
– Community Scheduler Scenario
Grid SLAs (4)• Motivating scenarios (cont’d)
– File transfer scenario
Grid SLAs• Resource virtualization
Multicriteria• Basic definitions
– Pareto Dominance– Pareto Optimality– Pareto-optimal set– Pareto Front
Multicriteria (2)• Motivations
– Various stakeholders and their preferences– Job scheduling– Application-Level scheduling– Hard constraints and soft constraints
Multicriteria (3)• Approach
– Criteria• Related to stakeholders• Related to entire system• Time criteria• Cost criteria• Resource utilization criteria
– Modeling preferences
Multicriteria (4)• Selection method
– Rule-based system requirements• Expression of policies• Execution of different scheduling procedures• Adaptation to the environment• Selection of the best solution
– Multicriteria optimization
Example (cont’d)
Example (cont’d)• Aggregate criteria
– End user satisfaction
– Resource Owner Satisfaction
– VO overall performance
Example (cont’d)