Towards Provision of Quality of Service

Guarantees in Job Scheduling

Mohammad Islam Pavan BalajiP. Sadayappan D. K. Panda

Computer Science and EngineeringThe Ohio State University

Publicly Usable Supercomputer Centers and Compute Clusters– Becoming increasingly common (OSC, SDSC, etc)

– Allow people to run jobs and charge the user based on the job requirements

– Job requirements include Number of CPUs, CPU time, memory, etc.

– Jobs are submitted by the user with resource requirements Scheduled to execute at a later time

Dedicated job scheduler schedules the submitted jobs (Ex: PBS, Maui, Silver)

Jobs provide an estimate of the run-time together with the job

Independent Parallel Job Scheduling Model– Dynamically arriving Independent Parallel Jobs

– Resource mapping: Submitted Jobs to Resources

– Viewed as a two dimensional chart: Processors vs. Time

Job Schedulers Today

Two Dimensional Scheduling Grid

J1

J2

Time

Pro

cess

ors J3

Current Time

J5 J6J4Job Queue

Running Jobs

Job Scheduling Overview Submit jobs have several associated parameters

– Resources Required

– Arrival Time (time with the job is submitted by the user)

– Run-time estimate of the job (users need to provide a run-time estimate)

Queuing Order– FCFS (First Come First Served)

– SJF (Shortest Job First)

Reservations are made for some of the jobs– Conservative backfilling model

– EASY backfilling model

Backfilling Models

J1

J2

Time

Pro

cess

ors J3

Current Time

J5 J6J4Job Queue

Running JobsJ4

J5Reservation

Violation

J5

J6

Conservative gives reservations to all jobs; EASY gives reservations to just one job

Guarantees in Service

Number of Techniques studied over the years– Backfilling (Ex: Conservative, EASY, No Guarantee)

– Priority based scheduling Differentiated service to different classes of jobs

Soft Real-time or Best Effort guarantees to the completion time

Hard Real-Time or “Deadline-based” scheduling– Allow Users to specify the deadline they desire

– Cost model based on Resources Used AND Deadline Specified

– Requires a deadline-based scheduling algorithm

QoS for Job Scheduling Two Components in providing QoS

– Job Scheduling Component Admission Control

– Can we meet the specified deadline?

Once admitted, cannot miss the specified deadline

Studied as a part of our previous work [islam03:qops]

– Cost Model Component Based on Resources Used AND Deadline Specified

More urgent jobs are charged more

Role of user characteristics in the charging model– Tolerance of users to missed deadlines

We deal with this component of QoS in this paper

[islam03:qops] “QoPS: A QoS based scheme for Parallel Job Scheduling”, M. Islam, P. Balaji, P. Sadayappan and D. K. Panda. Published in JSSPP ’03 and LNCS ‘04.

Overview

Introduction and Motivation

Cost Model for Supercomputer Centers

Understanding User-Tolerance

Dealing with User-Tolerance

Artificial Slack

Kill-and-Restart

Experimental Results

Conclusions and Future Work

Cost Model in Supercomputer Centers Current Cost Model based on Resources used

– Cost independent of the response time of the job

– User cannot request for a quicker service

Some schedulers allow differentiated service (Ex: NERSC)– Use high priority, normal and low priority queues

– High Priority queue charges more; Low Priority queue charges less

– No guarantees provided to the user about the response time

– Idea of charging the user based on the service/priority: Still Relevant !

Two components in the Cost Model– Resource Charge

– QoS Charge

Cost Model Components

Resource Charge– Depends on the CPU, Memory, Disk Space, etc.

– We only consider the CPU resource; extendable to other resources

QoS Charge– Depends on the urgency of the job

– Depends on the difficulty in scheduling the job; based on two components Current load in the system

Slowdown of the category to which the job belongs (Ex: Short-Wide)– Slowdown depicts the factor by which the job is delayed compared to its runtime

Resource Charge = Processors x Run-TimeQoS Charge = Category Slowdown / Requested Slowdown

Overview

Introduction and Motivation

Cost Model for Supercomputer Centers

Understanding User-Tolerance

Dealing with User-Tolerance

Artificial Slack

Kill-and-Restart

Experimental Results

Conclusions and Future Work

Understanding User Tolerance

If the requested deadline cannot be met?

– Some users might submit another job or to another supercomputer center

– Some users might re-submit the job with another deadline

What new deadline? Hit-and-Trial?

– We modified QoPS to provide some feedback about the best possible deadline

We model user-tolerance based on a Tolerance Factor (TF)

– Factor of extension in the requested deadline, the user can accept

– If the (Earliest Possible Deadline) < (TF x Requested Deadline)

The user is assumed to accept the new deadline

– Else

The user is assumed to reject the new deadline

Feedback based QoPS Algorithm

This is for the case when the user requested deadline cannot be met

– Say this deadline is D1

When a new job arrives:– Try to place the new job without disturbing any other job

– Calculate the completion time of the job

– This is a definitely feasible deadline (D2)

Use a binary search for N iterations to find a better deadline– For each iteration, run QoPS with the new deadline

– If feasible, try a tighter deadline; else try a looser deadline

Impact of User ToleranceResource Charge

5

5.5

6

6.5

7

7.5

8

1 1.2 1.5 2 2.5 3 3.5 4

Tolerance Factor (TF)

Res

ourc

e C

harg

e (x

10^

8)

load 1.3 load 1.6

QoS Charge

20000

40000

60000

80000

100000

120000

140000

160000

1 1.2 1.5 2 2.5 3 3.5 4

Tolerance Factor (TF)Q

oS C

harg

e

load 1.3 load 1.6

• Increase in Resource Charge: Intuitive• Decrease in QoS Charge; as the users get more tolerant, the center loses QoS money !

• This is because of accepting cheaper jobs which would have otherwise been rejected

• Overall profit of the Supercomputer Center depends on the ratio of the two charges

Overview

Introduction and Motivation

Cost Model for Supercomputer Centers

Understanding User-Tolerance

Dealing with User-Tolerance

Artificial Slack

Kill-and-Restart

Experimental Results

Conclusions and Future Work

Dealing with User-Tolerance

User-Tolerance is not an always win situation!– Depends on the QoS charge to Resource charge ratio

If QoS charge were 0, user-tolerance is always beneficial

If Resource charge were 0, user-tolerance is never beneficial

– We need some way to deal with this behavior of user-tolerance Allows us to maximize profit depending on the QoS and Resource charges

We propose two approaches to counter user-tolerance– Providing Artificial Slack

– Kill and Restart Mechanism

Artificial Slack

If a requested deadline cannot be met

– Provide the best possible deadline to the job

Maximizes the revenue for the job

Creates a very tight schedule; difficulty in accepting later arriving jobs

– Provide the job with an artificial slack

Loses out on the revenue for the job

The overall schedule is loose enough to accept later arriving jobs

– Offered Deadline extended

Arrival Time + (Earliest Deadline – Arrival Time) x Slack Factor

Kill and Restart

Some supercomputer centers support a kill-and-restart mechanism

A running job can be killed and restarted: No permanent modifications

Some centers such as the OSC support such mechanisms

Our approach utilizing the kill-and-restart mechanism

– If a jobs requested deadline cannot be met

Kill a running job and try scheduling this job

Verify that both the new job and the killed job can be scheduled

If they can be scheduled, accept the schedule

– It is to be noted that this approach might result in wastage of resources

Overview

Introduction and Motivation

Cost Model for Supercomputer Centers

Understanding User-Tolerance

Dealing with User-Tolerance

Artificial Slack

Kill-and-Restart

Experimental Results

Conclusions and Future Work

Impact of Slack Factor (SF)Resource Charge (Load = 1.3)

5

5.2

5.4

5.6

5.8

6

6.2

1 1.5 2 2.5 3 4

Slack Facfor (SF)

Res

ourc

e C

harg

e (x

10^

8)

TF = 1.0 TF = 2.0TF = 3.0 TF = 4.0

QoS Charge (Load = 1.3)

37000

38000

39000

40000

41000

42000

43000

44000

45000

1 1.5 2 2.5 3 4

Slack Factor (SF)Q

oS C

harg

e

TF = 1 TF = 2 TF = 3 TF = 4

• Counter Impact as User Tolerance !• Drop of Resource charge; more smaller jobs accepted, force larger jobs to be

rejected• Increase in QoS charge; additional slack forces significantly delayed jobs to be

rejected

Impact of Kill-and-Restart (TF = 4.0)Resource Charge

5

5.5

6

6.5

7

7.5

8

1 1.5 2 2.5 3 4

Slack Factor (SF)

Res

ourc

e C

harg

e (x

10^

8)

NoKill KillRestart

QoS Charge

60000

70000

80000

90000

100000

110000

120000

130000

140000

150000

1 1.5 2 2.5 3 4

Slack Factor (SF)Q

oS C

harg

e

NoKill KillRestart

• Kill-and-Restart loses out on resource charge due to wastage of resources

• Gains on the QoS charge due to its ability to accept later arriving expensive jobs

Impact of Non-deadline Jobs (20% deadline)Resource Charge

7.3

7.4

7.5

7.6

7.7

7.8

7.9

8

1 1.5 2 2.5 3 4

Slack Factor (SF)

Res

ourc

e C

harg

e (x

10^

8)

NoKill KillRestart

QoS Charge

15000

17000

19000

21000

23000

25000

27000

29000

31000

33000

35000

1 1.5 2 2.5 3 4

Slack Factor (SF)Q

oS C

harg

e

NoKill KillRestart

• Benefits of Kill-and-Restart are even higher

• Benefits seen in both cost components

Overview

Introduction and Motivation

Cost Model for Supercomputer Centers

Understanding User-Tolerance

Dealing with User-Tolerance

Artificial Slack

Kill-and-Restart

Experimental Results

Conclusions and Future Work

Concluding Remarks

Considerable Research on the topic of Parallel Job Scheduling

The issue of provision of QoS has received little attention

In this paper, we extend our previous QoS based scheme, QoPS

– A feedback mechanism to provide the best possible deadline

– Study the impact of user-tolerance on overall revenues

– Propose schemes to minimize the negative impacts of user-tolerance

Demonstrated capabilities and issues based on simulation studies

Future Work

Incorporate the QoPS scheme into Maui/Moab at the OSC

Study QoS aspects for multi-site schedulers

Clusters with heterogeneous systems

Extend the scheme to multiple resources

Study QoS aspects for network flow scheduling

Thank You !

http://www.cse.ohio-state.edu/~saday

{islammo, balaji, saday, panda}@cse.ohio-state.edu

Backup Slides

The Basic QoPS Algorithm When a job (JN+1) arrives

– If J1, J2, …, JN are already present and scheduled in that order

– Place the job (JN+1) at the start of all jobs Try scheduling the jobs in that order

– If all jobs are able to meet their deadlines, Great ! Admit it !– If some job fails, we have two options:– Option1:

Consider the failed job as a critical job Push the failed job to the start of the schedule and retry ‘k’ number of such re-orderings of existing jobs are allowed If (number of re-orderings > k) switch to option 2

– Option2: Back off exponentially in the position at which you try placing job (JN+1) and retry

J3 J2J2 J1

Working of the QoPS Algorithm

J12 J11 J7J8 J6 J5J10 J9 J4 J1

J13

J13

J13

J1

J2

J3

J3J13

J13

J1

J2

J3

J1 J13 J2J3

J13

J1

J4

J3

J2

J6 J5 J4J1J13 J2J3

Max. Violations Allowed = 2

Current Violations = 0Current Violations = 1Current Violations = 2Current Violations = 0

Simulation Approach

CTC/SDSC Trace

Load Variation

Deadline Calculator

Deadline-based Trace

QoPSSimulation

MSBSimulation

MRTSimulation

EASYSimulation

Duplication/Expansion

Trace Generation Many job logs available, but no associated deadlines Synthetic Deadline Generation

– Generate a schedule for the job trace using EASY– For any job J, if the Turnaround time in this schedule is T– Deadline for J = Arrival Time + max (runtime, (1-SF) x T)– SF is the “Stringency factor” (0 < SF < 1)

0 would give the least stringent deadlines and 1 the most stringent

Some jobs might not come with deadlines– Very lax deadlines to prevent starvation– If ‘T’ is the current expected Turnaround time,

Deadline = Arrival Time + max (24hrs, R x T)

– R is the “Relaxation Factor” of the schedule