IT 344: Operating

Systems

Winter 2008

Module 9

Scheduling

Chia-Chi Tengccteng@byu.e

du265 CTB

04/21/23 2

Review: semaphores (blocking lock)

• Each semaphore has an associated queue of threads– when wait(sem) is called by a thread,

• if sem was “available” (>0), decrement sem and let thread continue

• if sem was “unavailable” (<=0), place thread on associated queue; run some other thread

– when signal(sem) is called by a thread• if thread(s) are waiting on the associated queue, unblock one

– place it on the ready queue– might as well let the “signaling” thread continue execution

• otherwise (when no threads are waiting on the sem), increment sem

– the signal is “remembered” for next time wait(sem) is called

• Semaphores thus have history

04/21/23 3

Usage

• From the programmer’s perspective, wait and signal on a binary semaphore are just like Acquire and Release on a lock

wait(sem)...do whatever stuff requires mutual exclusion; could conceivablybe a lot of code...

signal(sem)

– same lack of programming language support for correct usage

• Important differences in the underlying implementation, however

04/21/23 4

Problems with semaphores (and locks)

• They can be used to solve any of the traditional synchronization problems, but:– semaphores are essentially shared global variables

• can be accessed from anywhere (bad software engineering)

– there is no connection between the semaphore and the data being controlled by it

– used for both critical sections (mutual exclusion) and for coordination (scheduling)

– no control over their use, no guarantee of proper usage

• Thus, they are prone to bugs– another (better?) approach: use programming language

support

04/21/23 5

One More Approach: Monitors

• A monitor is a programming language construct that supports controlled access to shared data– synchronization code is added by the compiler

• why does this help?

• A monitor encapsulates:– shared data structures– procedures that operate on the shared data– synchronization between concurrent threads that invoke those

procedures

• Data can only be accessed from within the monitor, using the provided procedures– protects the data from unstructured access

• Addresses the key usability issues that arise with semaphores

Administrivia

• HW this week• Lectures

– This week: scheduling– Next week: memory management

• Quiz 2 next Tuesday: scheduling• Quiz 3 the following week: memory management• Midterm: tentatively Feb 20, take home, open book

04/21/23 6

04/21/23 7

Scheduling

• In discussing processes and threads, we talked about context switching– an interrupt occurs (device completion, timer interrupt)– a thread causes an exception (a trap or a fault)

• We glossed over the choice of which process or thread is chosen to be run next– “some thread from the ready queue”

• This decision is called scheduling• scheduling is policy

• context switching is mechanism

04/21/23 8

Classes of Schedulers• Batch

– Throughput / utilization oriented– Example: audit inter-bank funds transfers each night

• Interactive– Response time oriented– Example: X tunneling, X host – X client

• Real time– Deadline driven– Example: embedded systems (cars, airplanes, etc.)

• Parallel– Speedup driven– Example: “space-shared” use of a 1000-processor machine for

large simulations

• Others…

We’ll be talking primarily about interactive schedulers for now.

04/21/23 9

Multiple levels of scheduling decisions

• Long term– Should a new “job” be “initiated,” or should it be held?– typical of batch systems– what might cause you to make a “hold” decision?

• Medium term– Should a running program be temporarily marked as non-

runnable (e.g., swapped out)?

• Short term– Which thread should be given the CPU next? For how long?– Which I/O operation should be sent to the disk next?– On a multiprocessor:

• should we attempt to coordinate the running of threads from the same address space in some way?

• should we worry about cache state (processor affinity)?

04/21/23 10

Scheduling Goals I: Performance

• Many possible metrics / performance goals (which sometimes conflict)– maximize CPU utilization– maximize throughput (requests completed / s)– minimize average response (turn-around) time (average time from submission of request to completion of response)

– minimize average waiting time (average time from submission of request to start of execution)

– minimize energy (joules per instruction) subject to some constraint

04/21/23 11

Scheduling Goals II: Fairness

• No single, compelling definition of “fair”– How to measure fairness?

• Equal CPU consumption? (over what time scale?)

– Fair per-user? per-process? per-thread?– What if one process is CPU bound and one is IO bound?

• Sometimes the goal is to be unfair:– Explicitly favor some particular class of requests (priority system), but…

– avoid starvation (be sure everyone gets at least some service)

04/21/23 12

The basic situation

Schedulable units Resources

Scheduling:- Who to assign each resource to- When to re-evaluate your

decisions

04/21/23 13

When to assign?• Pre-emptive vs. non-preemptive schedulers

– Non-preemptive• once you give somebody the green light, they’ve got it until they

relinquish it– an I/O operation– allocation of memory in a system without swapping

– Preemptive• you can re-visit a decision

– setting the timer allows you to preempt the CPU from a thread even if it doesn’t relinquish it voluntarily

– in any modern system, if you mark a program as non-runnable, its memory resources will eventually be re-allocated to others

• Re-assignment always involves some overhead– Overhead doesn’t contribute to the goal of any scheduler

• We’ll assume “work conserving” policies– Never leave a resource idle when someone wants it

• Why even mention this? When might it be useful to do something else?

04/21/23 14

Algorithm #1: FCFS/FIFO

• First-come first-served / First-in first-out (FCFS/FIFO)– schedule in the order that they arrive– “real-world” scheduling of people in (single) lines

• supermarkets, bank tellers, McD’s, Starbucks …

– typically non-preemptive• no context switching at supermarket!

– jobs treated equally, no starvation• In what sense is this “fair”?

• Sounds perfect!– in the real world, when does FCFS/FIFO work well?

• even then, what’s it’s limitation?

– and when does it work badly?

04/21/23 15

FCFS/FIFO example

• Suppose the duration of A is 5 time unit, and the durations of B and C are each 1

• What are the average response (turn around) time?

Job A B C

CB Job A

time

1

2

04/21/23 16

FCFS/FIFO example

• Suppose the duration of A is 5, and the durations of B and C are each 1– average response time for schedule 1 (assuming A, B, and

C all arrive at about time 0) is (5+6+7)/3 = 18/3 = 6– average response time for schedule 2 is (1+2+7)/3 = 10/3 =

3.3– consider also “elongation factor” – a “perceptual” measure:

• Schedule 1: A is 5/5, B is 6/1, C is 7/1 (worst is 7, ave is 4.7)• Schedule 2: A is 7/5, B is 1/1, C is 2/1 (worst is 2, ave is 1.5)

Job A B C

CB Job A

time

1

2

04/21/23 17

• Average response time can be lousy– small requests wait behind big ones

• May lead to poor utilization of other resources– if you send me on my way, I can go keep another resource

busy– FCFS may result in poor overlap of CPU and I/O activity

FCFS/FIFO drawbacks

04/21/23 18

Algorithm #2: SPT/SJF/SPN

• Shortest processing time first / Shortest job first / Shortest process next (SPT/SJF/SPN)– choose the request with the smallest service requirement

• Provably optimal with respect to average response time

04/21/23 19

SPT/SJF/SPN optimality

tk

sf sg

tk+sf tk+sf+sg

• In any schedule that is not SPT/SJF/SPN, there is some adjacent pair of requests f and g where the service time (duration) of f, sf, exceeds that of g, sg

• The total contribution to average response time of f and g is (tk+sf) + (tk+sf+sg) = 2tk+2sf+sg

• If you interchange f and g, their total contribution will be 2tk+2sg+sf, which is smaller because sg < sf

• If the variability among request durations is zero, how does FCFS compare to SPT for average response time?

04/21/23 20

• It’s non-preemptive – So?

• … but there’s a preemptive version – SRPT (Shortest Remaining Processing Time first) – that accommodates arrivals (rather than assuming all requests are initially available)

• Sounds perfect!– what about starvation?– can you know the processing time of a request?– can you guess/approximate? How?

SPT/SJF/SPN drawbacks

04/21/23 21

Algorithm #3: RR

• Round Robin scheduling (RR)– ready queue is treated as a circular FIFO queue– each request is given a time slice, called a quantum

• request executes for duration of quantum, or until it blocks– what signifies the end of a quantum?

• time-division multiplexing (time-slicing)

– great for timesharing• no starvation

• Sounds perfect!– how is RR an improvement over FCFS?– how is RR an improvement over SPT?– how is RR an approximation to SPT?– what are the warts?

Example of RR with Time Quantum = 20• Example: Process Burst Time

P1 53 P2 8 P3 68 P4 24

– The Gantt chart is:

– Waiting time for P1=(68-20)+(112-88)=72P2=(20-0)=20

P3=(28-0)+(88-48)+(125-108)=85P4=(48-0)+(108-68)=88

– Average waiting time = (72+20+85+88)/4=66¼– Average completion time = (125+28+153+112)/4 = 104½

• Thus, Round-Robin Pros and Cons:– Better for short jobs, Fair (+)– Context-switching time adds up for long jobs (-)

P1 P2 P3 P4 P1 P3 P4 P1 P3 P3

0 20 28 48 68 88 108 112 125 145 153

Round-Robin Discussion• How do you choose time slice?

– What if too big?• Response time suffers

– What if infinite ()?• Get back FIFO

– What if time slice too small?• Throughput suffers!

• Actual choices of timeslice:– Initially, UNIX timeslice one second:

• Worked ok when UNIX was used by one or two people.

• What if three compilations going on? 3 seconds to echo each keystroke!

– In practice, need to balance short-job performance and long-job throughput:

• Typical time slice today is between 10ms – 100ms• Typical context-switching overhead is 0.1ms – 1ms• Roughly 1% overhead due to context-switching

Comparisons between FCFS and RR• Assuming zero-cost context-switching time, is RR always

better than FCFS?• Simple example: 10 jobs, each take 100s of CPU time

RR scheduler quantum of 1sAll jobs start at the same time

• Completion Times:

– Both RR and FCFS finish at the same time– Average response time is much worse under RR!

• Bad when all jobs same length

• Also: Cache state must be shared between all jobs with RR but can be devoted to each job with FIFO– Total time for RR longer even for zero-cost switch!

Job # FIFO RR1 100 9912 200 992… … …9 900 99910 1000 1000

04/21/23 25

RR drawbacks

• What if all jobs are exactly the same length?– What would the pessimal schedule be?

• What do you set the quantum to be?– no value is “correct”

• if small, then context switch often, incurring high overhead

• if large, then response time degrades

– treats all jobs equally• if I run 100 copies of SETI@home, it degrades your service

• how might I fix this?

04/21/23 26

Algorithm #4: Priority

• Assign priorities to requests– choose request with highest priority to run next

• if tie, use another scheduling algorithm to break (e.g., RR)

– to implement SJF, priority = expected length of CPU burst

• Abstractly modeled (and usually implemented) as multiple “priority queues”– put a ready request on the queue associated with its priority

• Sounds perfect!

04/21/23 27

Priority drawbacks

• How are you going to assign priorities?

• Starvation– if there is an endless supply of high priority jobs, no low-

priority job will ever run

• Solution: “age” threads over time– increase priority as a function of accumulated wait time– decrease priority as a function of accumulated processing

time– many ugly heuristics have been explored in this space

04/21/23 28

Combining algorithms

• In practice, any real system uses some sort of hybrid approach, with elements of FCFS, SPT, RR, and Priority

• Example: multi-level feedback queues (MLFQ)– there is a hierarchy of queues– there is a priority ordering among the queues– new requests enter the highest priority queue– each queue is scheduled RR– queues have different quanta– requests/jobs move between queues based on execution

history– in what situations might this approximate SJF?

04/21/23 29

UNIX scheduling

• Canonical scheduler is pretty much MLFQ– 3-4 classes spanning ~170 priority levels

• timesharing: lowest 60 priorities• system: middle 40 priorities• real-time: highest 60 priorities

– priority scheduling across queues, RR within• process with highest priority always run first• processes with same priority scheduled RR

– processes dynamically change priority• increases over time if process blocks before end of quantum• decreases if process uses entire quantum

• Goals:– reward interactive behavior over CPU hogs

• interactive jobs typically have short bursts of CPU

04/21/23 30

Scheduling the Apache web server SRPT

• What does a web request consist of? (What’s it trying to get done?)

• How are incoming web requests scheduled, in practice?

• How might you estimate the service time of an incoming request?

• Starvation under SRPT is a problem in theory – is it a problem in practice?– “Kleinrock’s conservation law”

(Recent work by Bianca Schroeder and Mor Harchol-Balter at CMU)

04/21/23 31© 2003 Bianca Schroeder & Mor Harchol-Balter, CMU

04/21/23 32

Summary

• Scheduling takes place at many levels• It can make a huge difference in performance

– this difference increases with the variability in service requirements

• Multiple goals, sometimes conflicting• There are many “pure” algorithms, most with some

drawbacks in practice – FCFS, SPT, RR, Priority• Real systems use hybrids• Recent work has shown that SPT/SRPT – always

known to be beneficial in principle – may be more practical in some settings than long thought