Virtual Memory

2

Last Week Memory Management

• Increase degree of multiprogramming– Entire process needs to fit into memory

• Dynamic Linking and Loading• Swapping • Contiguous Memory Allocation

– Dynamic storage memory allocation problem• First-fit, best-fit, worst-fit• Fragmentation - external and internal

– Paging• Structure of the Page Table

– Segmentation

3

Goals for Today

• Virtual memory• How does it work?

– Page faults– Resuming after page faults

• When to fetch?• What to replace?

– Page replacement algorithms• FIFO, OPT, LRU (Clock)

– Page Buffering– Allocating Pages to processes

4

What is virtual memory?• Each process has illusion of large address space

– 232 for 32-bit addressing

• However, physical memory is much smaller• How do we give this illusion to multiple processes?

– Virtual Memory: some addresses reside in disk

page table

Physical memory

disk

Virtual memory

page 0page 1page 2page 3

page 4

page N

5

Virtual memory• Separates users logical memory from

physical memory.– Only part of the program needs to be in

memory for execution– Logical address space can therefore be much

larger than physical address space– Allows address spaces to be shared by several

processes– Allows for more efficient process creation

6

Virtual Memory

• Load entire process in memory (swapping), run it, exit– Is slow (for big processes)– Wasteful (might not require everything)

• Solutions: partial residency– Paging: only bring in pages, not all pages of process– Demand paging: bring only pages that are required

• Where to fetch page from?– Have a contiguous space in disk: swap file (pagefile.sys)

7

How does VM work?

• Modify Page Tables with another bit (“is present”)– If page in memory, is_present = 1, else is_present = 0– If page is in memory, translation works as before– If page is not in memory, translation causes a page fault

Disk

Mem

32 :P=14183 :P=0177 :P=15721 :P=0

Page Table

0123

8

Page Faults

• On a page fault:– OS finds a free frame, or evicts one from memory (which one?)

• Want knowledge of the future?

– Issues disk request to fetch data for page (what to fetch?)

• Just the requested page, or more?

– Block current process, context switch to new process (how?)

• Process might be executing an instruction

– When disk completes, set present bit to 1, and current process in ready queue

9

Steps in Handling a Page Fault

10

Resuming after a page fault

• Should be able to restart the instruction

• For RISC processors this is simple:– Instructions are idempotent until references are done

• More complicated for CISC:– E.g. move 256 bytes from one location to another

– Possible Solutions:

• Ensure pages are in memory before the instruction executes

11

When to fetch?

• Just before the page is used!– Need to know the future

• Demand paging:– Fetch a page when it faults

• Prepaging:– Get the page on fault + some of its neighbors, or– Get all pages in use last time process was swapped

12

Performance of Demand Paging

• Page Fault Rate 0 p 1.0– if p = 0 no page faults – if p = 1, every reference is a fault

• Effective Access Time (EAT)EAT = (1 – p) x memory access

+ p (page fault overhead + swap page out + swap page in + restart overhead

)

13

Demand Paging Example

• Memory access time = 200 nanoseconds

• Average page-fault service time = 8 milliseconds

• EAT = (1 – p) x 200 + p (8 milliseconds) = (1 – p x 200 + p x 8,000,000

= 200 + p x 7,999,800

• If one access out of 1,000 causes a page fault EAT = 8.2 microseconds. This is a slowdown by a factor of 40!!

14

What to replace?

• What happens if there is no free frame?– find some page in memory, but not really in use, swap it out

• Page Replacement– When process has used up all frames it is allowed to use– OS must select a page to eject from memory to allow new page– The page to eject is selected using the Page Replacement Algo

• Goal: Select page that minimizes future page faults

15

Page Replacement

• Prevent over-allocation of memory by modifying page-fault service routine to include page replacement

• Use modify (dirty) bit to reduce overhead of page transfers – only modified pages are written to disk

• Page replacement completes separation between logical memory and physical memory – large virtual memory can be provided on a smaller physical memory

16

Page Replacement

17

Page Replacement Algorithms

• Random: Pick any page to eject at random– Used mainly for comparison

• FIFO: The page brought in earliest is evicted– Ignores usage– Suffers from “Belady’s Anomaly”

• Fault rate could increase on increasing number of pages

• E.g. 0 1 2 3 0 1 4 0 1 2 3 4 with frame sizes 3 and 4

• OPT: Belady’s algorithm– Select page not used for longest time

• LRU: Evict page that hasn’t been used the longest– Past could be a good predictor of the future

18

Example: FIFO, OPT

Reference stream is A B C A B D A D B C

OPTIMALA B C A B D A D B C B

toss A or Dtoss C5 Faults

FIFOA B C A B D A D B C B

toss A

ABCDABC

toss ?7 Faults

19

First-In-First-Out (FIFO) Algorithm

• Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5• 3 frames (3 pages can be in memory at a time per

process): 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

• 4 frames: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

• Belady’s Anomaly: more frames more page faults

1

2

3

1

2

3

4

1

2

5

3

4

9 page faults

1

2

3

1

2

3

5

1

2

4

510 page faults

44 3

20

FIFO Illustrating Belady’s Anomaly

21

Optimal Algorithm

• Replace page that will not be used for longest period of time

• 4 frames example 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

• How do you know this?• Used for measuring how well your algorithm performs

1

2

3

46 page faults

4 5

22

Least Recently Used (LRU) Algorithm

• Reference string: 1, 2, 3, 4, 1, 2, 5, 1, 2, 3, 4, 5

• Counter implementation– Every page entry has a counter; every time page is

referenced through this entry, copy the clock into the counter

– When a page needs to be changed, look at the counters to determine which are to change

5

2

4

3

1

2

3

4

1

2

5

4

1

2

5

3

1

2

4

3

23

Implementing Perfect LRU

• On reference: Time stamp each page• On eviction: Scan for oldest frame• Problems:

– Large page lists– Timestamps are costly

• Approximate LRU– LRU is already an approximation!

0xffdcd: add r1,r2,r30xffdd0: ld r1, 0(sp)

t=4

t=14

t=14

t=5

13

14

14

24

LRU: Clock Algorithm

• Each page has a reference bit– Set on use, reset periodically by the OS

• Algorithm:– FIFO + reference bit (keep pages in circular list)

• Scan: if ref bit is 1, set to 0, and proceed. If ref bit is 0, stop and evict.

• Problem:– Low accuracy for large memory

R=1R=0

R=1

R=1

R=1R=0R=0

R=1

R=0

R=0

R=1

25

LRU with large memory

• Solution: Add another hand– Leading edge clears ref bits– Trailing edge evicts pages with ref bit 0

• What if angle small?• What if angle big?

R=1R=0

R=1

R=1

R=1R=0R=0

R=1

R=0

R=0

R=1

26

Clock Algorithm: Discussion

• Sensitive to sweeping interval– Fast: lose usage information– Slow: all pages look used

• Clock: add reference bits– Could use (ref bit, modified bit) as ordered pair– Might have to scan all pages

• LFU: Remove page with lowest count– No track of when the page was referenced– Use multiple bits. Shift right by 1 at regular intervals.

• MFU: remove the most frequently used page• LFU and MFU do not approximate OPT well

27

28

Page Buffering

• Cute simple trick: (XP, 2K, Mach, VMS)– Keep a list of free pages– Track which page the free page corresponds to– Periodically write modified pages, and reset modified bit

addevict

modified list(batch writes = speed)

unmodified free list

used

free

29

Allocating Pages to Processes

• Global replacement– Single memory pool for entire system– On page fault, evict oldest page in the system– Problem: protection

• Local (per-process) replacement– Have a separate pool of pages for each process– Page fault in one process can only replace pages from its own

process– Problem: might have idle resources

30

Allocation of Frames• Each process needs minimum number of

pages• Example: IBM 370 – 6 pages to handle SS

MOVE instruction:– instruction is 6 bytes, might span 2 pages– 2 pages to handle from– 2 pages to handle to

• Two major allocation schemes– fixed allocation– priority allocation

Virtual Memory II: Thrashing

Working Set AlgorithmDynamic Memory Management

32

Last Time

• We’ve focused on demand paging– Each process is allocated pages of memory– As a process executes it references pages– On a miss a page fault to the O/S occurs– The O/S pages in the missing page and pages out some target

page if room is needed

• The CPU maintains a cache of PTEs: the TLB.• The O/S must flush the TLB before looking at page

reference bits, or before context switching (because this changes the page table)

33

Example: LRU, =3R 3 7 9 5 3 6 3 5 6 7 9 7 9 3 8 6 3 6 8 3 5 6

3 7 9 5 3 6 3 5 6 7 9 7 9 3 8 6 3 6 8 3 5 6

S 3 7 9 5 3 6 3 5 6 7 9 7 9 3 8 6 3 6 8 3 5

3 7 9 5 5 6 3 5 6 6 6 7 9 3 8 8 3 6 8 3

In 3 7 9 5 3 6 7 9 3 8 6 5 6

Out 3 7 9 3 5 6 7 9 6 8

R(t): Page referenced at time t. S(t): Memory state when finished doing the paging at time t. In(t): Page brought in, if any. Out(t): Page sent out. : None.

While “filling” memory we are likely to get a page fault on almost every reference. Usually we don’t include these events when computing the hit ratio

Hit ratio: 9/19 = 47%Miss ratio: 10/19 = 53%

34

Goals for today

• Review demand paging• What is thrashing?• Solutions to thrashing

– Approach 1: Working Set– Approach 2: Page fault frequency

• Dynamic Memory Management– Memory allocation and deallocation and goals– Memory allocator - impossibility result

• Best fit• Simple scheme - chunking, binning, and free• Buddy-block scheme

• Other issues

35

Thrashing

• Def: Excessive rate of paging that occurs because processes in system require more memory – Keep throwing out page that will be referenced soon

– So, they keep accessing memory that is not there

• Why does it occur?– Poor locality, past != future

– There is reuse, but process does not fit

– Too many processes in the system

36

Approach 1: Working Set

• Peter Denning, 1968– He uses this term to denote memory locality of a program

Def: pages referenced by process in last time-units comprise its working set

For our examples, we usually discuss WS in terms of , a “window” in the page reference string. But while this is easier on paper it makes less sense in practice!

In real systems, the window should probably be a period of time, perhaps a second or two.

37

Working Sets

• The working set size is num pages in the working set – the number of pages touched in the interval [t-Δ+1..t].

• The working set size changes with program locality.– during periods of poor locality, you reference more pages.

– Within that period of time, you will have a larger working set size.

• Goal: keep WS for each process in memory.– E.g. If WSi for all i runnable processes > physical memory, then suspend a process

38

Working Set Approximation

• Approximate with interval timer + a reference bit• Example: = 10,000

– Timer interrupts after every 5000 time units– Keep in memory 2 bits for each page– Whenever a timer interrupts copy and sets the values of all

reference bits to 0– If one of the bits in memory = 1 page in working set

• Why is this not completely accurate?– Cannot tell (within interval of 5000) where reference occured

• Improvement = 10 bits and interrupt every 1000 time units

39

Using the Working Set

• Used mainly for prepaging

– Pages in working set are a good approximation

• In Windows processes have a max and min WS size

– At least min pages of the process are in memory

– If > max pages in memory, on page fault a page is replaced

– Else if memory is available, then WS is increased on page fault

– The max WS can be specified by the application

40

Theoretical aside

• Denning defined a policy called WSOpt– In this, the working set is computed over the

next references, not the last: R(t)..R(t+-1)

• He compared WS with WSOpt.– WSOpt has knowledge of the future…– …yet even though WS is a practical algorithm

with no ability to see the future, we can prove that the Hit and Miss ratio is identical for the two algorithms!

41

Key insight in proof

• Basic idea is to look at the paging decision made in WS at time t+-1 and compare with the decision made by WSOpt at time t

• Both look at the same references… hence make same decision– Namely, WSOpt tosses out page R(t-1) if it

isn’t referenced “again” in time t..t+-1– WS running at time t+-1 tosses out page R(t-

1) if it wasn’t referenced in times t...t+-1

42

How do WSOpt and WS differ?

• WS maintains more pages in memory, because it needs time “delay” to make a paging decision– In effect, it makes the same decisions, but it

makes them after a time lag– Hence these pages hang around a bit longer

43

How do WS and LRU compare?

• Suppose we use the same value of – WS removes pages if they aren’t referenced

and hence keeps less pages in memory– When it does page things out, it is using an

LRU policy!– LRU will keep all pages in memory,

referenced or not

• Thus LRU often has a lower miss rate, but needs more memory than WS

44

Approach 2: Page Fault Frequency

• Thrashing viewed as poor ratio of fetch to work• PFF = page faults / instructions executed • if PFF rises above threshold, process needs more memory

– not enough memory on the system? Swap out.

• if PFF sinks below threshold, memory can be taken away

45

Working Sets and Page Fault Rates

Page fault rate

transition

Working set

stable

46

Dynamic Memory Management

• Notice that the O/S kernel can manage memory in a fairly trivial way:– All memory allocations are in units of “pages”– And pages can be anywhere in memory… so a simple

free list is the only data structure needed

• But for variable-sized objects, we need a heap:– Used for all dynamic memory allocations

• malloc/free in C, new/delete in C++, new/garbage collection in Java

• Also, managing kernel memory

– Is a very large array allocated by OS, managed by program

47

Allocation and deallocation• What happens when you call:

– int *p = (int *)malloc(2500*sizeof(int));• Allocator slices a chunk of the heap and gives it to the program

– free(p);• Deallocator will put back the allocated space to a free list

• Simplest implementation:– Allocation: increment pointer on every allocation– Deallocation: no-op– Problems: lots of fragmentation

heap (free memory)

current free positionallocation

48

Memory allocation goals

• Minimize space– Should not waste space, minimize fragmentation

• Minimize time– As fast as possible, minimize system calls

• Maximizing locality– Minimize page faults cache misses

• And many more

• Proven: impossible to construct “always good” memory allocator

49

Memory Allocator

• What allocator has to do:– Maintain free list, and grant memory to requests– Ideal: no fragmentation and no wasted time

• What allocator cannot do:– Control order of memory requests and frees– A bad placement cannot be revoked

• Main challenge: avoid fragmentation

20 20 2010 10

a

malloc(20)?

b

50

Impossibility Results

• Optimal memory allocation is NP-complete for general computation

• Given any allocation algorithm, streams of allocation and deallocation requests that defeat the allocator and cause extreme fragmentation

51

Best Fit Allocation

• Minimum size free block that can satisfy request• Data structure:

– List of free blocks– Each block has size, and pointer to next free block

• Algorithm:– Scan list for the best fit

20 30 30 37

52

Best Fit gone wrong

• Simple bad case: allocate n, m (m<n) in alternating orders, free all the m’s, then try to allocate an m+1.

• Example:– If we have 100 bytes of free memory

– Request sequence: 19, 21, 19, 21, 19

– Free sequence: 19, 19, 19

– Wasted space: 57!

19 21 19 21 19

19 21 19 21 19

53

A simple scheme

• Each memory chunk has a signature before and after– Signature is an int– +ve implies the a free chunk– -ve implies that the chunk is currently in use– Magnitude of chunk is its size

• So, the smallest chunk is 3 elements:– One each for signature, and one for holding the data

54

Which chunk to allocate?

• Maintain a list of free chunks– Binning, doubly linked lists, etc

• Use best fit or any other strategy to determine page– For example: binning with best-fit

• What if allocated chunk is much bigger than request?– Internal fragmentation

– Solution: split chunks• Will not split unless both chunks above a minimum size

• What if there is no big-enough free chunk?– sbrk (changes segment size) or mmap

– Possible page fault

55

What happens on free?

• Identify size of chunk returned by user• Change sign on both signatures (make +ve)• Combine free adjacent chunks into bigger chunk

– Worst case when there is one free chunk before and after– Recalculate size of new free chunk– Update the signatures

• Don’t really need to erase old signatures

56

Example

Initially one chunk, split and make signs negative on malloc

+8

+8

-2

-2

+4

+4

p = malloc(2 * sizeof (int));

57

Example

q gets 4 words, although it requested for 3

+8

+8

-2

-2

-4

-4

p = malloc(2 * sizeof (int));

q = malloc(3 * sizeof (int));

58

Design features

• Which free chunks should service request– Ideally avoid fragmentation… requires future knowledge

• Split free chunks to satisfy smaller requests– Avoids internal fragmentation

• Coalesce free blocks to form larger chunks– Avoids external fragmentation

3020 10 30 30

59

Buddy-Block Scheme

• Invented by Donald Knuth, very simple

• Idea: Work with memory regions that are all powers of 2 times some “smallest” size– 2k times b

• Round each request up to have form b*2k

60

Buddy-Block Scheme

61

Buddy-Block Scheme

• Keep a free list for each block size (each k)– When freeing an object, combine with adjacent free regions if

this will result in a double-sized free object

• Basic actions on allocation request:– If request is a close fit to a region on the free list, allocate that

region. – If request is less than half the size of a region on the free list,

split the next larger size of region in half– If request is larger than any region, double the size of the heap

(this puts a new larger object on the free list)

62

How to get more space?

• In Unix, system call sbrk()

• Used by malloc if heap needs to be expanded

• Notice that heap only grows on “one side”

/* add nbytes of valid virtual address space */ void *get_free_space(unsigned nbytes) { void *p; if(!(p = sbrk(nbytes))) error(“virtual memory exhausted”); return p;}

63

Malloc & OS memory management

• Relocation– OS allows easy relocation (change page table)– Placement decisions permanent at user level

• Size and distribution– OS: small number of large objects– Malloc: huge number of small objects

heap

codedatastack

64

Other Issues – Program Structure• Int[128,128] data;• Each row is stored in one page

• Program 1 – for (j = 0; j <128; j++) for (i = 0; i < 128; i++)

data[i,j] = 0;

– 128 x 128 = 16,384 page faults

• Program 2 – for (i = 0; i < 128; i++) for (j = 0; j < 128; j++) data[i,j] = 0;

– 128 page faults

65

OS and Paging

• Process Creation:– Allocate space and initialize page table for program and data

– Allocate and initialize swap area

– Info about PT and swap space is recorded in process table

• Process Execution– Reset MMU for new process

– Flush the TLB

– Bring processes’ pages in memory

• Page Faults• Process Termination

– Release pages