CAR: CLOCK with ADAPTIVE REPLACEMENT - Department of Computer

FAST 2004, San Francisco, CA

CAR: Bansal & Modha April 2, 2004 © 2003 IBM Corporation

CAR: CLOCK with ADAPTIVE REPLACEMENT

Sorav Bansal, Stanford UniversityDharmendra S. Modha, IBM Research

CAR: Bansal & Modha

© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004

The Caching Problem

expensive, but fast cheap, but slow

How to manage the cache?

Assume demand paging: Which page to replace?

How to maximize the hit ratio?

CAR: Bansal & Modha


☺Constant Time☺Recency

L R U

CAR: Bansal & Modha


“Frequency”

Scan Resistance☺Constant Time☺Recency

L R U

CAR: Bansal & Modha


☺“Frequency”☺Scan Resistance

☺Constant Time☺Recency

LFU

L R U

CAR: Bansal & Modha


☺☺“Frequency”☺☺Scan Resistance☺☺Constant Time☺☺Recency

A R C

LFU

L R U

CAR: Bansal & Modha


☺☺“Frequency”

Lock Contention/ MRU Overhead

☺☺Scan Resistance☺☺Constant Time☺☺Recency

A R C

LFU

L R U

CAR: Bansal & Modha


“In Multics a paging algorithm has been developed that has

the implementation ease and low overhead of the FIFO and

is an approximation to LRU.”

Fernando J. Corbato, 1990 Turing Award Winner

CAR: Bansal & Modha


☺☺“Frequency”

☺Lock Contention/ MRU Overhead

☺☺Scan Resistance☺☺☺Constant Time☺☺☺Recency

C L OCK

A R C

LFU

L R U

CAR: Bansal & Modha


0

0

1

0

100

0

1

1

0

0

1

0

1

01

1 10

1

1

0

1

TAIL

HE AD

CLOCK

CAR: Bansal & Modha


0

0

1

0

100

0

1

1

0

0

1

0

1

01

1 10

1

1

0

1

TAIL

HE AD

CLOCK

HIT: Set the reference bit to “1”

MISS: Insert at the TAIL, initialize the reference bit to “0”

CAR: Bansal & Modha


0

0

1

0

100

0

1

1

0

0

1

0

1

01

1 10

1

1

0

1

TAIL

HE AD

CLOCK

REPLACEMENT POLICY: Evict the first “0” page

Reset “1” pages to “0”—“second chance”—“delayed MRU”

CAR: Bansal & Modha


CLOCK Applications and Exposition

Multics

UNIX/AIX/LINUX/BSD

VAX/VMS

DB2

Oracle? Windows? Solaris?

Major OS Textbooks– Tanebaum & Woodhull– Silberschatz & Galvin

CAR: Bansal & Modha


Prior Work on LRU versus CLOCK

CLOCK– GCLOCK– Two-handed CLOCK

LRU– (LFU)– FBR– LRU-2– 2Q– LRFU– LIRS– MQ– ARC

CAR: Bansal & Modha




LRU– (LFU)– FBR– LRU-2– 2Q– LRFU– LIRS– MQ– ARC 1968

CAR: Bansal & Modha




LRU– (LFU)– FBR– LRU-2– 2Q– LRFU– LIRS– MQ– ARC 1968 1998

CAR: Bansal & Modha


☺☺☺“Frequency”

☺☺Lock Contention/ MRU Overhead

☺☺☺Scan Resistance☺☺☺☺Constant Time☺☺☺☺Recency

C A R

C L OCK

A R C

LFU

L R U

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21

"Frequency""Recency"

TAIL TAIL

HEADHEAD

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

CLOCKs T1 and T2 contain cache pages

LRU lists B1 and B2 contain recently evicted history pages

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

Size of T1 roughly equals B2Size of T2 roughly equals B1

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

T1(0) & B1 pages seen exactly once recentlyT1(1), T2, & B2 pages seen at least twice recently

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

Maintain a target size for CLOCK T1

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

HIT in T1 or T2: Set reference bit to “1”

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

MISS in B1: Set reference bit to “0”, move to TAIL of T2, and increase target size of T1

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

MISS in B2: Set reference bit to “0”, move to TAIL of T2, and decrease target size of T1

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

TOTAL MISS: Set reference bit to “0”, move to TAIL of T1

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

CACHE REPLACEMENT POLICY:Replace from T1 if larger than target; else from T2

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

During replacement in T1, if “1” page is found, make “0” and move to T2 TAIL, move evicted page to B1

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

During replacement in T2, if “1” page is found, make “0”, move evicted page to B2

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

DIRECTORY REPLACEMENT POLICY:Replace from B1 if T1+B1 = c; else from B2

CAR: Bansal & Modha


2000000 6000000 10000000 14000000 197760000

8192

16384

24576

32768P4

Virtual Time (Request Number)

Tar

get S

ize

for

List

T1

CAR: Bansal & Modha


CART = CAR + Temporal Filtering

CAR/ARC: two hits to a page is a criterion for promotion from T1 to T2

CART: promotion from T1 to T2 happens only if two hits are “far”

CAR: Bansal & Modha


SPC-1 like Workload

21.9020.0020.009.319.1910485768.527.627.563.643.665242883.563.29 3.231.631.632621441.781.66 1.620.770.781310720.900.840.820.370.3765536

CARTCARARCCLOCKLRUCache Size (4K pages)

CAR: Bansal & Modha


87.2687.2687.2687.2687.30419430471.7771.7171.4169.5068.68209715257.6457.9257.1952.24 50.86157286441.8341.0040.4429.0427.62104857626.1224.4724.348.668.0652428815.0714.2514.303.273.232621448.207.767.781.581.591310724.274.054.070.790.79655362.202.072.080.400.40327681.101.03 1.04 0.20 0.20 16384

CARTCARARCCLOCKLRUCache Size (4k Pages)

Merge(S) Workload

CAR: Bansal & Modha


1

2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P1

Cache Size (Number of 512 byte pages)

CAR: Bansal & Modha


4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P2


CAR: Bansal & Modha


2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P3


CAR: Bansal & Modha


4

8

16

32

Hit

Rat

io (

%)

CAR

CLOCK

1024 4096 16384 65536 262144

P4


CAR: Bansal & Modha


8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P5


CAR: Bansal & Modha


1

2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P6


CAR: Bansal & Modha


2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P7


CAR: Bansal & Modha


0.5

1

2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P8


CAR: Bansal & Modha


4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P9


CAR: Bansal & Modha


2

4

8

16

32

Hit

Rat

io (

%)

CAR

CLOCK

1024 4096 16384 65536 262144

P10


CAR: Bansal & Modha


8

16

32

64

Hit

Rat

io (

%)

CAR

CLOCK

1024 4096 16384 65536 262144

P11


CAR: Bansal & Modha


8

16

32

Hit

Rat

io (

%)

CAR

CLOCK

1024 4096 16384 65536 262144

P12


CAR: Bansal & Modha


4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

1024 4096 16384 65536 262144

P13


CAR: Bansal & Modha


8

16

32

Hit

Rat

io (

%)

CAR

CLOCK

1024 4096 16384 65536 262144

P14


CAR: Bansal & Modha


8

16

32

64H

it R

atio

(%

)

CAR

LRU

1024 4096 16384 65536 262144

4

ConCat(P)


CAR: Bansal & Modha


8

16

32

Hit

Rat

io (

%)

CAR

LRU

8192 32768 131072 524288

Merge(P)


CAR: Bansal & Modha


4

8

16

32

Hit

Rat

io (

%)

CAR

CLOCK

65536 262144 1048576 4194304

2

DS1


CAR: Bansal & Modha


2

4

8

16

32

Hit

Rat

io (

%) CAR

CLOCK

65536 131072 262144 523288 1048576

S1


CAR: Bansal & Modha


2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

65536 131072 262144 524288 1048576

S2


CAR: Bansal & Modha


2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

65536 131072 262144 524288 1048576

S3


CAR: Bansal & Modha


1

2

4

8

16

32

64H

it R

atio

(%

)

CAR

CLOCK

16384 65536 262144 1048576 4194304

Merge(S)


CAR: Bansal & Modha


1

2

4

8

16

32

Hit

Rat

io (

%)

CAR

CLOCK

65536 262144 1048576 4194304

SPC1


CAR: Bansal & Modha


CAR: CONCLUSIONS

Simple and Low Overhead

Self-tuning: captures “recency” and “frequency”

Scan-Resistant

Low Lock Contention and MRU Overhead

Outperforms CLOCK on all workloads examined

Comparable to (sometimes even better than) ARC!

CAR: Bansal & Modha


0

0

1

01

000

1

1

0

0

1

0

1

01

1 10

1

1

0

1

0

00

1

0

1

10 1

0

1

1

MRU

LRU

MRU

LRU

T

T2

1

B B21


TAIL TAIL

HEADHEAD

CAR: Bansal & Modha


CAR: Set-up

Clocks T1 and T2 contain cache pagesB1 and B2 contain recently evicted history pages – Alex Haley: “History is written by the winners”– CAR/ARC: “History is written by the losers”Size of T1 roughly equals B2Size of T2 roughly equals B1T1(0) and B1 contain pages that have been seen exactly once recently = “Recency”T1(1), T2, and B2 contain pages that have been seen at least twice recently = “Frequency”

CAR: Bansal & Modha


CAR: Algorithm

HIT in T1 or T2: Set reference bit to 1MISS in B1: Set reference bit to 0, move to tail of T2, and increase target size of T1MISS in B2: Set reference bit to 0, move to tail of T2, and decrease target size of T1TOTAL MISS: Set reference bit to 0, move to tail of T1CACHE REPLACEMENT POLICY: Replace from T1 if larger than target; else from T2– During Replacement in T1, if “1” page is found, make “0” and

move to T2 tail, move evicted page to B1– During Replacement in T2, if “1” page is found, make “0”, move

evicted page to B2DIRECTORY REPLACEMENT POLICY: Replace from B1 if T1+B1 = c; else from B2

CAR: Bansal & Modha


CLOCK

HIT: Set the reference bit to “1”

MISS: Insert at the TAIL, initialize the reference bit to “0”

REPLACEMENT POLICY:– Evict the first “0” page – Reset “1” pages to “0”—giving

them a “second chance”

KEY INSIGHT:– Reseting “1” to “0” is “delayed

MRU”—removing it from hit path to miss path

0

0

1

0

100

0

1

1

0

0

1

0

1

01

1 10

1

1

0

1

TAIL

HE AD

CLOCK

CAR: CLOCK with ADAPTIVE REPLACEMENT - Department of Computer

Documents

Transcript of CAR: CLOCK with ADAPTIVE REPLACEMENT - Department of Computer