FAST 2004, San Francisco, CA
CAR: Bansal & Modha April 2, 2004 © 2003 IBM Corporation
CAR: CLOCK with ADAPTIVE REPLACEMENT
Sorav Bansal, Stanford UniversityDharmendra S. Modha, IBM Research
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
The Caching Problem
expensive, but fast cheap, but slow
How to manage the cache?
Assume demand paging: Which page to replace?
How to maximize the hit ratio?
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
☺Constant Time☺Recency
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
“Frequency”
Scan Resistance☺Constant Time☺Recency
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
☺“Frequency”☺Scan Resistance
☺Constant Time☺Recency
LFU
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
☺☺“Frequency”☺☺Scan Resistance☺☺Constant Time☺☺Recency
A R C
LFU
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
☺☺“Frequency”
Lock Contention/ MRU Overhead
☺☺Scan Resistance☺☺Constant Time☺☺Recency
A R C
LFU
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
“In Multics a paging algorithm has been developed that has
the implementation ease and low overhead of the FIFO and
is an approximation to LRU.”
Fernando J. Corbato, 1990 Turing Award Winner
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
☺☺“Frequency”
☺Lock Contention/ MRU Overhead
☺☺Scan Resistance☺☺☺Constant Time☺☺☺Recency
C L OCK
A R C
LFU
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
0
100
0
1
1
0
0
1
0
1
01
1 10
1
1
0
1
TAIL
HE AD
CLOCK
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
0
100
0
1
1
0
0
1
0
1
01
1 10
1
1
0
1
TAIL
HE AD
CLOCK
HIT: Set the reference bit to “1”
MISS: Insert at the TAIL, initialize the reference bit to “0”
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
0
100
0
1
1
0
0
1
0
1
01
1 10
1
1
0
1
TAIL
HE AD
CLOCK
REPLACEMENT POLICY: Evict the first “0” page
Reset “1” pages to “0”—“second chance”—“delayed MRU”
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
CLOCK Applications and Exposition
Multics
UNIX/AIX/LINUX/BSD
VAX/VMS
DB2
Oracle? Windows? Solaris?
Major OS Textbooks– Tanebaum & Woodhull– Silberschatz & Galvin
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
Prior Work on LRU versus CLOCK
CLOCK– GCLOCK– Two-handed CLOCK
LRU– (LFU)– FBR– LRU-2– 2Q– LRFU– LIRS– MQ– ARC
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
Prior Work on LRU versus CLOCK
CLOCK– GCLOCK– Two-handed CLOCK
LRU– (LFU)– FBR– LRU-2– 2Q– LRFU– LIRS– MQ– ARC 1968
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
Prior Work on LRU versus CLOCK
CLOCK– GCLOCK– Two-handed CLOCK
LRU– (LFU)– FBR– LRU-2– 2Q– LRFU– LIRS– MQ– ARC 1968 1998
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
☺☺☺“Frequency”
☺☺Lock Contention/ MRU Overhead
☺☺☺Scan Resistance☺☺☺☺Constant Time☺☺☺☺Recency
C A R
C L OCK
A R C
LFU
L R U
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
CLOCKs T1 and T2 contain cache pages
LRU lists B1 and B2 contain recently evicted history pages
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
Size of T1 roughly equals B2Size of T2 roughly equals B1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
T1(0) & B1 pages seen exactly once recentlyT1(1), T2, & B2 pages seen at least twice recently
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
Maintain a target size for CLOCK T1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
HIT in T1 or T2: Set reference bit to “1”
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
MISS in B1: Set reference bit to “0”, move to TAIL of T2, and increase target size of T1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
MISS in B2: Set reference bit to “0”, move to TAIL of T2, and decrease target size of T1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
TOTAL MISS: Set reference bit to “0”, move to TAIL of T1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
CACHE REPLACEMENT POLICY:Replace from T1 if larger than target; else from T2
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
During replacement in T1, if “1” page is found, make “0” and move to T2 TAIL, move evicted page to B1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
During replacement in T2, if “1” page is found, make “0”, move evicted page to B2
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
DIRECTORY REPLACEMENT POLICY:Replace from B1 if T1+B1 = c; else from B2
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2000000 6000000 10000000 14000000 197760000
8192
16384
24576
32768P4
Virtual Time (Request Number)
Tar
get S
ize
for
List
T1
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
CART = CAR + Temporal Filtering
CAR/ARC: two hits to a page is a criterion for promotion from T1 to T2
CART: promotion from T1 to T2 happens only if two hits are “far”
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
SPC-1 like Workload
21.9020.0020.009.319.1910485768.527.627.563.643.665242883.563.29 3.231.631.632621441.781.66 1.620.770.781310720.900.840.820.370.3765536
CARTCARARCCLOCKLRUCache Size (4K pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
87.2687.2687.2687.2687.30419430471.7771.7171.4169.5068.68209715257.6457.9257.1952.24 50.86157286441.8341.0040.4429.0427.62104857626.1224.4724.348.668.0652428815.0714.2514.303.273.232621448.207.767.781.581.591310724.274.054.070.790.79655362.202.072.080.400.40327681.101.03 1.04 0.20 0.20 16384
CARTCARARCCLOCKLRUCache Size (4k Pages)
Merge(S) Workload
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
1
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P1
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P2
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P3
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
4
8
16
32
Hit
Rat
io (
%)
CAR
CLOCK
1024 4096 16384 65536 262144
P4
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P5
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
1
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P6
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P7
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0.5
1
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P8
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P9
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2
4
8
16
32
Hit
Rat
io (
%)
CAR
CLOCK
1024 4096 16384 65536 262144
P10
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
8
16
32
64
Hit
Rat
io (
%)
CAR
CLOCK
1024 4096 16384 65536 262144
P11
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
8
16
32
Hit
Rat
io (
%)
CAR
CLOCK
1024 4096 16384 65536 262144
P12
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
1024 4096 16384 65536 262144
P13
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
8
16
32
Hit
Rat
io (
%)
CAR
CLOCK
1024 4096 16384 65536 262144
P14
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
8
16
32
64H
it R
atio
(%
)
CAR
LRU
1024 4096 16384 65536 262144
4
ConCat(P)
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
8
16
32
Hit
Rat
io (
%)
CAR
LRU
8192 32768 131072 524288
Merge(P)
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
4
8
16
32
Hit
Rat
io (
%)
CAR
CLOCK
65536 262144 1048576 4194304
2
DS1
Cache Size (Number of 512 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2
4
8
16
32
Hit
Rat
io (
%) CAR
CLOCK
65536 131072 262144 523288 1048576
S1
Cache Size (Number of 4096 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
65536 131072 262144 524288 1048576
S2
Cache Size (Number of 4096 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
65536 131072 262144 524288 1048576
S3
Cache Size (Number of 4096 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
1
2
4
8
16
32
64H
it R
atio
(%
)
CAR
CLOCK
16384 65536 262144 1048576 4194304
Merge(S)
Cache Size (Number of 4096 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
1
2
4
8
16
32
Hit
Rat
io (
%)
CAR
CLOCK
65536 262144 1048576 4194304
SPC1
Cache Size (Number of 4096 byte pages)
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
CAR: CONCLUSIONS
Simple and Low Overhead
Self-tuning: captures “recency” and “frequency”
Scan-Resistant
Low Lock Contention and MRU Overhead
Outperforms CLOCK on all workloads examined
Comparable to (sometimes even better than) ARC!
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
0
0
1
01
000
1
1
0
0
1
0
1
01
1 10
1
1
0
1
0
00
1
0
1
10 1
0
1
1
MRU
LRU
MRU
LRU
T
T2
1
B B21
"Frequency""Recency"
TAIL TAIL
HEADHEAD
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
CAR: Set-up
Clocks T1 and T2 contain cache pagesB1 and B2 contain recently evicted history pages – Alex Haley: “History is written by the winners”– CAR/ARC: “History is written by the losers”Size of T1 roughly equals B2Size of T2 roughly equals B1T1(0) and B1 contain pages that have been seen exactly once recently = “Recency”T1(1), T2, and B2 contain pages that have been seen at least twice recently = “Frequency”
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
CAR: Algorithm
HIT in T1 or T2: Set reference bit to 1MISS in B1: Set reference bit to 0, move to tail of T2, and increase target size of T1MISS in B2: Set reference bit to 0, move to tail of T2, and decrease target size of T1TOTAL MISS: Set reference bit to 0, move to tail of T1CACHE REPLACEMENT POLICY: Replace from T1 if larger than target; else from T2– During Replacement in T1, if “1” page is found, make “0” and
move to T2 tail, move evicted page to B1– During Replacement in T2, if “1” page is found, make “0”, move
evicted page to B2DIRECTORY REPLACEMENT POLICY: Replace from B1 if T1+B1 = c; else from B2
CAR: Bansal & Modha
© 2003 IBM CorporationFAST 2004, San Francisco, CA April 2, 2004
CLOCK
HIT: Set the reference bit to “1”
MISS: Insert at the TAIL, initialize the reference bit to “0”
REPLACEMENT POLICY:– Evict the first “0” page – Reset “1” pages to “0”—giving
them a “second chance”
KEY INSIGHT:– Reseting “1” to “0” is “delayed
MRU”—removing it from hit path to miss path
0
0
1
0
100
0
1
1
0
0
1
0
1
01
1 10
1
1
0
1
TAIL
HE AD
CLOCK
Top Related