1
Line Distillation: Increasing Cache Capacity by
Filtering Unused Words in Cache Lines
Moinuddin K. Qureshi M. Aater Suleman
Yale N. Patt
HPCA 2007
2
Introduction
Caches are organized at linesize granularity Helps when spatial locality is high
Unused words when spatial locality is low
Unused words occupy space without contributing to cache hits
Filtering unused words allows cache to store more cache lines
3
Problem: Not all words are useful
On average less than 60% words used (4.7/8)
Cache line (64B) divided into 8 words of 8B each(1 MB 8-way L2 cache)
Word
s u
sed
per
line (
avg
)
4
Goal: Improving cache performance
Smaller linesize can result in fewer unused words
Smaller linesize degrades cache performance
Linesize of 32B increases MPKI for 14 of 16 benchmarksAverage MPKI increases by 25%
Insight:Words usage stabilizes as line traverses from MRU to LRU
Goal: Improving cache performance by filtering unused words
5
Insight
Footprint = 8-bits per line that tracks word usage
Most footprint updates occurearly in recency stack
Max recency position before footprint update
78%
5%
6%
11%
MRUPos 1Pos 2Pos 3Pos 4Pos 5Pos 6LRU
Recency Stack
Line Distillation (LDIS):Evict unused words when
line crosses certain recency
6
Outline
Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary
7
Framework for LDIS
PROCESSOR
ICACHE DCACHE
footprint
LOC WOC
L2 Cache
Distill Cache
valid bits
(sectored)
Line Organized Cache Word Organized Cache
Line from memory
8
Distill Cache (Operation)
Traditional cache (4-way)
LOC WOC
MRU LRU
B AC
Four cases:1. Cache Miss: Access to line D2. LOC Hit: Access to line B3. WOC Hit: Access to line A (word A0)4. Hole Miss: Access to line A (word
A1)Words used? Evict
A[1:6]Install A0,A7
(A0,A7 used)
Install Line D in LOC and update LRU state
Same as traditional cache
Send A0 and A7 to L1 and valid bitsInvalidate all words of A in WOC.
Fetch A from Memory and install in LOC
DA0,A7
9
Median Threshold Filtering
A line with many used words can evict several lines from WOC
A0 B0 C0 D0 E0 F0 G0 H0Line X has all 8 words used
X0 X1 X2 X3 X4 X5 X6 X78 Lines
evicted from WOC
WOC
Increase lines in WOC by not installing lines for which used words > threshold “K”
K = median words used in LOC line (computed at runtime)
10
Outline
Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary
11
Methodology
Configuration:
L2 cache: 1MB 8-way 64B linesize
(Distill cache gives 6 ways to LOC and 2 ways to WOC)
Out-of-order processor with 16KB 2-way L1s
400 cycle memory
Benchmarks:
15 SPEC2K benchmarks + health from olden suite
(A 250M instruction slice using SimPoint for SPEC2K)
12
ResultsLDIS (No MT) LDIS (with MT)
LDIS (MT) reduces MPKI by 25%
(%)
Reduct
ion
in L
2 M
PK
I
13
Reverter Circuit (RC)
Tournament selection: Distill cache vs. traditional cache Dynamic set sampling with 32 sets [Qureshi+ ISCA’06]
For sets A, C, D, F, H:if (SCTR > 75%) Enable LDISif (SCTR < 25%) Disable LDIS
ATD-LRU
Distill cache
Set B
Set E
Set G
Set A
Set CSet D
Set F
Set H
Set B
Set E
Set G
Set A
Set CSet D
Set F
Set H
Set BSet ESet G
SCTR
- +
(storage overhead of ATD: 1KB)
14
Results with RCLDIS (MT, No RC) LDIS (MT,RC)
RC disables LDIS when it increases MPKI.
LDIS (MT,RC) reduces MPKI by 30%
(%)
Reduct
ion
in L
2 M
PK
I
15
Overheads
Storage Tags for WOC + footprint bits: 12.2%
overhead
LatencyTag-access (LOC+WOC) increases by one
cycle WOC hits incur two cycles to rearrange words
PowerAdditional power of WOC tag-store
16
IPC Results
LDIS improves average IPC by 12%
(%)
IPC
Im
pro
vem
en
t
17
Outline
Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary
18
Compression vs. LDIS
Several proposals to increase capacity via compression
Compression and LDIS fundamentally different Compression exploits redundancy in stored data LDIS leverages unused words for spare capacity
Footprint Aware Compression (FAC) combines both
FAC compresses used words before installing in WOC
19
Results for FAC
Compression and LDIS interact positively.
FAC reduces MPKI by 50%
LDIS Compression FAC
(%)
Reduct
ion
in L
2 M
PK
I
50
40
30
20
10
0
20
Outline
Background Line Distillation Experimental Evaluation Interaction with Compression Related Work and Summary
21
Related work
Spatial-Temporal Cache -Gonzales+ [ICS’95]
Spatial Locality Prediction –Johnson+ [ISCA’97]
Variable Linesize Cache –Veidenbaum+ [ICS’99]
Spatial Footprint Prediction –Kumar+ [ISCA’98], Pujara+ [HPCA’06]
Spatial Pattern Prediction -Chen+ [HPCA’05]
LDIS is particularly suited for large caches and outperforms predictor-based techniques without
requiring separate structure for tracking spatial footprint
22
Contributions
Line Distillation: Filter unused words without a separate footprint predictor
Distill cache: Utilize extra capacity created by LDIS
Median Threshold Filtering and Reverter Circuit: Improve performance and robustness of LDIS Result: LDIS (MT+RC) reduces MPKI by 30%
Footprint Aware Compression: LDIS + compressionResult: FAC reduces MPKI by 50%
23
Questions
24
Result comparing capacity
25
Line Size vs. MPKI
26
Distribution of Hit-Miss
27
Average words usage (detailed)
28
Result for 3 types of LDIS
29
Replacement
LRU in LOC
WOC needs variable sized replacement
Only power-of-two sizes allowed in WOC
Placement constrained to alignment boundary
Random selection in case of multiple candidates
30
Background (pictorial)
31
Result LDIS vs. FAC (detailed)
32
Comparison with SFP
33
Appendix A: Other SPEC Benchmarks
34
Appendix B: Cache Size vs. Density
35
Summary
Many words in cache lines remain unused
Unused words unlikely to be accessed in less recent part of LRU stack Line Distillation (LDIS)
Distill-cache utilizes extra capacity created by LDIS
LDIS reduces MPKI by 30% and improves IPC by 12%
“Footprint Aware Compression” combines LDIS and compression to reduce MPKI by 50%
Top Related