Dynamic Loop Caching Meets Preloaded Loop Caching – A

Hybrid Approach

Dynamic Loop Caching Meets Preloaded Loop Caching – A

Hybrid Approach

Ann Gordon-Ross and Frank Vahid*Department of Computer Science and Engineering

University of California, Riverside*Also with the Center for Embedded Computer Systems, UC Irvine

This work was supported in part by the U.S. National Science Foundation and a U.S. Dept. of Education GAANN Fellowship

International Conference on Computer Design, 2002

2

IntroductionIntroduction

• Memory access can consume 50% of an embedded microprocessor’s system power– Instruction fetching usually

more than half of that power– Caches tend to be power

hungry

• ARM920T: caches consume half of total power (Segars 01)

• M*CORE: unified cache consumes half of total power (Lee/Moyer/Arends 99)

ARM920T. Source: Segars ISSCC’01

I-Mem

L1 Cache

Processor

3

Filter CacheFilter Cache

• Tiny L0 cache (~64 instruct.)– Kin/Gupta/Mangione-

Smith97 – Has very low dynamic

power• Short internal bitlines • Close to microprocessor

• Power/energy savings, but:– Performance penalty of

21% due to high miss rate (Kin’97)

– Tag comparisons consume power

L1 Cache

Filter Cache (L0)

Processor

4

Dynamically Loaded Tagless Loop CacheDynamically Loaded Tagless Loop Cache• Tiny cache that passively

fills with loops as they execute (Lee/Moyer/Arends 99)

• Not really first level of memory– Rather, an alternative

• Operation– Filled when short backwards

branch detected in instruction stream

• Compared to filter cache...– No tags – even lower power– Missless – no performance

penalty

L1 Cache

Dynamic Loop Cache

Mux

Processor

5

Dynamically Loaded Tagless Loop CacheDynamically Loaded Tagless Loop Cache

L1 Cache

Dynamic Loop Cache

Mux

Processor

…

mov r1, 2

…

sbb -2

• Tiny cache that passively fills with loops as they execute (Lee/Moyer/Arends 99)

• Not really first level of memory– Rather, an alternative

• Operation– Filled when short backwards

branch detected in instruction stream

• Compared to filter cache...– No tags – even lower power– Missless – no performance

penalty

6

Dynamically Loaded Tagless Loop CacheDynamically Loaded Tagless Loop Cache

L1 Cache

Dynamic Loop Cache

Mux

Processor

…

mov r1, 2

…

sbb -2

• Tiny cache that passively fills with loops as they execute (Lee/Moyer/Arends 99)

• Not really first level of memory– Rather, an alternative

• Operation– Filled when short backwards

branch detected in instruction stream

• Compared to filter cache...– No tags – even lower power– Missless – no performance

penalty

7

Dynamically Loaded Tagless Loop CacheDynamically Loaded Tagless Loop Cache

L1 Cache

Dynamic Loop Cache

Mux

Processor

…

mov r1, 2

…

sbb -2

• Tiny cache that passively fills with loops as they execute (Lee/Moyer/Arends 99)

• Not really first level of memory– Rather, an alternative

• Operation– Filled when short backwards

branch detected in instruction stream

• Compared to filter cache...– No tags – even lower power– Missless – no performance

penalty

8

Dynamically Loaded Tagless Loop Cache – ResultsDynamically Loaded Tagless Loop Cache – Results• We ran 10 Powerstone

benchmarks (from Motorola) on a MIPS processor instruction-set simulator– Average L1 fetch reduction

was 30%– Closely matched results of

[Lee et al 99].– L1 fetch reductions

translate to system power savings of 10-15%

L1 Cache

Dynamic Loop Cache

Mux

Processor

9

Dynamically Loaded Tagless Loop Cache - LimitationDynamically Loaded Tagless Loop Cache - Limitation• Does not support loops

with control of flow changes (cof)– Only supports sequential

instruction fetching since it was filled passively during a loop iteration

• Does not see instructions not executed on that iteration

– A cof thus terminates loop cache filling or fetching

– cof’s unfortunately include common if-then-else statements within a loop

L1 Cache

Dynamic Loop Cache

Mux

Processor

…

mov r1, 2

mov r2, 3

bne r1, r2, 2

…

sbb -4

…

10

Dynamically Loaded Tagless Loop Cache - LimitationDynamically Loaded Tagless Loop Cache - Limitation• Does not support loops

with control of flow changes (cof)– Only supports sequential

instruction fetching since it was filled passively during a loop iteration

• Does not see instructions not executed on that iteration

– A cof thus terminates loop cache filling or fetching

– cof’s unfortunately include common if-then-else statements within a loop

L1 Cache

Dynamic Loop Cache

Mux

Processor

…

mov r1, 2

mov r2, 3

bne r1, r2, 2

…

sbb -4

…

11

Dynamically Loaded Tagless Loop Cache - LimitationDynamically Loaded Tagless Loop Cache - Limitation

L1 Cache

Dynamic Loop Cache

Mux

Processor

• Does not support loops with control of flow changes (cof)– Only supports sequential

instruction fetching since it was filled passively during a loop iteration

• Does not see instructions not executed on that iteration

– A cof thus terminates loop cache filling or fetching

– cof’s unfortunately include common if-then-else statements within a loop

…

mov r1, 2

mov r2, 3

bne r1, r2, 2

…

sbb -4

…

12

Dynamically Loaded Tagless Loop Cache - LimitationDynamically Loaded Tagless Loop Cache - Limitation

• Lack of support of cof’s results in support of only half of small frequent loops in the benchmarks

13

Preloaded Tagless Loop CachePreloaded Tagless Loop Cache

• Embedded systems typically execute a fixed application– Can determine critical

loops/subroutines through profiling

– Can preload critical regions into loop cache – whose contents will not change

• Preloaded loop cache (Gordon-Ross/Cotterell/Vahid CAL’02)– Tagless, missless– Supports more loops

than dynamic loop cache

L1 Cache

Preloaded Loop Cache

Mux

Processor

Dmem.Processor

Periph. Pmem.

…

mov r1, 2

mov r2, 3

bne r1, r2, 2

…

sbb -4

…

14

Preloaded Tagless Loop CachePreloaded Tagless Loop Cache

L1 Cache

Preloaded Loop Cache

Mux

Processor

• Embedded systems typically execute a fixed application– Can determine critical

loops/subroutines through profiling

– Can preload critical regions into loop cache – whose contents will not change

• Preloaded loop cache (Gordon-Ross/Cotterell/Vahid CAL’02)– Tagless, missless– Supports more loops

than dynamic loop cache

Dmem.Processor

Periph. Pmem.

…

mov r1, 2

mov r2, 3

bne r1, r2, 2

…

sbb -4

…

15

Preloaded Tagless Loop Cache - ResultsPreloaded Tagless Loop Cache - Results

• Results– 128 instruction

preloaded loop cache reduces L1 fetches by nearly twice that of dynamic (30%) for the benchmarks studied (Powerstone and Mediabench)

16

Preloaded Tagless Loop Cache - DisadvantagesPreloaded Tagless Loop Cache - Disadvantages

• Preloaded loop cache has some limitations too– Occasionally dynamic

loop cache is actually better

– Preloading also requires• Fixed application• Profiling

– Limited number of loops can be preloaded

– We really want both!

Instruction fetch power savings

17

Solution: A Hybrid Loop CacheSolution: A Hybrid Loop Cache

• Functions as both a dynamic and a preloaded loop cache

• 2 levels of cache storage– Main Loop Cache – for

instruction fetching– 2nd Level Storage -

preloaded loops are stored here

L1 Cache

Microprocessor

Main LoopCache

2nd Level Storage

Mux

Loop Cache Controller

Preloaded Loop Filler

Loop Match

Memory

Addr

Data

Addr

Data

Addr

Data

Control signals

Control

Control Control

LARs

18

Hybrid Loop Cache - FunctionalityHybrid Loop Cache - Functionality

• Dynamic Loop Cache functionality– On a short backwards branch, main loop

cache is filled dynamically

• Preloaded Loop Cache functionality– On a cof, if the next instruction falls within

a preloaded region of code, that region is filled into the main loop cache from 2nd level storage

– After being filled, instructions can be fetched from the main loop cache

19

Hybrid Loop Cache - ResultsHybrid Loop Cache - Results

• Hybrid performance – Best in 9 out of 13

benchmarks– Equally well in 1– In the remaining 3,

the hybrid loop cache performed nearly as good or better the strictly dynamic approach but was outperformed by the preloaded approach

Instruction fetch power savings

20

Hybrid Loop Cache – Additional ConsiderationHybrid Loop Cache – Additional Consideration

• Hybrid loop cache can behave like a dynamic loop cache – If designer does not

wish to profile/preload– Power savings are

almost identical to the dynamic loop cache when no loops are preloaded

21

ConclusionsConclusions

• Hybrid loop cache reduced embedded system instruction fetch power by average of 51%– 90% savings in several cases– Outperformed dynamic and preloaded loop caches

• Dynamic 23%, Preloaded 35%, Hybrid 51%

• Can work as dynamic, preloaded, or both– More capacity than a preloaded loop cache– Can be used transparently as dynamic loop cache

• With nearly identical results

• Hybrid loop cache may be a good addition to low power embedded microprocessor architectures