LeadOut: Composing Low-Overhead Techniques...

Universityof Illinois

http://iacoma.cs.uiuc.edu/

LeadOut: Composing Low-Overhead Techniques for Single-Thread Performance

Brian Greskamp, Ulya Karpuzcu, Josep Torrellas

http://iacoma.cs.uiuc.edu


100

1000

10000

100000

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

Year

100

1K

10K

100KTo

p SP

ECin

t92

Rat

e

Ulya Karpuzcu LeadOut

Per-Thread Performance Trend

2

100

1000

10000

100000

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

Year

100

1K

10K

100KTo

p SP

ECin

t92

Rat

e



2

Historical

100

1000

10000

100000

19

93

19

94

19

95

19

96

19

97

19

98

19

99

20

00

20

01

20

02

20

03

20

04

20

05

20

06

20

07

20

08

20

09

20

10

20

11

20

12

Year

100

1K

10K

100KTo

p SP

ECin

t92

Rat

e



2

Historical

Contemporary


Near-Future CMP Environment

3



3

• Sequential applications need fast cores



3


• Throughput applications demand more cores



3



• Amdahl’s Law: Most applications need some fast cores



3




• Faster cores without compromising core count?



3





• Configurable Timing Speculation



3





• Configurable Timing Speculation

• V/f Boosting


Timing Speculation (TS)

4



4

Boost core frequency beyond nominal at constant supply voltage


Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4



Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4

Rat

ed

Rat

ed



Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4

TSTS

• Increase f at constant V

Rat

ed

Rat

ed



Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4

TSTS

• Increase f at constant V

Rat

ed

Rat

edTiming errors



Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4

TSTS

• Increase f at constant V• Support for error detection and correction

Rat

ed

Rat

edTiming errors



Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4

Gain

TSTS


Rat

ed

Rat

edTiming errors



Err

or

Rate

(P

E)

FreqP

erf

orm

an

ce

Freq


4

Gain

TSTS


Rat

ed

Rat

edTiming errors


Assuming a high P/T headroom, PE limits performance gain

Ulya Karpuzcu LeadOut 5

Voltage-Frequency Boosting


Voltage-Frequency BoostingBoost core frequency beyond nominal by increasing V


Voltage-Frequency BoostingBoost core frequency beyond nominal by increasing V No timing errors (PE = 0)



FreqPe

rfor

man

ceP E

Freq



FreqPe

rfor

man

ceP E

Freq

Rated



FreqPe

rfor

man

ceP E

Freq

Rated V/f Boosting



FreqPe

rfor

man

ceP E

Freq

Rated V/f Boosting

Gai

n



FreqPe

rfor

man

ceP E

Freq

Rated V/f Boosting

Gai

n

Assuming a high P/T headroom, V limits performance gain


Contribution: LeadOut


• Individual application of TS or V/f Boosting is suboptimal




• Unable to bring the multicore up to its P/T envelope





• Available P/T headroom remains untapped






• TS and V/f boosting are complementary







• Bounded by different constraints








• LeadOut








• LeadOut

• Synergistically combine TS and V/f Boosting








• LeadOut

• Synergistically combine TS and V/f Boosting

• Speed-ups for single thread performance multiply



Example TS Architecture [PACT07]: Paceline

7

Many-Core CMP


OS sees: One core

8

CheckerLeader



Critical ThreadOS sees: One core

8

CheckerLeader



Leader Checker



Leader CheckerSpeculative Clock Rated ClockExample TS Architecture [PACT07]: Paceline


Leader CheckerSpeculative Clock Rated Clock

=?




=?=?




=?=?=?


Rated Clock Rated Clock

=?

Thread 0 Thread 1

OS sees: Two cores




Cor

e 0

Example V/f Boosting: Intel Turbo Boost


Nominal

Freq

uenc

y

Cor

e 0

Cor

e 1

Cor

e 2

Cor

e 3

Cor

e 0



Nominal

Freq

uenc

y

Cor

e 0

Cor

e 1

Cor

e 2

Cor

e 3

Lightly Threaded Workload

Turbo

Cor

e 0



Nominal

Freq

uenc

y

Cor

e 0

Cor

e 1

Cor

e 2

Cor

e 3


Turbo

Cor

e 0

Freq

uenc

y

Cor

e 0

Cor

e 1

Cor

e 2

Cor

e 3



Nominal

Freq

uenc

y

Cor

e 0

Cor

e 1

Cor

e 2

Cor

e 3


Turbo

Cor

e 0

Freq

uenc

y

Cor

e 0

Cor

e 1

Cor

e 2

Cor

e 3Inactive cores

power-gated



Composing the Techniques

12



12

The techniques are orthogonal



• Suppose as much P/T headroom as necessary

12





• Paceline: Performance gain constrained by PEMAX

12





• Paceline: Performance gain constrained by PEMAX

• V/f Boosting: Performance gain constrained by VMAX

12



V/f Boosting + Paceline


Boost core frequency beyond nominal by increasing V; tolerating occasional timing errors


FreqPe

rfor

man

ceP E

Freq

RatedV/f Boosting




FreqPe

rfor

man

ceP E

Freq

RatedV/f Boosting





Gai

n


Multicore Loading

Condition

Bounding ConstraintsBounding ConstraintsBounding ConstraintsBounding ConstraintsBounding ConstraintsBounding Constraints Gain from combining

?

Multicore Loading

ConditionV/f BoostV/f Boost PacelinePaceline Compos.Compos.

Gain from combining

?

Multicore Loading

Condition T/P V T/P PE T/P V/PE

Gain from combining

?

Very High ✓ ✓ ✓ Unlikely

High to Moderate ✓ ✓ ✓ Likely

Low ✓ ✓ ✓ Definitely



Multicore Loading

Condition


?

Multicore Loading


Gain from combining

?

Multicore Loading


Gain from combining

?






Multicore Loading

Condition


?

Multicore Loading


Gain from combining

?

Multicore Loading


Gain from combining

?





Additional techniques can be applied


LeadOut Operation

18


LeadOut Operation

18

• At any given time, CMP executes a mix of speed-critical and throughput-oriented threads


LeadOut Operation

18


• Run throughput threads unoptimized


LeadOut Operation

18


• Run throughput threads unoptimized• Optimization Problem


LeadOut Operation

18



1. Which technique to use for a speed-critical thread?


LeadOut Operation

18



1. Which technique to use for a speed-critical thread?2. How to optimally set V/f for chosen technique?


LeadOut Evaluation

19


LeadOut Evaluation

19

• Simulated 32nm CMP with 16 cores


LeadOut Evaluation

19

• Simulated 32nm CMP with 16 cores • Detailed modeling of leakage, temperature, variation


LeadOut Evaluation

19

• Simulated 32nm CMP with 16 cores • Detailed modeling of leakage, temperature, variation • Applications: SPECint2000 benchmarks


LeadOut Evaluation

19

• Simulated 32nm CMP with 16 cores • Detailed modeling of leakage, temperature, variation • Applications: SPECint2000 benchmarks• 50 Monte Carlo die samples


Power-Performance TradeoffPe

r Cor

e Po

wer

(W)

Performance

Increasing Load4

6

8

10

12

11

13

14

1 1.1 1.2 1.3 1.4

Unoptimized

Paceline

V/f-Boost

V/f-Boost + Paceline



r Cor

e Po

wer

(W)

Performance

Increasing Load4

6

8

10

12

11

13

14

1 1.1 1.2 1.3 1.4

Unoptimized

Paceline

V/f-Boost




r Cor

e Po

wer

(W)

Performance

Increasing Load4

6

8

10

12

11

13

14

1 1.1 1.2 1.3 1.4

Unoptimized

Paceline

V/f-Boost


~20% speed-up at most at 2x power cost



r Cor

e Po

wer

(W)

Performance

Increasing Load4

6

8

10

12

11

13

14

1 1.1 1.2 1.3 1.4

Unoptimized

Paceline

V/f-Boost




r Cor

e Po

wer

(W)

Performance

Increasing Load4

6

8

10

12

11

13

14

1 1.1 1.2 1.3 1.4

Unoptimized

Paceline

V/f-Boost


20% speed-up at most at 2x power cost

38% speed-up at most at 3x power cost



r Cor

e Po

wer

(W)

Performance

Increasing Load4

6

8

10

12

11

13

14

1 1.1 1.2 1.3 1.4

Unoptimized

Paceline

V/f-Boost




Also in the Paper


• Detailed analysis for different load conditions

26

Also in the Paper



• Including V/f Boosting with activity migration

26

Also in the Paper




• Sensitivity analysis

26

Also in the Paper





• Thermal design points, power grid designs, guardband

26

Also in the Paper






• A hierarchical controller design to dynamically set

26

Also in the Paper







• Technique to apply

26

Also in the Paper







• Technique to apply

• Per core V/f assignment for the chosen technique

26

Also in the Paper


Conclusion


• Two low overhead techniques for sequential acceleration

27

Conclusion



• V/f Boosting and Timing Speculation

27

Conclusion




• Individual application of any technique suboptimal

27

Conclusion





• Techniques are complementary

27

Conclusion






• LeadOut: A highly-configurable CMP

27

Conclusion







• Combining V/f Boosting and TS synergistically

27

Conclusion







• Combining V/f Boosting and TS synergistically

• 34% thread speedup at 220% power increase

27

Conclusion

Universityof Illinois

http://iacoma.cs.uiuc.edu/

LeadOut: Composing Low-Overhead Techniques for Single-Thread Performance

Brian Greskamp, Ulya Karpuzcu, Josep Torrellas



LeadOut: Composing Low-Overhead Techniques...

Documents

Transcript of LeadOut: Composing Low-Overhead Techniques...