ACEMD: High-throughput Molecular Dynamics with NVIDIA Kepler GPUs
Reverse engineering power management on NVIDIA GPUs - A ... · Reverse engineering power management...
29
Reverse engineering power management on NVIDIA GPUs A detailed overview Martin Peres Ph.D. student at LaBRI, Bordeaux September 25, 2013
Transcript of Reverse engineering power management on NVIDIA GPUs - A ... · Reverse engineering power management...
Reverse engineering power management onNVIDIA GPUsA detailed overview
Martin Peres
Ph.D. student at LaBRI, Bordeaux
September 25, 2013
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Summary
1 Introduction
2 General overview of ways to save power
3 PCOUNTER
4 PTHERM
5 PDAEMON
6 Conclusion
1 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Introduction – Motivation
Power management in computers, why?
To lower the power consumption of Data Centers;
To increase the battery life of mobile computers;
To have quieter and slimmer devices.
Reverse engineering power management, why?
Power management is:
at least partially-assisted by software;
almost entirely non-documented;
often considered to be a manufacturer secret;
thus poorly studied/implemented in open drivers;
this is especially true in the GPU world.
2 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Summary
1 Introduction
2 General overview of ways to save powerOrigin of the power consumptionUsual ways of saving powerAreas of application
3 PCOUNTER
4 PTHERM
5 PDAEMON
6 Conclusion
3 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Origin of the power consumption
Power consumption of a logic gate
P = Pstatic + Pdynamic
Pstatic : Small transistors leak current even when “blocked”
Pstatic = V ∗ IleakIleak depends on the voltage and the etching of the transistors.
Pdynamic : Fighting the gate capacitance when switching
Pdynamic = CfV 2;
C : Capacitance of the gate (fixed);
f : Frequency at which the gate is switched;
V : Voltage at which the gate is powered.
4 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
The dynamic and static power cost
(a) Total chip dynamic and staticpower dissipation trends based onthe International TechnologyRoadmap for Semiconductors(2003).
(b) Source:http://chipdesignmag.com/
display.php?articleId=3310
(2009)
5 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Usual ways of saving power
Usual ways of saving power
Clock gating: Cuts the dynamic-power cost;
Power gating: Cuts all the power cost;
Reclocking: Adjusts the clock frequency and voltage.
Clock gating: Stopping the clock of un-used gates
Update rate: Every clock cycle;
Effectiveness: Cuts the dynamic-power cost entirely;
Drawbacks: Increase of the complexity of the clock tree;
Executed by: Hardware.
6 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Usual ways of saving power
Power gating: Shutting down the power of un-used gates
Update rate: Around a microsecond;
Effectiveness: Cuts the power cost entirely;
Drawbacks: May need to save the context before shutdown;
Executed by: Hardware and/or software.
Reclocking: Dynamic Voltage/Frequency Scaling (DVFS)
Update rate: Around a millisecond;
Effectiveness: Impacts the static- and dynamic-power cost;
Drawbacks: Affects performance;
Executed by: Software.
7 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Optimal DVFS policy
Optimal DVFS policy to stay in the power budget
Find the bottleneck using performance counters;
Lower the clocks of all the other clock domains;
Lower the voltage of the power domains based on clocks;
Increase the clock of the bottleneck clock domain;
Repeat and learn about application patterns.
Constraints
finding the bottleneck fast-enough;
predicting the needed-voltage based on clocks’ frequencies;
calculating the memory timmings on-the-fly;
supporting any combinaison of clocks.
8 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
A simple clock domain’s clock tree
MUX_0
DOM6SPLL NVPLL
MUX_1
P_DIVIDER
MUX_2
NVCLK
HOST_REFCRYSTAL
RPLL
Figure : Clock tree for the core clock domain on nv849 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Usual ways of saving power
Places to apply the proposed solutions
card-level power gating (optimus);
internal engines;
VGA DACs;
PCIe port (ASPM);
anything using a clock and being part of a power domain.
10 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PCIe ASPM impact
Figure : Maximum power consumption of the PCIe port at various linkconfigurations.
11 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Summary
1 Introduction
2 General overview of ways to save power
3 PCOUNTER
4 PTHERM
5 PDAEMON
6 Conclusion
12 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PCOUNTER – Overview
Performance counters
are blocks in modern processors that monitor their activity;
count hardware events such as cache hit/misses;
are tied to a clock domain;
provide load information needed for DVFS’s decision making.
Cycles
Events
Signal
Clock 4
1
Figure : Example of a simple performance counter
13 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PCOUNTER – Overview of a domain
Cycles
Events
Macrosignal
Clock X
XTruthTable
Multi-plexer
S0S1S3S4
Events
Macrosignal XTruth
TableMulti-plexer
S0S1S3S4
Events
Macrosignal XTruth
TableMulti-plexer
S0S1S3S4
Signals
Events
Macrosignal XTruth
TableMulti-plexer
S0S1S3S4
/256
/256
/256
/256
/256
Figure : Schematic view of a domain from PCOUNTER14 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PCOUNTER – Other counters?
MP counters
per-channel/process counters in PGRAPH;
same logic as PCOUNTER;
require running an opencl kernel to read them;
share some in-engine multiplexers with PCOUNTER.
PDAEMON
4 global counters;
very simplified logic;
usually about the business of the other engines.
15 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Counters – Which signals are known?
PCOUNTER signals
very chipset-dependent;
about 150 signals reverse engineered on nv50;
thanks to Marcin (mwk) and Samuel Pitoiset (GSoC 2013).
MP counters signals
all GPGPU signals exported by cupti on Fermi+ reversed;
thanks to Christoph Bumiller (calim) and Samuel Pitoiset.
PDAEMON’s signals
5 signals known;
thanks to Marcin Koscielnicki (mwk).
16 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Summary
1 Introduction
2 General overview of ways to save power
3 PCOUNTER
4 PTHERMThermal managementFSRMPower regulation
5 PDAEMON
6 Conclusion
17 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PTHERM – Thermal management
PTHERM’s thermal management
sends IRQs to the host when reaching temperature thresholds;
can cut the power of the card through a GPIO;
can force the fan to the maximum speed;
can lower the frequency of the main engine of the GPU(through FSRM).
18 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PTHERM – Frequency-Switching Ratio Modulation
Frequency-Switching Ratio Modulation (FSRM)
is used to lower the frequency of the main engine of the GPU;
is useful to lower the temperature or the power consumption;
is triggered automatically when reaching thresholds.
How can the FSRM lower power consumption?
A divided clock is generated from the main engine’s clock;
The clock must be divided by a power-of-two (2 to 16);
It can generate any clock frequency between these two clocks;
With a lower clock, an engine consumes less power.
19 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PTHERM – Frequency-Switching Ratio Modulation
0 50 100 150 200 2500
1
2
3
4
·108
Obtained
Expected
FSRM’s PWM duty cycle (8 bit)
Cor
efr
equ
ency
(Hz)
Figure : Frequency of the core clock (original @ 408MHz) when using a16-divider and varying the FSRM
20 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PTHERM – Power estimation
Calculating the power consumption
PTHERM estimates power consumption by:
reading every block’s activity (in use or not);
summing the weighted activity blocks signals;
applying a low pass filter.
Figure : Extract of NVIDIA’s patent on power estimation (US8060765)
21 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PTHERM – Power limitation
PTHERM’s power limitation can
read the power consumption by counting the active blocks;
update the FSRM ratio to stay in the power budget;
use two hysteresis windows for altering the FSRM ratio;
do all that automatically.
0 5 10 15 20 25
80
100
120
140
high 2
high 1
low 1
low 2
Refresh cycles
Inst
anta
neo
us
pow
er(W
)
100
150
200
250
FS
RM
’sP
WM
du
tycy
cle
(8b
it)
PowerFSRM
Figure : Example of the power limiter in the dual window mode22 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Power limitation – Actual implementation of NVIDIA
Power limitation – Actual implementation
NVIDIA doesn’t use PTHERM to implement power limitation;
It may read power consumption from the voltage controller;
and downclock the card when exceeding the budget.
Figure : Effect of disabling the power limiter on the Geforce GTX 580.Copyrights to W1zzard from techpowerup.com.
23 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Summary
1 Introduction
2 General overview of ways to save power
3 PCOUNTER
4 PTHERM
5 PDAEMON
6 Conclusion
24 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
PDAEMON – An embedded RTOS in your GPU
PDAEMON
is an RTOS embedded in every new NVIDIA GPU (Fermi+);
clocked at 200MHz and is programmed in the FµC ISA;
has access to all the registers of the card;
can catch all the interrupts from the GPU to the Host;
features internal performance counters.
NVIDIA’s usage of PDAEMON
Fan management;
Hardware scheduling (for memory reclocking);
Power gating and power budget enforcement;
Performance and system monitoring.
25 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Summary
1 Introduction
2 General overview of ways to save power
3 PCOUNTER
4 PTHERM
5 PDAEMON
6 ConclusionConclusion & Future work
26 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Conclusion
The GPU as an autonomic system
The GPU can:
self-configure: thanks to PDAEMON that can act as a driver;
self-optimise: using the performance counters;
self-heal: recovering from over-temperature/current;
self-protect: GPU users are isolated in separate VM.
Future works
Implement stable reclocking across all GPUs;
Write a test-bed for DVFS algorithms implementations;
Document clock- and power-gating details;
Reverse engineer more performance-counter signals.
27 / 28
Introduction General overview of ways to save power PCOUNTER PTHERM PDAEMON Conclusion
Questions & Discussions
Questions & Discussions
28 / 28
