Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks
Felix Zahn, Pedro Yebenes, Steffen Lammel, Pedro J. Garcia, Holger Fröning
2nd IEEE International Workshop on High-Performance
Interconnection Networks in the Exascale and Big-Data Era
Barcelona, 12th March, 2016
Does network power ma.er at all?
§ Pitfall: don’t make assumptions based on maximum power ratings • At TDP, processors
outshine anything • But are processors always
operating at 100% load? • Energy-proportional: at
x% load, a component consumes only x% power
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 1
It does!
§ System power • Scalable energy-efficient network • Direct network, integrated switches
§ Dynamic range of components § Many memory-bound applications
• E.g., emerging integer applications (R. Murphy, Sandia) & graph computations • DFS & BFS • Connected Components • Isomorphism • Shortest Path • Graph Partitioning • BLAST (alignment search) • zChaff (satisfiability)
§ Exception: compute-bound applications with perfect overlap
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 2
Mo5va5on
§ Serialization technology dominates power consumption • Clock recovery, high frequency, equalization, pre-emphasis, …
§ It is link width that matters, not frequency • CML = Current Mode Logic • Linear scaling for 10GHz case • Frequency dependent part is CMOS only
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 3
71%$
14%$
15%$
Power&share&for&NIC&with&integrated&switch&
Links$(6)$
PCIe$
Core$
0.0#
0.5#
1.0#
1.5#
2.0#
2.5#
3.0#
3.5#
4#Lanes# 8#Lanes# 12#Lanes#
Power&con
sump-
on&[n
ormalized
]&
Link&power&scaling&
2.5#GHz#
5#GHz#
10#GHz#
Energy-‐propor5onality?
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 4
Energy-‐propor5onality vs. today’s interconnects
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 5
Energy-‐propor5onality – today’s possibili5es
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 6
§ Energy/data: 34.4 pJ/bit (65nm TSMC-produced Serializers only)
Concept
§ Application-dependent potential for energy saving • Best case: no transition time, links switched on/off depending on whether they are busy or idling
• Worst case (common case today): all links run with full power
§ Setup: OMNeT++-based simulator
• 3D Torus topology • 64 nodes • XYZ-dimension-order routing
7 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
Workloads -‐ NAMD
§ Satellite Tobacco Mosaic Virus (STMV), 64 MPI tasks
8 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
Workloads – Graph500
§ scale factor = 12, edge factor = 16, 64 MPI tasks
9 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
Results – Link inac5vity
10 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
§ Execution time: NAMD 3449 ms, Graph500 95.2 ms
Results – U5liza5on
§ Link utilization is highly volatile due to: • Application • Dimension • Routing algorithm
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 11
Results – Idle 5me
12 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
6% 2%
Results – Power saving poten5als
13 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
20% 24%
Discussion
§ Network power matters • Today’s interconnection networks contribute up to 20% to the total power consumption of large computing systems
§ Interconnects are highly energy dis-proportional • But they provide the opportunity to implement power saving strategies
§ Huge potential power savings • An optimal and fully energy-proportional NIC would save about 75% of the network power
14 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
Outlook
§ Understanding energy consumption • Power saving possibilities depend highly on workloads • Communication for HPC applications is complex to model => trace based network simulation
§ Analysis of different hardware parameter • Transition time, granularity of link width, etc. => Useful input on design decisions for future hardware § Network energy model
• Predicting energy consumed by the network based on communication characteristics
Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016 15
Thank you! Questions?
16 Analyzing the Energy (Dis-)Proportionality of Scalable Interconnection Networks - Felix Zahn - HiPINEB workshop - 12th March, 2016
Credits Discussions: Maximilian Thürmer, Markus Müller, Benjamin Klenk, Alexander Matz, Daniel Schlegel
(Heidelberg University), Sébas5en Rumley (Columbia University), Francisco Andujar, Jesus Escudero, Juan
Villar (Universidad de Cas5lla-‐La Mancha)
Current main interac4ons
Top Related