Work Stealing for Irregular Parallel Applications on Computational Grids
Vladimir JanjicUniversity of St Andrews
12th December 2011
November 21, 2006
In this talk…
Feudal Stealing algorithm for scheduling irregular parallel applications
Combination of Grid-GUM and Cluster-aware Random Stealing
Irregular parallel applications -- task trees highly unbalanced
November 21, 2006
What is work stealing?
Work Stealing -- passive, distributed, dynamic scheduling method
Idle “thieves” steal work from busy “victims”
November 21, 2006
To steal or not to steal?
Why stealing? Dynamic, adaptive and “cheap” Does not require prior knowledge of task dependencies =>
good for irregular applications Inherently distributed => scalability
Why not stealing? Not optimal Possibly slow work distribution
November 21, 2006
Work stealing on Computational Grids
Grid-GUM (GpH), Satin (Java d&c), Javelin (Java), Atlas (Java)
The main problem : Steal attempts can be expensive due to high latencies
Especially for irregular applications, where all work may be concentrated on a few nodes
The main questions are where to send steal attempts and how to respond to them
Use load information (Grid-GUM)
November 21, 2006
Cluster-aware Random Stealing (CRS)
Local (within a cluster) and remote (outside of cluster) stealing done in parallel
Works well for regular applications on heterogeneous environments with a lot of parallelism
Not so well for irregular
November 21, 2006
Feudal Stealing
Use the CRS algorithm as a base
Local stealing done using Random Stealing
Remote stealing done via cluster head nodes Only head nodes (and a victim) visited Head nodes hold load information
3 2 1
4
7
5
8 9
6
Cluster 0
Local load Remote LoadPE Load Cl Time Load 2 2 1 1000 23 3 3 2 0 0 4 0 3 2000 0 5 7 6 0 7 0 8 1 9 5
November 21, 2006
How is load information in head nodes obtained?
Load of nodes inside the cluster periodically sent Load of remote clusters updated from remote-steal messages
Cluster load information attached to remote-steal messages (similar to Grid-GUM)
November 21, 2006
Evaluation of Feudal Work Stealing
Using simulations, on generic benchmarks for load balancing algorithms (UTS -- Unbalanced Tree Search)
For regular and less-irregular applications, performs as well as CRS and better than Grid-GUM
For highly-irregular applications, better than CRS and Grid-GUM
November 21, 2006
Comparison of Feudal Stealing, CRS and Grid-GUM
QuickTime™ and a decompressor
are needed to see this picture.
November 21, 2006
Conclusions
Feudal Stealing works well for irregular parallel applications on Computational Grids
Sacrifices some desirable features of “pure” work stealing in order to make better selection of remote targets
Tested only using simulations. Implementation in Grid-GUM under way
Tested only on artificial applications (unbalanced tree search)
November 21, 2006
More info
Vladimir Janjic, Load Balancing of Irregular Parallel Applications On Heterogeneous Distributed Computing Environments, PhD Thesis, University of St Andrews, 2011
Vladimir Janjic, Kevin Hammond, Think Locally Steal Globally : Using Dynamic Load Information in Work-Stealing on Computational Grids, Submitted to CCGrid 2012
Vladimir Janjic, Kevin Hammond, Feudal Work-Stealing, In preparration, planned for submission to EuroPar 2012
Top Related