Engineering a Scalable High Quality Graph...

1 April 29, 2010 Manuel Holtgrewe, Peter Sanders, Christian Schulz – Institute for Theoretical Computer Science, Algorithmics IIScalable High Quality Graph Partitioner

KIT – die Kooperation von Forschungszentrum Karlsruhe GmbH und Universitat Karlsruhe (TH)

Institute for Theoretical Computer Science, Algorithmics II

Engineering a Scalable High QualityGraph PartitionerManuel Holtgrewe, Peter Sanders,Christian Schulz


Balanced Graph Partitioning



Partition graph G = (V , E) into k disjoint blocks s.t.

total node weight of each block ≤ 1 + ε

ktotal node weight

total weight of cut edges as small as possible

Applications:finite element simulations, VLSI design, route planning, . . .

Multi-Level Graph Partitioning



Successful in existing systems:DiBaP, Chaco, Jostle, Metis, Scotch,. . .

Why Yet Another MGP Algorithm?Karlsruhe Parallel Partitioner



scalable parallel algorithm for the case k =# of processorslarge inputshigh qualityunderstand / engineer multi level methodbridge gaps theory↔ practice

Matching SelectionGoals:



1. large edge weights sparsify2. large #edges few levels3. uniform node weights “represent” input4. small node degrees “represent” input

unclear objective gap to approx. weighted matchingwhich only considers 1.,2.

Our Solution:Apply approx. weighted matching to general edge rating function

input graph

...

contract

match

Edge Ratings Tried



ω({u, v})

expansion({u, v}) :=ω({u, v})

c(u) + c(v)

expansion∗({u, v}) :=ω({u, v})c(u)c(v)

expansion∗2({u, v}) :=ω({u, v})2

c(u)c(v)

innerOuter({u, v}) :=ω({u, v})

Out(v) + Out(u)− 2ω(u, v)where c = node weight, ω =edge weight,Out(u) := ∑{u,v}∈E ω({u, v})

Approx. Weighted Matching



Use parallel prepartitioner (optional)currently geometric recursive bipartitioning when coords.availableGlobal Path Algorithm [MaueSanders 2007] locallyparallel [ManneBisseling 2007] for border nodes

Initial Partitioning



Every PE can perform initial partitioning using different seedsCompared PARTY, PMETIS, and SCOTCH

SCOTCH yielded best results

match

... ...local improvement

uncontractcontract

outputpartition

input graph

partitioning

initial

Local Improvement – Basic Approach



Use linear time local search [Fiduccia Mattheyses 82]on multiple pairs of blocks in parallel

input graph

...

initial

...

outputpartition

local improvement

partitioning

match

contract uncontract

Finding Pairs of Blocks



Color edges of quotient graph Qseveral parallel algorithms triedEach color→ matching in Q→ independent block pairs

Pairwise Local Search



two local searches

adopt best

exchange boundaries

New queue selection strategy topGain

Results – Example



Streetnetwork Europe |V | = 18M, |E | = 44M, k = 16,KaPPa↔ParMetis

Scalablity



k

tota

l ti

me

[s]

510

2050

100

200

500

4 8 16 32 64 128 256 512 1024

510

2050

100

200

500

4 8 16 32 64 128 256 512 1024

510

2050

100

200

500

4 8 16 32 64 128 256 512 1024

510

2050

100

200

500

4 8 16 32 64 128 256 512 1024

510

2050

100

200

500

4 8 16 32 64 128 256 512 1024

510

2050

100

200

500

4 8 16 32 64 128 256 512 1024

KaPPa−StrongKaPPa−FastKaPPa−Minimalscotchkmetisparmetis

Streetnetwork Europe (|V | = 18M, |E | = 44M)

QualityComparison with Other Systems



Geometric mean, imbalance ε = 0.03:14 graphs (78 136–18M nodes) ×k ∈ {2, 4, 8, 16, 64}

Variant avg. cut balance avg. time [s]KaPPa-Strong 24 227 1.028 36.93KaPPa-Fast +2% 1.028 21.40KaPPa-Minimal +3% 1.028 5.94seq. scotch +7% 1.027 5.95kmetis +18% 1.026 0.79parmetis +30% 1.041 0.59

Walshaw instances, road networks, Florida Sparse MatrixCollection, random Delaunay triangulations, random geometricgraphs, social networks.

Sources of Quality Improvement



1. edge ratings largest effect by far2. more localized refinement.

Surprise! previous parallelizations caused worse quality3. better queue selection4. better matching algorithms5. more time invested

Quality – Walshaw BenchmarkZero percent case



Graph 2 4 8 16 32 64data - 383 - - - -uk - 43 - - - -add32 - - - - - 493bcsstk29 - 8159 - - - -bcsstk32 - 9492 - - - -fetooth - 7142 11935 18382 - -ferotor - 7480 13292 - - -598a - 8154 - 26427 - -feocean - 1902 4299 - - -144 - 15804 26175 - 58462 -wave - 17475 30511 - - -m14b - 13391 - - - -auto - 28048 - - - -

Quality – Walshaw BenchmarkOne percent case



Graph 2 4 8 16 32 64data - 383 - - - -add32 - - - - - 493bcsstk29 - 8159 - - - -memplus - 9559 - - - -fepwt - 704 - - 5612 -bcsstk32 - 9492 - - - -febody - 636 - 1931 - -fetooth - - 11935 - - -ferotor - 7480 - - - -598a - 8154 16467 26427 40674 -feocean - - - 8432 13660 21548wave - 17475 30511 44611 64551 -m14b - 13391 26666 43975 - -auto - - 47650 79847 - -

Quality – Walshaw BenchmarkThree percent case



Graph 2 4 8 16 32 64add32 - - - - - 493wingnodal - 3561 - - - -bcsstk30 - 16537 34513 - - -bcsstk31 - 2676 7181 - - -fepwt - 704 1419 - - -bcsstk32 - 8799 21023 36613 - -febody - 601 1054 1800 2947 -brack2 - 2839 - - - -fetooth - 6946 11662 17760 - -ferotor 1963 7222 12852 - - -598a - 7955 - 25966 39829 -feocean - 1698 3974 - - 21033144 - - 25491 - - -wave - 16936 28839 43063 62743 87325m14b - 13136 26057 42783 67326 100286auto - 26379 45525 77611 122902 -

Quality – Walshaw BenchmarkFive percent case



Graph 2 4 8 16 32 64add32 - - - - - 493cti - 890 - - - -bcsstk30 - 16332 34350 - - -bcsstk31 2676 7152 - - - -fepwt - 701 1409 2777 5546 -bcsstk32 - 4644 8481 35965 - -febody - 601 1054 1784 2887 4888brack2 - 2739 6781 - - -fetooth - 6864 11662 17603 - -ferotor 1955 7045 12566 20132 - -598a - 7851 15721 25808 39369 58031feocean - 1697 3941 7722 - 20667144 - - 25259 38225 56926 -wave - 16820 28700 42800 62520 86663m14b - 13136 25679 42608 66793 99063auto - 25883 45039 76488 122378 173968

Conclusion – KaPPa



Record setting quality for large graphsScalable parallelismConcept of edge ratingeasy to integrate into other systems?Make results for weighted matchings usableParallelization and quality improvement by quotient graph edgecoloringImproved queue selection strategiesMore information: http://arxiv.org/pdf/0910.2004includes some errata

Future Work



Getting fasterReconsider every aspect of MGP.Example: n-level GP based on observations from routeplanning. More: edge ratings, stopping rules, initial partitioning,refinement algorithms 6=local search, overall strategy,. . .Parallel external partitioning of Tera-Node-NetworksSoftware releaseGeneralizations (clustering, objective functions, hypergraphs,. . . )

Recursive Coordinate Bisection



Similar to Bentley’s kd-treesParallel QUICK-SELECT

Could be improved using Parallel Randomized Selectionalgorithm by Blume et al.

Uncontract and Local Improvement



BFS to reduce communication volumeQueue Selection Strategies of local searchimproved quality1. use queue promising larger gain (TopGain)2. use queue of heavier block (MaxLoad) if one block is

overloaded

References I



B. Hendrickson and R. W. LelandA Multi-Level Algorithm For Partitioning Graphsin Supercomputing 1995

G. Karypis and V. KumarMultilevel Graph Partitioning SchemesICCP (3) 1995, p. 113-122

References II



C. Walshaw, M. Cross, M. G. EverettDynamic mesh partitioning: a unified optimisation andload-balancing algorithmTech. Rep. 95/IM/06, Univ. Greenwich, London

M. J. Berger and S. H. BokhariA Partitioning Strategy For Nonuniform Problems OnMultiprocessorsIEEE Trans. Comput., 36(5):570580, 1987

References III



H. D. SimonPartitioning of Unstructured Problems for Parallel ProcessingComputing Systems in Engineering

A. Pothen, H. D. Simon, K. P. LiuPartitioning Sparse Matrices with Eigenvectors of Graphs.SIAM J. on Matrix Analysis and Applications, 11(3):430-452,1990.

References IV



J. Maue and P. SandersEngineering Algorithms for Approximate Weighted MatchingExperimental Algorithms, p. 242-255

B. Monien, R. PreisQuality Matching and Local Improvement for MultilevelGraph-PartitioningParallel Computing, 26(12):1609-1634, 2000

References V



B. W. Kernighan, S. LinAn Efficient Heuristic Procedure for Partitioning GraphsBell Systems Technical Journal 14:291-307

C. SchulzScalable Parallel Refinement of Graph PartitionsDiplomarbeit, Universitt Karlsruhe (TH)

References VI



C. Fiduccia, R. MattheysesA Linear-Time Heuristic for Improving Network PartitionsDesign Automation, 1982

M. Halappanavar, F. Dobrian, A. PothenMatchings in Massive Graphs of Terrascale Computers viaApproximationTo appear, 2009

References VII



F. Pellegrini, J. RomanScotch: A Software Package for Static Mapping by DualRecursive Bipartitioning of Process and Architecture GraphsHigh-Performance Computing and Networking, p. 493-498

F. Manne and R. H. BisselingA Parallel Approximation Algorithm for the Weighted MaximumMatching Problem7th International Conference on Parallel Processing andApplied Mathematics (PPAM), 2007,

References VIII



Fiduccia, C. M. and Mattheyses, R. M.A Linear-Time Heuristic for Improving Network Partitions19th Conference on Design Automation, 1982

Engineering a Scalable High Quality Graph...

Documents

Transcript of Engineering a Scalable High Quality Graph...