Introducing Apache Giraph for Large Scale Graph Processing

23
Introducing Apache Giraph for Large Scale Graph Processing Sebastian Schelter PhD student at the Database Systems and Information Management Group of TU Berlin Committer and PMC member at Apache Mahout and Apache Giraph mail [email protected] blog http://ssc.io
  • date post

    18-Oct-2014
  • Category

    Education

  • view

    19.426
  • download

    3

description

 

Transcript of Introducing Apache Giraph for Large Scale Graph Processing

Page 1: Introducing Apache Giraph for Large Scale Graph Processing

Introducing Apache Giraph for Large Scale Graph Processing

Sebastian Schelter

PhD student at the Database Systems and Information Management Group of TU Berlin

Committer and PMC member at Apache Mahout and Apache Giraph

mail [email protected] blog http://ssc.io

Page 2: Introducing Apache Giraph for Large Scale Graph Processing

Graph recap

graph: abstract representation of a set of objects (vertices), where some pairs of these objects are connected by links (edges), which can be directed or undirected

Graphs can be used to model arbitrary things like road networks, social networks, flows of goods, etc.

Majority of graph algorithmsare iterative and traverse the graph in some way A

C

B

D

Page 3: Introducing Apache Giraph for Large Scale Graph Processing

Real world graphs are really large!

• the World Wide Web has several billion pages with several billion links

• Facebook‘s social graph had more than 700 million users and more than 68 billion friendships in 2011

• twitter‘s social graph has billions of follower relationships

Page 4: Introducing Apache Giraph for Large Scale Graph Processing

Why not use MapReduce/Hadoop?

• Example: PageRank, Google‘s famous algorithm for measuring the authority of a webpage based on the underlying network of hyperlinks

• defined recursively: each vertex distributes its authority to its neighbors in equal proportions

),( ijj j

j

id

pp

Page 5: Introducing Apache Giraph for Large Scale Graph Processing

Textbook approach to PageRank in MapReduce

• PageRank p is the principal eigenvector of the Markov matrix M defined by the transition probabilities between web pages

• it can be obtained by iteratively multiplying an initial PageRank vector by M (power iteration)

row 1 of M

pi

row n of M

pi+1row 2 of M

0pMp

k

Page 6: Introducing Apache Giraph for Large Scale Graph Processing

Drawbacks

• Not intuitive: only crazy scientists think in matrices and eigenvectors

• Unnecessarily slow: Each iteration is scheduled as separate MapReduce job with lots of overhead– the graph structure is read from disk

– the map output is spilled to disk

– the intermediary result is written to HDFS

• Hard to implement: a join has to be implemented by hand, lots of work, best strategy is data dependent

Page 7: Introducing Apache Giraph for Large Scale Graph Processing

Google Pregel

• distributed system especially developed for large scale graph processing

• intuitive API that let‘s you ‚think like a vertex‘

• Bulk Synchronous Parallel (BSP) as execution model

• fault tolerance by checkpointing

Page 8: Introducing Apache Giraph for Large Scale Graph Processing

Bulk Synchronous Parallel (BSP)

processors

local computation

communication

barrier synchronization

superstep

Page 9: Introducing Apache Giraph for Large Scale Graph Processing

Vertex-centric BSP

• each vertex has an id, a value, a list of its adjacent vertex ids and the corresponding edge values

• each vertex is invoked in each superstep, can recompute its value and send messages to other vertices, which are delivered over superstep barriers

• advanced features : termination votes, combiners, aggregators, topology mutations

vertex1

vertex2

vertex3

vertex1

vertex2

vertex3

vertex1

vertex2

vertex3

superstep i superstep i + 1 superstep i + 2

Page 10: Introducing Apache Giraph for Large Scale Graph Processing

Master-slave architecture

• vertices are partitioned and assigned to workers – default: hash-partitioning

– custom partitioning possible

• master assigns and coordinates, while workers execute vertices and communicate with each other

Master

Worker 1 Worker 2 Worker 3

Page 11: Introducing Apache Giraph for Large Scale Graph Processing

PageRank in Pregel

class PageRankVertex {

void compute(Iterator messages) {

if (getSuperstep() > 0) {

// recompute own PageRank from the neighbors messages

pageRank = sum(messages);

setVertexValue(pageRank);

}

if (getSuperstep() < k) {

// send updated PageRank to each neighbor

sendMessageToAllNeighbors(pageRank / getNumOutEdges());

} else {

voteToHalt(); // terminate

}

}}

),( ijj j

j

id

pp

Page 12: Introducing Apache Giraph for Large Scale Graph Processing

PageRank toy example

A B C

.33 .33 .33

.17

.17

.33.17

Superstep 0

.17

.17 .50 .34

.09

.09

.34.25

Superstep 1

.25

.25 .43 .34

.13

.13

.34.22

Superstep 2

.22

Input graph

Page 13: Introducing Apache Giraph for Large Scale Graph Processing

Cool, where can I download it?

• Pregel is proprietary, but:

– Apache Giraph is an open source implementation of Pregel

– runs on standard Hadoop infrastructure

– computation is executed in memory

– can be a job in a pipeline (MapReduce, Hive)

– uses Apache ZooKeeper for synchronization

Page 14: Introducing Apache Giraph for Large Scale Graph Processing

Giraph‘s Hadoop usage

TaskTracker

worker worker

TaskTracker

worker worker

TaskTracker

worker worker

TaskTracker

master workerZooKeeperJobTracker NameNode

Page 15: Introducing Apache Giraph for Large Scale Graph Processing

Anatomy of an execution

Setup• load the graph from disk• assign vertices to workers• validate workers health

Compute• assign messages to workers• iterate on active vertices• call vertices compute()

Synchronize• send messages to workers• compute aggregators• checkpoint

Teardown• write back result• write back aggregators

Page 16: Introducing Apache Giraph for Large Scale Graph Processing

Who is doing what?

• ZooKeeper: responsible for computation state– partition/worker mapping – global state: #superstep– checkpoint paths, aggregator values, statistics

• Master: responsible for coordination– assigns partitions to workers– coordinates synchronization– requests checkpoints– aggregates aggregator values– collects health statuses

• Worker: responsible for vertices– invokes active vertices compute() function– sends, receives and assigns messages– computes local aggregation values

Page 17: Introducing Apache Giraph for Large Scale Graph Processing

What do you have to implement?

• your algorithm as a Vertex

– Subclass one of the many existing implementations: BasicVertex, MutableVertex, EdgeListVertex, HashMapVertex, LongDoubleFloatDoubleVertex,...

• a VertexInputFormat to read your graph

– e.g. from a text file with adjacency lists like <vertex> <neighbor1> <neighbor2> ...

• a VertexOutputFormat to write back the result

– e.g. <vertex> <pageRank>

Page 18: Introducing Apache Giraph for Large Scale Graph Processing

Starting a Giraph job

• no difference to starting a Hadoop job:

$ hadoop jar giraph-0.1-jar-with-dependencies.jar o.a.g.GiraphRunner o.a.g.examples.ConnectedComponentsVertex --inputFormat o.a.g.examples.IntIntNullIntTextInputFormat --inputPath hdfs:///wikipedia/pagelinks.txt --outputFormat o.a.g.examples.ComponentOutputFormat --outputPath hdfs:///wikipedia/results/--workers 89 --combiner o.a.g.examples.MinimumIntCombiner

Page 19: Introducing Apache Giraph for Large Scale Graph Processing
Page 20: Introducing Apache Giraph for Large Scale Graph Processing

What‘s to come?

• Current and future work in Giraph– graduation from the incubator

– out-of-core messaging

– algorithms library

• 2-day workshop after Berlin Buzzwords– topic: ‚Parallel Processing beyond MapReduce‘

– meet the developers of Giraph and Stratospherehttp://berlinbuzzwords.de/content/workshops-berlin-buzzwords

Page 21: Introducing Apache Giraph for Large Scale Graph Processing

Everything is a network!

Page 22: Introducing Apache Giraph for Large Scale Graph Processing

Further resources

• Apache Giraph homepagehttp://incubator.apache.org/giraph

• Claudio Martella: “Apache Giraph: Distributed Graph Processing in the Cloud” http://prezi.com/9ake_klzwrga/apache-giraph-distributed-graph-processing-in-the-cloud/

• Malewicz et al.: „Pregel – a system for large scale graph processing“, PODC 09http://dl.acm.org/citation.cfm?id=1582723

Page 23: Introducing Apache Giraph for Large Scale Graph Processing

Thank you.

Questions?