A Space Efficient Scheme for Persistent Graph Representation

A space efficientscheme for

persistent graphrepresentation

StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

A space efficient scheme forpersistent graph representation

Stavros Kontopoulosand

Georgios Drakopoulos

Computer Engineering and Informatics DepartmentUniversity of Patras

Greece

November 16, 2014



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Agenda

Overview

Motivation

Persistency

Application

Conclusions



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Agenda

Topics

Motivation

Persistency

Application

Conclusions



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

A graph era?

Prominent applications

Social media

Big data and IoT

Protein-protein interaction

Brain connectivity

Robust tools

Graph databases

Machine learing frameworks

Functional languages

Reasoners



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Connectivity is the key

Space

Edge patterns

Vertex patterns

Edge generatedEdge generating

Time

Insertion

Deletion



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

The time arrow

Data structures

Ephemeral

Persistent

Graph databases

Stronger S in BASE

Shorter E in BASE

Graph modeling

Event sampling

Parameter estimation



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Agenda

Topics

Motivation

Persistency

Application

Conclusions



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Research objectives

Persistent data structure

API

Framework

Efficiency

Transparency

Persistent graph models

Application

Demonstration

Notes

Other applications are possible



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Abstraction

Vertex

Unique number

Predicate(s)

Edge

Unique vertex pair

Predicate(s)

Notes

The graph is accessible only through the API



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Graph queries

Neighbours list

Return adjacency list of given vertex u

Return predicate(s) in subsequent queries

Edge information

Report whether edge (u , v) exists

Return predicate(s) in subsequent queries

Predicate list

Return predicate(s) of a given vertex or edge



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Vertex insertion

Single insertion

Vertex identifier

Optional predicate list

Multiple insertions

Vertex identifier

Optional predicate list

Neighbours list

Optional edge predicate list



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Vertex deletion

Single deletion

Vertex identifier

Notes

Multiple vertex deletions are not suppported

Similar operations for edges

Each operation returns a new graph version

Similar to functional languagesVertex is not physically deleted



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Implementation

Data structure

Store adjacency list for each vertex

Represent each such list as a string

Store all such strings in a trie

Keep only graph changes

Use a skip list to store states and state relations

State distance is based on a string distance metric

Working language

Scala implementation

Functional characteristics

Based on JVM



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Agenda

Topics

Motivation

Persistency

Application

Conclusions



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Kronecker product

Definition

A⊗ B4=

a1,1 B a2,1 B . . . a1,nA Ba2,1 B a2,2 B . . . a2,nA B

......

. . ....

amA,1 B amA,2 B . . . amA,nA B

∈ RmAmB×nAnB

Notes

Elementwise model

Easy implementation

Different connectivity patterns



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Kronecker graph modelDefinition

Generator graph

Select G0 as starting point

Basic community block

Determined by the problem

Dynamic equations

G[1] = G0 ∈ {0, 1}n0×n0 , n0 = |V0|G[τ ] = G[τ − 1]⊗G0 =

G0⊗ . . . ⊗G0︸︷︷︸τ−1

∈ {0, 1}nτ0 ×nτ0 , τ ≥ 2



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Kronecker graph properties

Large graph behavior

Power law in spectrum

Power law in primary eigenvector components

Power law in number of triangles

Deterministic graph generation

Probabilistic variant exists too

Challenging parameter estimation

Storing past graph states helps



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Algorithm

Steps

Let n0 be the size of the generator matrix

Do a mutiple insertion of the original vertices

Examine the community added at step τ

Maintain sequential vertex numbering

Then for each vi it holds that

qi = 1 + vi mod n0, 1 ≤ qi ≤ n0

ri = vi mod nτ−10 , 0 ≤ ri ≤ nτ−1

0 − 1

Do a multiple insertion of vertices belonging to the community

Check the generator matrix and connect with previouscommunities



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Agenda

Topics

Motivation

Persistency

Application

Conclusions



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Overview

Ready

Space efficient data structure

API

Persistent Kronecker graphs

In progress

Time efficiency

Persistent random graphs



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

J. Leskovec, D. Chakrabarti, J. Kleinberg, C. Faloutsos, andZ. Ghahramani, “Kronecker graphs: An approach to modelingnetworks,” J. Mach. Learn. Res., 2010.

M. Erwig, “Fully persistent graphs - which one to choose?” inImplementation of Functional Languages, ser. Lecture Notes inComputer Science. Springer Berlin Heidelberg, 1998, pp.123–140.

P. Dietz, “Fully persistent arrays,” in Algorithms and DataStructures, ser. Lecture Notes in Computer Science, F. Dehne,J.-R. Sack, and N. Santoro, Eds. Springer Berlin Heidelberg,1989, pp. 67–74.

P. Bose, K. Douıeb, and S. Langerman, “Dynamic optimality forskip lists and b-trees,” in Proceedings of the nineteenth annualACM-SIAM symposium on Discrete algorithms, ser. SODA ’08.SIAM, 2008, pp. 1106–1114.



StavrosKontopoulos

andGeorgios

Drakopoulos

Motivation

Persistency

Application

Conclusions

Questions?

A Space Efficient Scheme for Persistent Graph Representation

Science

Transcript of A Space Efficient Scheme for Persistent Graph Representation