Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17])...
Transcript of Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17])...
![Page 1: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/1.jpg)
Dependency-Driven Disk-based Graph ProcessingKeval Vora
Simon Fraser University
https://github.com/pdclab/lumos
USENIX ATC’19 - Renton, WAJuly 11, 2019
![Page 2: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/2.jpg)
Graph Processing12/20
![Page 3: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/3.jpg)
Iterative Graph Processing13/20PageRank computation
![Page 4: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/4.jpg)
Iterative Graph Processing
ab
c d
e
f
ab
c d
e
f
iteration t iteration t+1
13/20
![Page 5: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/5.jpg)
Out-of-Core Graph Processing
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =a b c d e fV =
P0 P1P0 P1
14/20iteration t
iteration t+1
partitions reside on disk
![Page 6: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/6.jpg)
Out-of-Core Graph Processing
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =a b c d e fV =
iteration t
iteration t+1
P0 P1 P0 P1
14/20
![Page 7: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/7.jpg)
Out-of-Core Graph Processing
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =a b c d e fV =
iteration t
iteration t+1
P0 P1 P0 P1
14/20
![Page 8: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/8.jpg)
Out-of-Core Graph Processing
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
iteration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
14/20
![Page 9: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/9.jpg)
I/O C
Out-of-Core Graph Processing
I/O C I/O C I/O C
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
iteration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
14/20
compute
![Page 10: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/10.jpg)
I/O C
Out-of-Core Graph Processing
I/O C I/O C I/O C
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
iteration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/O C I/O C I/O C I/O C
I/O = 69-90%
14/20
![Page 11: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/11.jpg)
I/O C
Lumos
I/O C
ae
b
cd ea
db
c f
a b c d e fV =• Out-of-Order processing
• Amortize I/O across multiple iterations
• Guarantee Bulk Synchronous Parallel Semantics
• Dependency-Driven Cross-Iterationvalue propagation• Safely compute future values
iteration t
P0 P1
15/20
![Page 12: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/12.jpg)
Computing Future Values
ae
b
cd
ae
b
cd
c f
ea
db
c f
ea
db
VF =
a b c d e fV =
a b c d e f
iteration t
iteration t+1
a b c d e fV =
P0 P0 P1
I/O C I/O C I/O C I/O C
P1
16/20future vertex values
![Page 13: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/13.jpg)
Computing Future Values
b
ae
b
cd
c
ea
db
c f
ea
b
f
d
VF =
a b c d e fV =
a b c d e f
ae
cditeration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/O C I/O C I/O C I/O C
16/20
![Page 14: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/14.jpg)
I/O I/O
Computing Future Values
b
ae
b
cd
c
ea
db
c f
ea
b
f
d
VF =
a b c d e fV =
a b c d e f
ae
cditeration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/O C I/O C I/O C C
16/20
![Page 15: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/15.jpg)
I/O
Computing Future Values
b
ae
b
cd
c
e
db
c f
ea
b
f
d
VF =
a b c d e fV =
a b c d e f
ae
cditeration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/O C I/O C I/O C C
16/20
![Page 16: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/16.jpg)
I/O
Computing Future Values
b
ae
b
cd
c
e
db
c f
ea
b
f
d
VF =
a b c d e fV =
a b c d e f
ae
cditeration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/O C I/O C I/O C C
16/20
future dependencies not met
![Page 17: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/17.jpg)
Computing Future Values
0
15
30
45
UK
TW TT FT UK
TW TT FT UK
TW TT FT
16 64 256Chunk Cyclic Random
% V
erti
ces
Edges Vertices
UK 1B 39.5M
TW 1.5B 41.7M
TT 2B 52.6M
FT 2.5B 68.3M
0
1
2
3
4
UK
TW TT FT UK
TW TT FT UK
TW TT FT
16 64 256Chunk Cyclic Random
% E
dges
17/2030-45% vertices compute future values, and they contribute for less than 4% edges
![Page 18: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/18.jpg)
Graph Computation
f
Partial Aggregation
18/20
![Page 19: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/19.jpg)
Graph Computation
f
Partial Aggregation
18/20
![Page 20: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/20.jpg)
Graph Computation
f
Partial Aggregation
18/20
![Page 21: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/21.jpg)
Graph Computation
f
Partial Aggregation
18/20
![Page 22: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/22.jpg)
Cross-Iteration Value Propagation
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
V = a b c d e f
Partial Aggregation
iteration t
iteration t+1
a b c d e fV =
P0 P0 P1
I/O C I/O C I/O C I/O C
P1
19/20
![Page 23: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/23.jpg)
Cross-Iteration Value Propagation
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
V = a b c d e f
Partial Aggregation
iteration t
iteration t+1
a b c d e fV =
P0 P0 P1
I/O C I/O C I/O C I/O C
P1
19/20
![Page 24: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/24.jpg)
I/O
Cross-Iteration Value Propagation
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
V = a b c d e f
Partial Aggregation
iteration t
iteration t+1
a b c d e fV =
P0 P0
I/O C I/O C I/O C I/O C
P1 P1
19/20
![Page 25: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/25.jpg)
Cross-Iteration Value Propagation
ae
b
cd
ae
b
cdea
db
c f
ea
db
c f
a b c d e fV =
V = a b c d e f
Partial Aggregation
iteration t
iteration t+1
a b c d e fV =
P0 P0
I/O C I/O C I/O C
P1 P1
19/20
I/O C
![Page 26: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/26.jpg)
Cross-Iteration Value Propagation
0
15
30
45
UK
TW TT FT UK
TW TT FT UK
TW TT FT
16 64 256Chunk Cyclic Random
% E
dges
10/20
future values computed across >45% edges
![Page 27: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/27.jpg)
Intra-Partition Propagation
ae
b
cd
ae
b
cdea
db
c f
e
d
f
a b c d e fV =
V = a b c d e f
iteration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/OI/O C I/O C C CI/OI/O
11/20
![Page 28: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/28.jpg)
Intra-Partition Propagation
ae
b
cd
ae
b
cdea
db
c f
e
d
f
a b c d e fV =
V = a b c d e f
iteration t
iteration t+1
a b c d e fV =
P0 P1 P0 P1
I/O C I/O C C CI/OI/O
11/20
![Page 29: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/29.jpg)
Lumos: Cross-Iteration Value Propagation
• Propagation across multiple future iterations• Synergy with Dynamic Shards [ATC’16]• Detailed comparison with out-of-order techniques• Generalized graph layout & partitioning strategies• Cross-iteration propagation across different partition sizes• Interplay with selective scheduling
• Lumos for Asynchronous Algorithms
12/20
![Page 30: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/30.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd
ae
b
cdea
db
c f
e
d
f
a b c d e fV =a b c d e fV =
V = a b c d e f
iteration t
iteration t+113/20shortest paths computation
![Page 31: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/31.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd
ae
b
cdea
db
c f
e
d
f
a b c d e fV =a b c d e fV =
V = a b c d e f
iteration t
iteration t+113/20
![Page 32: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/32.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd ea
db
c f
a b c d e fV =
V = a b c d e f • Monotonic selection (KickStarter [ASPLOS’17])
iteration t
13/20
![Page 33: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/33.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd ea
db
c f
a b c d e fV =
V = a b c d e f • Monotonic selection (KickStarter [ASPLOS’17])• Partial aggregations merged directly
iteration t
13/20
![Page 34: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/34.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd ea
db
c f
a b c d e fV =
• Monotonic selection (KickStarter [ASPLOS’17])• Partial aggregations merged directly
• Partial values safe to be propagated• Any change can be propagated immediately
iteration t
13/20
![Page 35: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/35.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd ea
db
c f
a b c d e fV =
• Monotonic selection (KickStarter [ASPLOS’17])• Partial aggregations merged directly
• Partial values safe to be propagated• Any change can be propagated immediately
iteration t
13/20
![Page 36: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/36.jpg)
Lumos for Asynchronous Algorithms
ae
b
cd ea
db
c f
a b c d e fV =
• Monotonic selection (KickStarter [ASPLOS’17])• Partial aggregations merged directly
• Partial values safe to be propagated• Any change can be propagated immediately
• Intra-partition propagation based on degree of asynchrony (ASPIRE [OOPSLA’14])it
eration t
13/20
![Page 37: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/37.jpg)
• Grid layout based on GridGraph [ATC’15]
Lumos System
source
target
14/20
![Page 38: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/38.jpg)
Lumos System
source
target
source =target =
14/20
![Page 39: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/39.jpg)
source
target
Lumos System14/20
future values propagated for upper triangle + diagonal edges
![Page 40: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/40.jpg)
I/O C
V =
V =
V =
I/O C
Lumos Systemiteration t
iteration t+1
14/20only lower triangle edges loaded
![Page 41: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/41.jpg)
I/O C
V =
V =
V =
I/O C
Lumos Systemiteration t
iteration t+1
14/20
![Page 42: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/42.jpg)
Light-weight Degree-Aware PartitioningHOF: Highest Out-Degree FirstHIL: Highest In-Degree LastHRF: Highest Out-Degree to In-Degree Ratio First
source
target
15/20
maximize edges in upper triangle
![Page 43: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/43.jpg)
Light-weight Degree-Aware Partitioning
0
25
50
75
100
16 32 64 128
256 16 32 64 128
256 16 32 64 128
256
TT FT YHHOF HIL HRF
% E
dges
# Partitions
HOF: Highest Out-Degree FirstHIL: Highest In-Degree LastHRF: Highest Out-Degree to In-Degree Ratio First
Edges Vertices
TT 2B 52.6M
FT 2.5B 68.3M
YH 6.6B 1.4B
15/20
cross-iteration propagation across 72-93% edges
![Page 44: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/44.jpg)
Experimental Setup
• Graph algorithms• PageRank (weighted and unweighted), Belief Propagation,
Co-Training Expectation Maximization, Dispersion, Label Propagation
• Performance on single disk• h1.2xlarge: 278MB/sec HDD read bandwidth
• Scaling I/O• d2.4xlarge: 195-768MB/sec read bandwidth over 1-4 HDDs• i3.8xlarge: 1.2-4.1GB/sec read bandwidth over 1-4 SSDs
16/20
![Page 45: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/45.jpg)
Performance
0
0.2
0.4
0.6
0.8
1G
ridG
raph
LUM
OS
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
TT YH
PR CoEM DP BP WPR LP PR CoEM DP BP WPR LPN
orm
aliz
ed T
ime
h1.2xlarge: 278MB/sec HDD read bandwidth
Edges Vertices
TT 2B 52.6M
YH 6.6B 1.4B
17/20
![Page 46: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/46.jpg)
Performance
0
0.2
0.4
0.6
0.8
1G
ridG
raph
LUM
OS
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
Gri
dGra
phLU
MO
S
TT YH
PR CoEM DP BP WPR LP PR CoEM DP BP WPR LP
Read Compute
Nor
mal
ized
Tim
e
h1.2xlarge: 278MB/sec HDD read bandwidth
Edges Vertices
TT 2B 52.6M
YH 6.6B 1.4B
17/20
![Page 47: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/47.jpg)
Scaling I/O
0
2K
4K
1 2 3 4
Exe
cuti
on T
ime
# Drives
HDD SSD
Edges Vertices
RMAT29 8.6B 537M
DeviceType
Single Drive
RAID-0 with k drives
k = 2 k = 3 k = 4
d2.4xlarge HDD 195MB/s 368MB/s 590MB/s 768MB/s
i3.8xlarge SSD 1.2GB/s 3.8GB/s 4.1GB/s 3.9GB/s
18/20
![Page 48: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/48.jpg)
0
25
50
75
100
16 32 64 128
256 16 32 64 128
256 16 32 64 128
256
TT FT YHHOF HIL HRF
% E
dges
# Partitions
Detailed Evaluation of Lumos
0 50
100 150 200 250
00:00 01:30
Rea
ds (
MB
/s)
Time
1 2
4 8
16
0 200 400 600 800
1000
00:00 01:30
I/O
Wai
t T
ime
(ms)
Time
1 2
4 8
16
0 0.2 0.4 0.6 0.8
1 1.2 1.4 1.6
GG LBL
GG LBL
GG LBL
GG LBL
TW TT FT YHDegrees OrderLayouts
Nor
mal
ized
Tim
e
0 0.2 0.4 0.6 0.8
1 1.2
GG LBL
GG LBL
GG LBL
GG LBL
TW TT FT YHPrimary Secondary
Nor
mal
ized
Spa
ce
0
0.2
0.4
0.6
0.8
1
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
GG LBL
TT FT YHPR CoEM DP BP WPR LP PR CoEM DP BP WPR LP PR CoEM DP BP WPR LP
Read Compute
Nor
mal
ized
Tim
e
19/20
![Page 49: Dependency-Driven Disk-based Graph Processing...•Monotonic selection (KickStarter[ASPLOS’17]) •Partial aggregations merged directly •Partial values safe to be propagated •Any](https://reader034.fdocuments.us/reader034/viewer/2022050103/5f41eac717582a02750e58f0/html5/thumbnails/49.jpg)
Conclusion
• Dependency-Driven Cross-iteration value propagation • Out-of-Order processing to reduce I/O• Guarantees Bulk Synchronous Parallel semantics
• Generic technique that fundamentally eliminates the performance barrier
github.com/pdclab/lumos