Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP...
Transcript of Temporal Graph Analytics with GRADOOP - fosdem.org · Temporal Graph Analytics with GRADOOP...
Temporal Graph Analytics with GRADOOP
Christopher Rost and Kevin GomezLeipzig University
About the speakers and the team
Kevin, PhD Student
Christopher, PhD Student
Prof. Dr. Erhard Rahm,Database Chair
Timo, M.Sc. Student
Lucas, B.Sc. Student
We’re hiring!
CENTER FORSCALABLE DATA ANALYTICSAND ARTIFICIAL INTELLIGENCE
● Visit us at www.scads.de
● Open positions and projects (Developer and PhD Students)
● Various topics:● Machine Learning (Graph-ML, Privacy)● Image Recognition● Blockchain (Distributed-Ledger)● Graph Analytics
Motivation
“Graphs are everywhere”
“Graphs are everywhere”
“Graphs are everywhere”
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
frien
dOf
friendO
f
friendO
f
friendOf
friendOf
friendO
f
Anna
Bob
Chris
Sally
Dutch
Arthur
Jack
SheldonRaj
Mike
James
John
“Graphs are everywhere”
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
frien
dOf
friendO
f
friendO
f
friendOf
friendOf
friendO
f
Anna
Bob
Chris
Sally
Dutch
Arthur
Jack
SheldonRaj
Mike
James
Johnage:29sex: f
yob: 1991
“Graphs are everywhere”
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
friendOf
frien
dOf
friendO
f
friendO
f
friendOf
friendOf
friendO
f
Anna
Bob
Chris
Sally
Dutch
Arthur
Jack
SheldonRaj
Mike
James
Johnage:29sex: f
yob: 1991
since: 5Y
“A open-source framework and research platform for efficient, distributed and domain independent management and analytics of heterogeneous and
temporal graph data.“
Architecture
HDFS / YARN
Distributed Operator Implementation
TPGMEPGM
G raph A nalytical L anguage (GrALa) Java 8
45k LOC
ALv 2.0
Graphs and collections of graphs
OperatorsOperators
Unary Binary
Log
ical G
rap
hG
rap
h C
ollecti
on
Algorithms
Frequent Subgraph
Temporal extension
Temporal extension
Temporal extension
Temporal extension
Time dependent operatorsOperators
Unary Binary
Log
ical G
rap
hG
rap
h C
ollecti
on
Algorithms
Frequent Subgraph
Time dependent operatorsOperators
Unary Binary
(Tem
p-)
Log
ical G
rap
h(T
em
p-)
Gra
ph
Collecti
on
Algorithms
Frequent Subgraph
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
Surgery
Accident
Cardiology
Oncology
Contact trackingSensors capturing who is close to whom
at what time.{empID1, empID2, t_from, t_to}
In case of an infection, which
hospital services are at risk of contracting
the virus X?
Use case: spread of airborne pathogens
Images by Vecteezy.com.
timet0 t1
t5 ... t
10 t
15 t
20
TemporalGraph contacts = mySource.getTemporalGraph();
TemporalGraph contacts = mySource.getTemporalGraph();
time
Employee[-,-)id: 32432srv: Oncology
Employee[-,-)id: 99934srv: Oncology
contact[ t
2 ,t
6 )
t0 t1
t5 ... t
10 t
15 t
20
timet0 t1
t5 ... t
10 t
15 t
20
Breaking news:
Employee of oncology infected by VirusX.
Detected at t20
.
timet0 t1
t5 ... t
10 t
15 t
20
Breaking news:
Employee of oncology infected by VirusX.
Detected at t20
.
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
contactmaxDur: 40 min
contactmaxDur: 122 min
contactmaxDur: 124 min
contactmaxDur: 146 min
contactmaxDur: 2 min
contactmaxDur: 9 min
timet0 t1
t5 ... t
10 t
15 t
20
Virus XSymptoms: 3rd eye growing
Transmission: if contact > 5 minIncubation period: 5 time units
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
contactmaxDur: 40 min
contactmaxDur: 122 min
contactmaxDur: 124 min
contactmaxDur: 146 min
contactmaxDur: 2 min
contactmaxDur: 9 min
timet0 t1
t5 ... t
10 t
15 t
20
contactmaxDur: 40 min
contactmaxDur: 122 min
contactmaxDur: 124 min
contactmaxDur: 146 min
contactmaxDur: 2 min
contactmaxDur: 9 min
Breaking news:
Employees of oncology and surgery quarantined
because of VirusX.
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
TemporalGraph contacts = mySource.getTemporalGraph();
contacts = contacts.snapshot(new FromTo(t15, t20));
contacts = contacts.groupBy((v v[‘srv’]),→ v[‘srv’]),[],(e e.getLabel()),→ v[‘srv’]),[new MaxDuration()]);
Conclusion● Distributed graph analysis system● Temporal property graph model
– Bitemporal support– Logical graphs and graph collections– Composing operators and algorithms
● Declarative workflow creation
● Visit Gradoop: http://gradoop.com● Read our Wiki: https://github.com/dbs-leipzig/gradoop/wiki● Start: https://github.com/dbs-leipzig/gradoop/wiki/Getting-started● Try the examples: https://github.com/dbs-leipzig/gradoop/wiki/Examples