Post on 06-May-2015
description
Tuesday
28.08 BIGD
ATA.
BE
8 Things I like about Datomic
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
+ something about temporal graph databases
About me
➡ Working as an it lead / software architect @ Janssen Diagnostics• Dealing with big scientific data sets
• Hands-on expertise in Big Data and NoSQL technologies
Who am I ...
Davy Suvee@DSUVEE
➡ Founder of datablend• Provide Big Data and NoSQL consultancy
• Share practical knowledge and Big Data use cases via blog
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
1 - Data Immutability
Datom [ 10 ‘:name’ ‘datablend’ ]
entity attribute value
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
1 - Data Immutability
Datom [ 10 ‘:name’ ‘datablend’ ]
entity attribute value
‘dataother’
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
1 - Data Immutability
Datom [ 10 ‘:name’ ‘datablend‘ ]
entity attribute value
[ 10 ‘:name’ ‘DATAOTHER‘ ]
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
defines time
1
2
tx tx
2 - Schema-Less kind of ...
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
Util.map(":db/id", Peer.tempid(":db.part/db"), ":db/ident", ":name", ":db/valueType", ":db.type/string", ":db/cardinality", ":db.cardinality/one", ":db/doc", "The name of an entity", ":db.install/_attribute", ":db.part/db"));
attribute definition
3 - Everything is a Datom
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:tel’ ‘01234566789‘ ]
[ 11 ‘:name’ ‘Davy Suvee‘ ][ 11 ‘:knows’ ‘java‘ ][ 11 ‘:knows’ ‘NoSQL‘ ][ 11 ‘:founded’ 10 ]
[ 99 ‘:db/ident’ ‘:name’ ][ 99 ‘:db/valueType’ ‘:db.type/string’ ][ 99 ‘:db/cardinality’ ‘:db.cardinality/one’ ][ 99 ‘:db/doc’ ‘The name of an entity’ ][ 99 ‘:db.install/_attribute’ ‘:db.part/db’ ]
3 - Everything Is A Datom
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
[ 10 ‘:name’ ‘datablend‘ 251]
transaction
3 - Everything Is A Datom
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
[ 10 ‘:name’ ‘datablend‘ 251]
transaction
[ 251 ‘:db/txInstant’ ‘35454920495’ ] timestamp
3 - Everything Is A Datom
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
[ 10 ‘:name’ ‘datablend‘ 251]
transaction
[ 251 ‘:db/txInstant’ ‘35454920495’ ] timestamp
[ 251 ‘:source’ ‘:external’ ][ 251 ‘:url’ ‘bigdata.be’ ]
meta-data
4 - Datalog
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
logic programming for your datoms
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:tel’ ‘01234566789‘ ]
[ 11 ‘:name’ ‘Davy Suvee‘ ][ 11 ‘:knows’ ‘java‘ ][ 11 ‘:knows’ ‘NoSQL‘ ][ 11 ‘:founded’ 10 ]
[:find ?entity ?name :where [?entity :name ?name]]
(10 ‘datablend‘)(11 ‘Davy Suvee‘)
4 - Datalog
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
logic programming for your datoms
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:tel’ ‘01234566789‘ ]
[ 11 ‘:name’ ‘Davy Suvee‘ ][ 11 ‘:knows’ ‘java‘ ][ 11 ‘:knows’ ‘NoSQL‘ ][ 11 ‘:founded’ 10 ]
[:find ?name :where [?person :name ?name] [?person :knows ‘java’]]
(‘Davy Suvee‘)
4 - Datalog
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
logic programming for your datoms
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:tel’ ‘01234566789‘ ]
[ 11 ‘:name’ ‘Davy Suvee‘ ][ 11 ‘:knows’ ‘java‘ ][ 11 ‘:knows’ ‘NoSQL‘ ][ 11 ‘:founded’ 10 ]
[:find ?name :where [?com :name ?name] [?person :knows ‘java’] [?person :founded ?com]]
(‘datablend‘)
4 - Datalog
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
... and as everything is a datom ...
[:find ?attribute :where [?attribute :db/valueType :db.type/string]]
4 - Datalog
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
... and as everything is a datom ...
[:find ?attribute :where [?attribute :db/valueType :db.type/string]]
[:find ?entity ?url :where [?entity :name “datablend” ?tx] [?tx :source :external] [?tx :url ?url]]
5 - Power your queries
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
[:find ?tech :where [_ :knows ?tech] [(.startsWith ?tech “J”)]]
any valid Java expression
5 - Power your queries
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
[:find ?tech :where [_ :knows ?tech] [(.startsWith ?tech “J”)]]
any valid Java expression
[[(ancestor ?person ?ancestor) [?person :person/parent ?ancestor]] [(ancestor ?person ?ancestor) [?person :person/parent ?parent] (ancestor ?parent ?ancestor)]]]
recursive rules
6 - Travel through time
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
Peer.q(query, conn.db());
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:name’ ‘DATAOTHER‘ ]tx
tx 12
(‘dataother’)
String query = “[:find ?name :where [10 :name ?name]]”;
6 - Travel through time
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
Peer.q(query, conn.db());
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:name’ ‘DATAOTHER‘ ]tx
tx 12
(‘dataother’)
Peer.q(query, conn.db().asOf(tx1)); (‘datablend’)
String query = “[:find ?name :where [10 :name ?name]]”;
6 - Travel through time
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
Peer.q(query, conn.db());
[ 10 ‘:name’ ‘datablend‘ ][ 10 ‘:name’ ‘DATAOTHER‘ ]tx
tx 12
(‘dataother’)
Peer.q(query, conn.db().asOf(tx1)); (‘datablend’)
String query = “[:find ?name :where [10 :name ?name]]”;
Peer.q(query, conn.db().with(futuretx)); (‘datamix’)
[ 10 ‘:name’ ‘datamix‘ ]
7 - Scalability reversed
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
8 - Pluggable data stores
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
➡ In memory (dev mode)
➡ SQL Database
➡ DYNAMO DB
➡ INFINISPAN
➡ Switch transparently and migration tools available
FluxGraph: Tinkerpop on top of Datomic
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
(http://github.com/datablend/blueprints)
FluxGraph
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
➡ Blueprints implementation on top of Datomic
FluxGraph
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
➡ Blueprints implementation on top of Datomic
public void setTime(Date checkpoint);➡ Graph Time Travel
FluxGraph
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
➡ Blueprints implementation on top of Datomic
public void setTime(Date checkpoint);
➡ Time-scoped iteration of vertices and edges
public Vertex getPreviousVersion();public Iterable<Edge> getNextVersions();public Iterable<Edge> getNextVersions(Filter filter);
➡ Graph Time Travel
Time-aware social graph
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
GraphConnect
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg
“The use of Temporal Graph Databases to ease the analysis of longitudinal patient data”
Questions?
info@datablend.be
Follow us
twitter.com/data_blendwww.datablend.be
Contact us
www.datablend.be info@datablend.be 0499/05.00.89 Diepestraat 23 - heist-op-den-berg