Graph Day Texas: Open Source Graph Projects from PokitDok
-
Upload
denise-gosnell-phd -
Category
Data & Analytics
-
view
324 -
download
1
Transcript of Graph Day Texas: Open Source Graph Projects from PokitDok
![Page 1: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/1.jpg)
A tour of the PokitDok Health Graph and some open source graph projects
Graph Day Texas, Jan 2016Denise Gosnell, PhD
Twitter and Github:@pokitdok
@denisekgosnell
![Page 2: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/2.jpg)
Confidential 2
PokitDok APIs:
The business of health,for developers.
https://platform.pokitdok.com/
Twitter and Github:@pokitdok
@denisekgosnell
![Page 3: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/3.jpg)
Confidential 3
PokitDok APIs: Marketplace
![Page 4: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/4.jpg)
Confidential 4
Doctor on Demand: Powered by PokitDok
Twitter and Github:@pokitdok
@denisekgosnell
![Page 5: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/5.jpg)
onward to graphs.
![Page 6: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/6.jpg)
6
What we built. The HealthGraph
What we’ve open sourced.A Gremlin-Python LibraryCustom Titan BuildDynamic JSON Graph [WIP]HealthGraph DSL [WIP]
Talk Outline: Twitter and Github:@pokitdok
@denisekgosnell
![Page 7: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/7.jpg)
The PokitDok HealthGraph
![Page 8: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/8.jpg)
Confidential 8
X12 Data Standard: ETL hell from the 1970s
Twitter and Github:@pokitdok
![Page 9: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/9.jpg)
Confidential 9
X12 Data Standard: ETL hell from the 1970s
Twitter and Github:@pokitdok
![Page 10: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/10.jpg)
Confidential 10
Health Graph: Transaction as Trees
• We treat transactions as first-class objects in the graph
• Buried in the depth of an X12 transactions are the entities of interest
Twitter and Github:@pokitdok
Interactive graph available at: https://fullmetalhealth.com/dsl/
![Page 11: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/11.jpg)
Confidential 11
HealthGraph: Property Graph Model
Twitter and Github:@pokitdok
@denisekgosnell
![Page 12: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/12.jpg)
Confidential 12
HealthGraph: Probabilistic Inferences
![Page 13: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/13.jpg)
Confidential 13
HealthGraph: Data Inferences
Twitter and Github:@pokitdok
@denisekgosnell
![Page 14: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/14.jpg)
Confidential 14
HealthGraph: Predictive Models
• What is the probability claim X will be denied?
• A new customer just searched for “family practice”; recommend the best provider within 10 miles.
• Given a CPT code, what is the expected reimbursement rate from insurance company A in zip code 37601?
Twitter and Github:@pokitdok
@denisekgosnell
![Page 15: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/15.jpg)
Confidential 15HealthGraph: Top 100k Providers
Twitter and Github:@pokitdok
@MacraeAlec
![Page 16: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/16.jpg)
PokitDok Open Source:
Gremlin Python
![Page 17: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/17.jpg)
Confidential 17
Our HealthGraph Production Stack
• Titan 0.5.3
• TinkerPop’s Blueprints 2.50
• Cassandra and Elastic Search
Gremlin-Python
Twitter and Github:@pokitdok
@denisekgosnell
![Page 18: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/18.jpg)
Confidential 18
• Lighter Context Switching between development tools and environments
• Incompatible syntax issues between Gremlin and Python
• Using Python.
Gremlin-Python MotivationTwitter and Github:
@corbinbs@denisekgosnell
![Page 19: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/19.jpg)
Confidential 19
Option 1: Grab our docker container1. Install Dockerhttps://www.docker.com/docker-toolbox
2. Jump in the “Docker Quickstart Terminal”
3. Fire up our example container:docker run -i -t pokitdok/gremlin-python-test-drive
Option 2: Shell script install1. Clone our repo:https://github.com/pokitdok/gremlin-python
2. Run the set-up scripts:$./test_drive/setup.sh &&./test_drive/run.sh
Gremlin-Python Test Drive Twitter and Github:@corbinbs
@denisekgosnell
![Page 20: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/20.jpg)
Confidential 20
Bi-Partite Graph Recommendation System
Customer
viewedscheduled_wit
h
Doctor
Twitter and Github:@pokitdok
@denisekgosnell
![Page 21: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/21.jpg)
Confidential 21
Bi-Partite Graph Recommendation System
Customer
viewed
Doctor
Twitter and Github:@pokitdok
@denisekgosnell
![Page 22: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/22.jpg)
Confidential 22
Bi-Partite Graph Recommendation System
Customer
viewed
Doctor
Twitter and Github:@pokitdok
@denisekgosnell
![Page 23: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/23.jpg)
Confidential 23
Bi-Partite Graph Recommendation System
Customer
viewed
Doctor
Twitter and Github:@pokitdok
@denisekgosnell
![Page 24: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/24.jpg)
Confidential 24
Bi-Partite Graph Recommendation System
Customer
viewed
Doctor
Twitter and Github:@pokitdok
@denisekgosnell
g.E.has(‘edge_type’,’scheduled_with’) .in_v() .group_count(ranked_docs, lambda it: it.full_name, lambda it: it.b+1.0)
![Page 25: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/25.jpg)
Confidential 25
Gremlin-Python Test Drive Twitter and Github:@corbinbs
@denisekgosnell
![Page 26: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/26.jpg)
PokitDok Open Source:
Custom Build of Titan 0.5.3 to Integrate with CDH5 Containers
![Page 27: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/27.jpg)
Confidential 27
Motivation for Release of Custom Build:Graph Production Stack:
Titan 0.5.x ships with Hadoop 2.2API Production Stack:
contains Cloudera’s CDH5 containers and Hadoop 2.6.0You guessed it:
infrastructure dependency errors upon integration the Hadoop 2.6.0 API is not fully backwards compatible with Hadoop 2.2
Twitter and Github:@pokitdok
![Page 28: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/28.jpg)
Confidential 28
Released:A modification of the Titan 0.5.3 build to upgrade to Hadoop 2.6.0 and resolve numerous conflicts among transitive dependencies.
… someone had to do it.
Grab it here: https
://github.com/pokitdok/titan/tree/0.5.3-
hadoop2.6.0
Tested for Cassandra but not Hbase.
Twitter and Github:@pokitdok
![Page 29: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/29.jpg)
HealthGraph Dynamic JSON Load
Open Source Version [WIP]
![Page 30: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/30.jpg)
Confidential 30
Dyanmic JSONLoader:
Goal: Bulk load of JSON from sequenced HDFS files straight to a Titan DB
Twitter and Github:@pokitdok
![Page 31: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/31.jpg)
Confidential 31
1. Extract PokitDok HealthGraph specific features
2. Move to Titan 1.0 and TP3 compatibility3. Release on PokitDok GitHub
Dyanmic JSONLoader Future WorkTwitter and Github:
@pokitdok
![Page 32: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/32.jpg)
HealthGraph DSL
Open Source Version [WIP]
![Page 33: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/33.jpg)
Confidential 33
X12 Data Standard: ETL hell from the 1970s
Twitter and Github:@pokitdok
![Page 34: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/34.jpg)
Confidential 34
X12 Spec Trees vs. Graph DSL:Twitter and Github:
@pokitdok
Interactive graph available at: https://fullmetalhealth.com/dsl/
![Page 35: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/35.jpg)
Confidential 35
Graph DSL with TinkerPop 2.5:Twitter and Github:
@pokitdok
![Page 36: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/36.jpg)
Confidential 36
1. Move to Titan 1.0 and TP3 compatibility2. Release on PokitDok GitHub3. Current Open Question:
We are looking for(ward to) more documentation on implementing custom gremlin steps(DSLs) in TP3
DSL Future WorkTwitter and Github:
@pokitdok
![Page 37: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/37.jpg)
and there will be more…!
![Page 39: Graph Day Texas: Open Source Graph Projects from PokitDok](https://reader035.fdocuments.us/reader035/viewer/2022070513/5885cac61a28ab42028b556f/html5/thumbnails/39.jpg)
A tour of the PokitDok Health Graph and some open source graph projects
Graph Day Texas, Jan 2016Denise Gosnell, PhD
Twitter and Github:@pokitdok
@denisekgosnell