1.INTRO TO GRAPHDATABASESUsing Tinkerpop, TitanDB, and Gremlin{email : calebjones@gmail.com,website : http://calebjones.info,twitter : @JonesWCaleb}

2. Overview Why Graphs? Order to complexity Use cases major players Graphs & Adjacency Matrices Tinkerpop Framework Blueprints, Frames, Pipes, Furnace, Gremlin, Rexster Titan using Cassandra Blog Application (lab) Traversals using Gremlin 3. WHY GRAPHS? 4. Warren Weaver 17th - 19th century Problems of simplicity How one element interacts withanother First half of 20th century Problem of disorganized complexity Many elements operating in a systemw/o regard to how they interact witheach other Predicted Problem of organized complexity Many elements operating in a systemtaking into account how they interactwith each other Would require computational powerfar beyond what was currentlyavailableScience and Complexity1948ENIAC (1946) 5. Organisms 6. Knowledge Classification 7. Organizational Hierarchy 8. Neurology 9. Order to Complexity Trees describe order Linear (simple lineage) Categorized Single dimensional Symmetrical Hierarchical Convergent modeling Networks describe complexity Non-linear (multi-lineage) Multi-categorical Multi-dimensional Asymmetrical Decentralized Divergent modeling 10. Types of Networks 11. Types of Networks 12. Types of Networks 13. Types of Networks 14. Types of Networks 15. Types of Networks 16. Types of Networks 17. Types of Networks 18. -DFN.LUEDUO*DIIRUG3DXO5HLQPDQ6RO%URGVN/DUU/LHEHU%LOO(YHUHWW'LFN$HUV6WHYH'LWNR6DP5RVHQ5LFKDUG+RZHOO$O*RUGRQ.HQ)HGXQLHLZLF]DUPLQH,QIDQWLQR)UDQN6SULQJHU6WDQ/HH0LNH(VSRVLWR-RH0DQHHO-RKQ7DUWDJOLRQH+HUE7ULPSH$O.XU]URN*DU)ULHGULFK-LP6WHUDQNR$UWLH6LPHNKLF6WRQH*HRUJH5RXVVRV6HH1RWHV$OH[7RWK9LQFHROOHWWD/3*UHJRU)UDQN*LDFRLD:HUQHU5RWK:DOO:R-RRGH6LQQRWW5R7KRPDV-DFN6SDUOLQJ'DQ$GNLQV*LO.DQH-HUU5)RHVVOG$PQDGQUX'RQ+HFN-RVKXD0LGGOHWRQ0LNH*XVWRYLFK%UHW%OHYLQV$OIUHGR$OFDOD%RE/DUNLQ(G+DQQLJDQ%RE+DOO'DQQ)LQJHURWK*HUURQZDDUROH6HXOLQJKULVWLH6FKHHOH-RKQ5RPLWD-5-RKQ9HUSRRUWHQ*DVSDU6DODGLQR*HRUJH7XVND-RKQ%XVFHPD0RUULH.XUDPRWR$QGDQFKXV,UYLQJ:DWDQDEH-RH5RVHQ+HUERRSHU1HDO$GDPV0DULH6HYHULQ3HWHU6DQGHUVRQ5RJHU6WHUQ%DUU:LQGVRU6PLWK-HDQ6LPHN'LDQ%DR$EOE%HXUVGLDQVN7RQ'H0]XDQ*ULJH7DRLUWJXHV3HUH]%RE6KDUHQ5LFDUGR9LOODPRQWH'DQUHVSL6DP*UDLQJHU$QGUHD+LOO6HQHQ$QWRQLR*HRUJH*R]XP%LOO:UD$OH[-D$UPDQGR*LO-RKQ%ROWRQ)UDQN0LOOHU0D[6FKHHOH+RZD0UGDU0FD6FXNPLHHUDN$UVLD5R]HJDU-RKQ:RUNPDQ6FRWW:LOOLDPV KULV(OLRSRXORV7RP3DOPHU/LQGD)LWHKULVODUHPRQW6DO%XVFHPD'HQQLV2QHLO5RQ:LOVRQ%RE0F/HRG'DYHRFNUXP0LFKHOOH:ULJKWVRQ)UDQNKLDUDPRQWH3KLO5DFKHOVRQ7RP2U]HFKRZVNL/HQ:HLQ3HWUD*ROGEHUJ.DUHQ0DQWOR%LOO0DQWOR0DUY:ROIPDQ$QQHWWH.DZHFNL'DYLG+XQW%UXFH3DWWHUVRQ5LFK%XFNOHU'RQ:DUILHOG$QQHWWH.DH-DQLFHRKHQ%RQQLH:LOIRUG-RKQRVWDQ]D%RE/DWRQ$UFKLH*RRGZLQ%RE%URZQ7RP6XWWRQ-RKQ%UQH 'DQ*UHHQ7HUU$XVWLQ'HQLVH:RKO-LP6KRRWHU*OQLV2OLYHU)UDQFRLVH0RXO0LFKDHO+LJJLQV 5LFN3DUNHUOHP5RELQV-LP1RYDN-LP6DOLFUXS/RXLVH-RQHV %UHQW$QGHUVRQ-RH5XELQVWHLQ-DQLFHKLDQJ%RE:LDFHN-LP6KHUPDQ5RQ=DOPH%LOO6LHQNLHZLF]-DQLQHDVH3DXO6PLWK/QQ9DUOH-DQLFHDVH3DXO%HFWRQ:DOWHU6LPRQVRQ(OLRW%URZQ6U$QQ1RFHQWL6WHYH/HLDORKD3DW%OHYLQV5LFN/HRQDUGL-KDULHV/6HSHURXVH7HUU.DYDQDJK$O:LOOLDPVRQ-XQH%ULJPDQUDLJ5XVVHOO'DYH0FDLJ0HO5XEL.LD$VDPLD63WXDGXLRO77XUWRURQQH6WHYH.LP-G6PLWK3KLOLS7DQ5XV:RRWRQKXFN$XVWHQ6HDQ3KLOOLSV6KDQQRQ%ODQFKDUG-RHDVH'DQ1RUWRQ'DYH/DQSKHDU(PHUVRQ0LUDQGD0DWW%DQQLQJ5RE-HQVHQ$ODQ'DYLV3DXO1HDU$UW$GDPV%XWFK*XLFH0DUF6LOYHVWUL3HWUD6FRWHVH.HUU*DPPLO-HII0DWVXGD(OHFWULFUDRQ'LJLWDOKDPHOHRQ6HDQ3DUVRQV6NRWWLHRXQJ-DVRQ.HLWK'DYH6KDUSH(G%UXEDNHU$QWRQLR)DEHOD(GJDU'HOJDGR)UDQN' $UPDWD'DQQ.0LNL0LNH5DLFKW*HRUJHV-HDQW$Q7GUKHRZP3DHVS'RHUHQLFN6DQGX)ORUHD0DUN3RZHUV.HYLQ1RZODQ.HYLQ6RPHUV%RE+DUUDV%LOO-DDVND'DUO(GHOPDQ*LQD*RLQJ-RH4XHVDGD3ROO:DWVRQHGULF1RFRQ5RE/LHIHOG-LP/HH.HQW:LOOLDPV(ULN/DUVHQ0LFKDHO+HLVOHU *UHJ:ULJKW.LHURQ'ZHU10HOLNHRPRWROOYLQV0LNH5RFNZLW]6X]DQQH*DIIQH%UDG9DQFD3WDDW%URVVHDX$UW7KLEHUW6WHYH%XFFHOODWR7RPRNR6DLWR/RLV%XKDOLV.HYLQXQQLQJKDP-RH5RVDV.HQ%DUU-LP6WDUOLQ0LFKDHO*ROGHQ.ODXV-DQVRQ/DUU36WURPDQ+LODU%DUWD)DELDQ1DFLH]D5XULN7OHU6WHYH%XWOHU/HH.:HHYHLQNVRQUDG%HQ5DDE-LP.UXHJHU'DQD0RUHVKHDG5LFKDUG6WDUNLQJV.HYLQ7LQVOH7RP5DQH-DQ$QWRQ+DUSV%UDQGRQ3HWHUVRQ/LVD3DWULFN0DUN3HQQLQJWRQ0LNH7KRPDV0DULH-DYLQV'DQ3DQRVLDQ$O0LOJURP5LFKDUG,VDQRYH-L6PDLKGHDXQ7JHPRIRQWH-RQ%DEFRFN5LFKDUG%HQQHWW-D-QR'VHXX0UVDHU]PDDQ$O9H-RH0DGXUHLUD7LP7RZQVHQG6W0HYDHWW(+SLWFLQNVJ6FRWW/REGHOO0DUN:DLG%LOO2DNOH5RQ*DUQH7RP*UXPPHWWRPLFUDIW%UDQ+LWFKDP6PLWKDUORV3DFKHFR3KLO+XJK)HOL[-RQDWKDQ%DEFRFN$QWKRQ:LQQ5RELQ5LJJVKULVWLDQ/LFKWQHU-RH3LPHQWHO-RH$QGUHDQL-HSK/RHE-J7HDP%XFFH-RKQ'HOO3DVTXDO)HUU.ROMD)XFKVKXFN'L[RQ-DVRQ/LHELJ6DOYDGRU/DUURFD0DUN%HUQDUGR$GDP.XEHUW0DUN0RUDOHV-RVH/DGURQQ-XDQ9ODVFR+XPEHUWR5DPRVKULV%DFKDOR'DQ%URZQ6WHYH6HDJOH$QG6PLWK0RQLFD.XELQD(G%HQHV KULV6RWRPDRU/LTXLG*UDSKLFV-RKQ:DWVRQ*UHJ/DQG-+:LOOLDPV'DUUO%DQNV-RKQDVVDGD7RPP/HH(GZDUGV.LHURQ*LOOHQ-D/HLVWHQDQLFN3DTXHWWH7RGG.OHLQ7HUU'RGVRQXOO+DPQHU3DXO0RXQWV6WHYH2OLII$OEHUW'HVFKHVQH-RVHSK+DUULV5DQ-%RHHQ.MDHPOOLQ6WHYH5XGH 5DOSK0DFFKLR/HLQLO)UDQFLVX3HWH)UDQFRKULV'LFNH2VFDU*RQ%JRULUDDQ0LOOHU0LNH60LOOHU0LFDKHO6WHZDUW2SWLF6WXGLRV5LFKDUG+RULH*OHQQ+HUGOLQJ*UHJJ6FKLJLHO-D)DHUEHU6FRWW+DQQD5LFKDUGDVH6WHYH%HK0OLQLFJKDHO6WHZDUW/RXLVH6LPRQVRQ%ULDQ+DEHUOLQ*UDKDP1RODQ0LNH6WHZDUW :HV$EERW5DQGDOO*UHHQ*HUPDQ*DUFLD0LFKDHO5DQ7RP'HUHQLFN1RUP5DSPXQG+L)L'HVLJQ,DQKXUFKLOO$YDORQ6WXGLR/DUU6WXFNHU$VKOH:RRG(GGLHDPSEHOO6HDQ3-KDLYOOLLSHVU3DXQOGLG0RDWW6PLWK$$ULDHUOR2QOL/YHRWSWLUHVWL'DYLG)LQFK0DWW0LOOD0LNH0DUWV6WHYH8%LOO7DQ0DUN)DUPHU2OLYLHURLSHOKULVKXFNU$QG3DUN-RQDWKDQ6LEDO3HWH0UL0NHLOOL$JODOUQHG'H[WHU9LQHV(GJDU7DGHR-XVWLQ3RQVRU5REHUW:HLQEHUJ-RH3UXHWW.DUO6WRU5REE0F1DEE-DVRQ:ULJKW-LP0RRQH'HDQ:KLWH)UDQNKR.HQQ/RSH]'DYLG$QWKRQ.UDIW6G6KRUHV(YHOQ6WHLQ-RKQ:DUQHU9LFWRU2OD]DED'DYLG0LFKHOLQLH%UXFH-RQHV+DUUDQGHODULR7RP%UHYRRUW7RP'H)DOFRRU3HWLW-RHDUDPDJQDKULVWLQD:HLUDUOR%DUEHUL7RP0DQGUDNH3HWHU,UR0DUN%URRNV6WHYH*HUEHUDUODRQZD%UHWW%UHHGLQJODWRQ+HQUKULVWLQD6WUDLQ0DUN*UXHQZDOG6WHYH(QJOHKDUWKULVWRV*DJH3HWHU'DYLG$QWKRQDVWULOOR$QJHO0HGLQD0LNH'HRGDWR-RKQ.DOLV]5REHUWR$JXLUUH6DFDVD%REE-LDHHK*DDVUHGQHU0LFKDHOKRL6RQLD2EDFN:HVOH:RQJ%HUQDUGKDQJ0DWW5DQ/DU6WXFNHU-RQ+ROGUHGJH:LOVRQ5DPRV7DQD+RULH-DVRQ/HYLQH%HQ2OLYHU0LNHDUHUDLJKU.0LVLONHHRV3WHUNLQVDUORV$OEHUWRFUX]XHYDV%ULDQ5HEHU0DWW)UDFWLRQ:LOOLDP0HVVQHUORHEV9DO6HPHLNV /DUU+DPDDUORV0RWD-RH%HQQHWW%XG/D5RVD-(3+25.5DJV.0DUROU%DROHOOVHUV-%HQFKPDU+NL)3LURGRXORFXWLURQV7RSRZ7EG5DFKHO'RGVRQ$OODQ+HLQ3$EKGHLOLUJ-*LPUDHQQRHY]:+,/(3257$,21LFN/RZH,6ELUPDLRPQH5R%ELDHQUVFRKQLTypes of Networks6FRWW(GHOPDQ 7RQ,VDEHOOD,UHQH9DUWDQRII'RQ1HZWRQ+RZDUG%HQGHUDUO*DIIRUG$QQHWWH.DZHFNL-RKQ7DUWDJOLRQH*HQH'D(UQHVW+DUW3DEOR0DUFRV3KLO5DFKHOVRQ)UDQNKLDUDPRQWH*DVSDU6DODGLQR*HRUJH5RXVVRV-RKQRVWDQ]D6WHYH(QJOHKDUW%RE%URZQ-LP6WDUOLQ0LNH(VSRVLWR*LO.DQH-RKQ5RPLWD-59LQFHROOHWWD(GJDU'HOJDGRDURO/D*HQHRODQ*HRUJH.OHLQ/LVD3DWULFN-LP0RRQH/LQGD/HVVPDQQ5R7KRPDV5RQ:LOVRQ6WHYH'LWNR/DUU,YLH/DUU/LHEHU0LNH3ORRJ'RQ+HFN3HWUD*ROGEHUJ-RKQ%XVFHPD-HDQ6LPHN%HQ6HDQ-DFN$EHO(PEHOOLVKHUV$VVHPEOHG%RE%XGLDQVN'DYHRFNUXP)UDQN*LDFRLD$UYHOO-RQHV)UDQN0LOOHU-0'H0DWWHLV$ODQ.XSSHUEHUJ3LWWVEXUJKRPLFVOXE'DQ*UHHQ(G+DQQLJDQ-LP6DOLFUXS%RE+DOO-DFNV/LWWOH+HOSHUV'DYLG0LFKHOLQLH7RP2U]HFKRZVNL6DO%XVFHPD-RH6WDWRQ-RH6LQQRWW/HQ:HLQ*HRUJH7XVND-HUU)HOGPDQ7RP6XWWRQ'DYLG'DKDUOHV1LFKRODV0LFKDHO.HOOHKHU5R7KRPDVDV--RQDK-DPHVRQ$UWLH6LPHN6WDQ/HHKDUORWWH-HWWHU,UYLQJ:DWDQDEH-DFN.LUE'LFN$HUV3DXO5HLQPDQ6RO%URGVNKLF6WRQH6DP5RVHQ:DOO:RRG5D+ROORZD6KHULJDLO/3*UHJRU0DULH6HYHULQ1G7RODVW3DQHO3URGXFWLRQ3DVWHXS6DP*UDLQJHU+RZDUG3XUFHOO%DUU:LQGVRU6PLWK6G6KRUHV)UDQN6SULQJHU-DQLFHKLDQJ.OH%DNHU%RE:LDFHN-RKQ:HOOLQJWRQ-DFN0RUHOOL+RZDUG0DFNLH:DOWHU6LPRQVRQ7RQ'H]XQLJD*DU)LHOGV.HLWK:LOOLDPV7RP3DOPHU%RE/DUNLQ%RQQLH:LOIRUG-LP6KHUPDQ$OIUHGR$OFDOD%RE0F/HRG-RH'HOEHDWR3HWHU6DQGHUVRQ/RXLVH-RQHV'DQQ)LQJHURWK+HUERRSHU0LNH6WHYHQV6KHOO/HIHUPDQ+DUODQ(OOLVRQ$ODQ1:HHDLOV$VGDPV%LOO(YHUHWW5LFK%XFNOHU'DQ$GNLQVKULVODUHPRQW*OQLV2OLYHU'HQLVH:RKO)UDQN0F/DXJKOLQ6WDQ*ROGEHUJ'DYLG+XQW)UDQN%ROOH-HDQ,]]R-XQH%UDYHUPDQ%LOO0DQWOR-RH5RVHQ7LWOH0DUY:ROIPDQ*HRUJH3HUH]-DQLFHRKHQ'DQUHVSL'RQ:DUILHOG.HLWK3ROODUG$O0LOJURP+XJK3DOH'XII9RKODQG$UFKLH*RRGZLQ-LP6KRRWHU*HUURQZD'RF0DUWLQ'DYLG$QWKRQ.UDIW5RJHU6OLIHU-RKQ%UQH7HUU$XVWLQ.HQ.ODF]DN.ODXV-DQVRQ)LQLVKHG$UW'LYHUVH+DQGV-RH5XELQVWHLQ1HORPWRY5RJHU6WHUQ'DYLG:HQ]HO%RE6KDUHQ5LFN3DUNHU5LFDUGR9LOODPRQWH6WHYH*HUEHU5XG1HEUHVDUPLQH,QIDQWLQR7RP'H)DOFR$O*RUGRQ0DULR6HQ)UDQFRLVH0RXO(ODLQH+HLQO%RE/DWRQ'LDQD$OEHUV-LP1RYDN6WHYHQ*UDQW0DUN*UXHQZDOGKULVWLH6FKHHOH6W'HYDHYH06LWLFPKRHQOOV'RQ3HUOLQ7RS2I3DJH%UHWW%UHHGLQJ*UHJ/DURFTXH$ODQ=HOHQHW]06DDUON7%UDULSJDKQW L-RKQ%HDWW$QQ1RFHQWL$QG0XVKQVN-XOLDQQD)HUULWHU-RKQ0RUHOOL%U*LDDQUY*HDUYH-RKQ:RUNPDQ0D[6FKHHOH3DXO%HFWRQ.HQW:LOOLDPVUDLJ'%DUDUUVHIQLH$OGXFN-HII-RKQVRQ(YDQ6NROQLFN3DWULFN2OOLIIH.DWKUQ%ROLQJHU*UHJDSXOOR-RKQ]RS6WHSKHQ%-RQHV5XULN7OHU%UDG9DQFDWD%LOO2DNOH 5DOSK0DFFKLR-LP5HGGLQJWRQ0LFNH5LWWHU%RE+DUUDV(O/LRRWS%HU]RZQ0DUF6LU.HQQ/RSH]3DXO5DQ7)LQH0LNH5RFNZLW])DELDQ1DFLH]D-DPHV)U5LN/HYLQVKULV,Y5HQHH:LWWHUVWDHWWHU-RH5RVDV(G/D]HOODUL%RE0DFNLH*OHQQ+HUGOLQJKULV(OLRSRXORV)UHG)UHGHULFNV%UDG.-RFH7HUU.DYDQDJK-RK/QXN6HWD5WHRPVVD +LJJLQV2DNOH0LNH:LHULQJR/DUU+DPD+HUE7ULPSH75RPHJ0JLRHU-JRDQQHV6WHYH(SWLQJ5RQ/LP7LQVOH6FRWW0RVVRII6NROQLN((GP%LUH5QLHEVHUR'-4)UDQN/RSH] 5DPRV-RH4XHVDGD5-RQHVKDUOHV%DUQHWW3DXO$EUDPV-HIIUH0RRUH0DULDQQH/LJKWOH0DOLEX0DULH-DYLQV0LFKDHO+HLVOHU-DQ'XXUVHPD*HR7I,RVPKHUZDWRHRVG5RE7RNDU6WHYH'XWUR.HYLQ.REDVLF*LQD*RLQJ0LFKDHO+LJJLQV-RKQ/HZDQGRZVNL$ULDQH/HQVKRHN.RVVRII$QGU-HZ*3-DRTQXHHVWWHKULV0DWWKV.HYLQ:HVW:LOOLDP0HVVQHUORHEV-LP/H5HRE/LHIHOGDUORV09RDWDO6$HPQWHKLRNQV:LQQ-RH%HQQHWW -RKQ'HOO%XG/D5RVD/HQ.DPDQVNL0:PDQ$OH[DQGURY)%HQHV)1DEMLRT/DJXQD0LNH7KRPDV+6HXF]WDRQUQHR*OODD]IRIQH*)RUHUGGRHQULF3NXVUFHOO0LNH*XVWRYLFK-LP+DOO%DEFRFN'RQDOG+XGVRQ%HQ5D-DREKQ.DOLV]D0QDFUHLD'/DHEFDFWDUL0LNH0DUWV2YL+RQGUX-R*HUDQWD0YDLHOLHKUPL6XVDQUHVSL6WHZDUW-RKQVRQ7RP*ULQGEHUJ5LFK5DQNLQ7LP']RQ%ROOHUV0DUVKDOO(OOLH'H9LOOH0LNH'HRGDWR-HII0DWVXGD.H7YLPLQR7WKLQVO'H]RQ-RH3LPHQWHOKULVWLDQ/LFKWQHU3KLO+XJK)HOL[-RH$QGUHDQL.ROMD)XFKV-HSK/RHE 6FRWW/REGHOO$QWKRQDVWULOOR6WHYH%XFFHOODWR%REELHKDVH0LNH.DQWHURYLFK7RP%UHYRRUW.HYLQ6RPHUV$QJHO0HGLQD6FRWW.REOLVK0DUN:DLG$OODQ-'DRFQRE5VLFHRQ)UDQN5REELQV/HH(OLDV6FRWW.ROLQV36PLWK$OH[6FKRPEXUJ6DP.DWRDUO3RWWV%LOO6LHQNLHZLF]0DWW5DQ$QGDQFKXV'HQQLV2QHLO5DQG'/DRQIIQLF7LHKURPDV-HDQ0DUF/RIILFLHU%XWFK*XLFH'DQ'DKULVWLH6FKHHOHDV0D[6FKHHOH*HUDUG-RQHV/DUU$OH[DQGHU+DUUD5QRGQHO)DUUHLRQ]+XGVRQ'DQ3DQRVLDQ'DQQ%XODQDGL.HOO3RDUYWH%VUHRVVHDX'DYLG5RVV/LQGD)LDWHUODRQZDDUROH6HXOLQJ0DUF6LOYHVWUL)UDQN7XUQHU:HUQHU5RWK%HY%HYHULGJH%UXFH3DWWHUVRQ-HII$FOLQ-RKQ9HUSRRUWHQ0LFKDHO)OHLVKHU6WHYH*DQ3DW%URGHULFN.DUHQ50RDEQWORDURVHOOD.HUU*DPPLO3HWUD6FRWHVH0LNHDUOLQ*HR6UJHHQ*HQR$]$QXQGPWURHQDLR+LOO$UW$G5DLPFNV/HRQDUGL6WHYH/HL3DODRXKOD6PLWK.HQ)5HLGFXKQDLUHGLZ+LFR]ZHOO2WWR%LQGHU9LQFH$ODVFLD$O$YLVRQ-RQDWKDQ%DEFRFN7LP7RZQVHQG3HWHU'DYLG3DVTXDO)HUU-RH0DGXUHLUD5REHUWR$JXLUUH6DFDVD$GDP.XEHUW,DQKXUFKLOORPLFUDIW0DUN3RZHUV6FRWW+DQQD0DUN0DRPUD6OHPVLWK5RELQ5LJJV-DH*DUGQHU7HD5PLF%KDXUFGFH6WDUNLQJV-(3+25.RU6HGOPHLHU$OO7KXPEVUHDWLYH 19. Types of NetworksNeuron Network of Mouse Millennium Simulation (2005)Largest astronomical simulation ever on the structure andevolution of galaxies in the universe.25 TB of data and 20 million galaxies 20. Use Cases Recommendation engines (avoidrelational N-JOIN or self-JOIN) Ranking/credibility (GooglesPageRank) Path finding (shortest, longest,mutual friends) Social (friendship, following, keyconnectors) 21. Graphs Node/Verticy: An entity that can have zero or more edgesconnected to it.1 2 3 Edge: An entity which connects two nodes. May bedirected or undirected1 2A B 22. Adjacency Matrix If graph is undirected, the adjacency matrix is symmetric Thus, transposition of matrix is the same graph 23. Adjacency Matrix Some graphs have different types or dimensions of edges 24. Property GraphsAttribute Valueid 2name BobAttribute Valueid E3type knowssince 2013-09-01Attribute Valueid 4name AliceAttribute Valueid 3name EveAttribute Valueid E2type knowssince 2013-09-01Attribute Valueid E4type siblingtwins trueAttribute Valueid 1name IvanAttribute Valueid E1type cousinseparation 1 25. Traversals Breadth-first 3, 2, 4, 1 Depth-first 3, 2, 1, 4 Breadth-first anddepth-first searchcan be combined. Filtering Ability to filter/sortpaths in traversal Aggregating Ability to aggregate/count properties as traversal occurs and affecttraversal with result of aggregation (e.g. power-grid load distr.) Backtracking Leave marker in traversal and come back to it when certain criteria ismet in a lower step1234 26. TINKERPOPGraph Framework 27. Tinkerpop A comprehensive, open-source graph framework(http://www.tinkerpop.com/)Property graphmodel that is DBagnostic. A kind ofJDBC for graphs.Data flow API forprocessing graphs.Underlyingcomponent forgraph traversalsDSL for traversingproperty graphs.Implemented inJSR-223.Maps betweendomain objects andthe graphs nodesand edges. LikeORM for graphs.Collection ofcommon graphanalysis algorithmsfor propertygraphs.Exposes anyblueprints graphvia a uniformRESTful API.Blueprints Pipes GremlinFrames Furnace Rexster 28. Tinkerpop Stack Different components all buildon each other Provides abstraction fromHTTP layer, to object mappinglayer, to traversal scripting, topluggable graph API Blueprints underpins the stackmaking it all DB agnostic Blueprints implementations: Neo4j, Sail, OrientDB, Dex *) Accumulo, ArangoDB, Bitsy,FluxGraph, FoundationDB,InfiniteGraph, MongoDB, Oracle-NoSQL, TitanDB * - Implemented by 3rd party 29. Tinkerpop - Rexter Provides REST and binary (RexPro - grizzly) protocols Flexible extension model (e.g. ad-hoc Gremlin queries) Server-side stored procedures (Gremlin) Browser-based interface (Dog House) Command-line tool for interacting with API Pluggable security SPARQL plugin to work against Sail graphs (OpenRDF) More information:https://github.com/tinkerpop/rexster/wiki 30. Tinkerpop - Furnace Collection of industry-standard algorithms fortraversing or analyzing graphs. Network generators (by clique or degree distribution) Search: A*, Breadth-first, Depth-first Shortest path Bellman-Ford (like Dijkstras but can handle neg. paths) PageRank Degree Distribution More information:https://github.com/tinkerpop/furnace/wiki 31. Tinkerpop - FramesMore Information: https://github.com/tinkerpop/frames/wiki 32. Tinkerpop - Pipes Dataflow framework for process graphs. Computational step becomes a node and an edge is acommunication channel between steps. Pipes are then chained and nested. Custom pipes can be created. Pipe types: Transform emit transformation of object Dozens of different types of transforms Filter decide whether to include/exclude object in traversal ~20 different types of filters sideEffect include object but produce side-effect from it ~15 different types of sideEffects (e.g. group, count, table, tree) Branch decide which step to take next in traversal Several different branching options 33. Tinkerpop - Blueprints Like JDBC but for graphs. Common API for Property Graphs which are very flexible Foundational component for Pipes, Gremlin, Frames,Furnace, and Rexster Supports transactions (if underlying DB engine does) Multi-threaded transactions supported Format readers/writers (GML, GraphML, GraphSON) More Information:https://github.com/tinkerpop/blueprints/wiki 34. Tinkerpop - Gremlin Graph traversal scripting language. Works against Blueprints API and is compiled intoFrames data-flows. Both native Java and Groovy (JSR-223) supported. Step library (https://github.com/tinkerpop/gremlin/wiki/Gremlin-Steps) Transform emit transformation of object Dozens of different types of transforms Filter decide whether to include/exclude object in traversal ~20 different types of filters sideEffect include object but produce side-effect from it ~15 different types of sideEffects (e.g. group, count, table, tree) Branch decide which step to take next in traversal Several different branching options 35. SQL Gremlin (secret decoder ring)Query SQL GremlinGet all users select*fromusersg.V(type,user).map()Get user names selectnamefromusersg.V(type,user).nameGet user names/ages selectname,agefromusersg.V(type,user).transform({[name:it.getProperty(name),age:it.getProperty(age)]})Get distinct user ages selectdistinct(age)fromusersg.V(type,user).age.dedup()Get oldest user selectmax(age)fromusersg.V(type,user).age.max() 36. SQL Gremlin (secret decoder ring)Query SQL GremlinSelect by equality select*fromuserswhereage=35g.V(type,user).has(age,35).map()Select by comparison select*fromuserswhereage21g.V(type,user).has(age,T.gt,21).map()Select by multiple criteria select*fromuserswheresex=Mandage25g.V(type,user).has(age,T.gt,25).has(sex,M).map()Order by age(switch a and b to do asc)select*fromusersorderbyagedescg.V(type,user).order({it.b.getProperty(age)=it.a.getProperty(age)}).map()Paging select*fromusersorderbyagedesclimit5offset5g.V(type,user).order({it.b.getProperty(age)=it.a.getProperty(age)})[5..10].map() 37. SQL Gremlin (secret decoder ring)Query SQL GremlinJoin selectusers.*fromusersinnerjoingroupsonusers.gId=groups.idwheregroups.name=devsg.V(type,groups).has(name,dev).in(inGroup).map()Join-on-join-on-join SELECTTOP(5)[t14].[ProductName]FROM(SELECTCOUNT(*)AS[value],[t13].[ProductName]FROM[customers]AS[t0]CROSSAPPLY(SELECT[t9].[ProductName]FROM[orders]AS[t1]CROSSJOIN[orderdetails]AS[t2]INNERJOIN[products]AS[t3]ON[t3].[ProductID]=[t2].[ProductID]CROSSJOIN[orderdetails]AS[t4]INNERJOIN[orders]AS[t5]ON[t5].[OrderID]=[t4].[OrderID]LEFTJOIN[customers]AS[t6]ON[t6].[CustomerID]=[t5].[CustomerID]CROSSJOIN([orders]AS[t7]CROSSJOIN[orderdetails]AS[t8]INNERJOIN[products]AS[t9]ON[t9].[ProductID]=[t8].[ProductID])WHERENOTEXISTS(SELECTNULLAS[EMPTY]FROM[orders]AS[t10]CROSSJOIN[orderdetails]AS[t11]INNERJOIN[products]AS[t12]ON[t12].[ProductID]=[t11].[ProductID]WHERE[t9].[ProductID]=[t12].[ProductID]AND[t10].[CustomerID]=[t0].[CustomerID]AND[t11].[OrderID]=[t10].[OrderID])AND[t6].[CustomerID][t0].[CustomerID]AND[t1].[CustomerID]=[t0].[CustomerID]AND[t2].[OrderID]=[t1].[OrderID]AND[t4].[ProductID]=[t3].[ProductID]AND[t7].[CustomerID]=[t6].[CustomerID]AND[t8].[OrderID]=[t7].[OrderID])AS[t13]WHERE[t0].[CustomerID]=N'ALFKI'GROUPBY[t13].[ProductName])AS[t14]ORDERBY[t14].[value]DESCg.V('customerId','ALFKI').as('customer).out('ordered').out('contains').out('is').as('products).in('is').in('contains').in('ordered').except('customer).out('ordered').out('contains').out('is').except('products).groupCount().cap().orderMap(T.decr[0..5].productName 38. Gremlin Resources Tinkerpop resources https://github.com/tinkerpop/gremlin/wiki/Basic-Graph-Traversals https://github.com/tinkerpop/gremlin/wiki/Gremlin-Steps https://github.com/tinkerpop/gremlin/wiki/Using-Gremlin-through-Java https://groups.google.com/forum/#!forum/gremlin-users https://github.com/tinkerpop/gremlin/wiki/SPARQL-vs.-Gremlin http://markorodriguez.com/2011/08/03/on-the-nature-of-pipes/ http://sql2gremlin.com/ http://gremlindocs.com/ Groovy http://groovy.codehaus.org/Beginners+Tutorial http://groovy.codehaus.org/Collections Misc http://www.fromdev.com/2013/09/Gremlin-Example-Query-Snippets-Graph-DB.html http://markorodriguez.com/2011/06/15/graph-pattern-matching-with-gremlin-1-1/ 39. GREMLINDemo Dataset Lab 40. Tinkerpop - Gremlingremling=TinkerGraphFactory.createTinkerGraph()==tinkergraph[vertices:6edges:6]gremling.V.count()==6gremling.E.count()==6gremling.v(1)==v[1]gremling.v(1).map=={age=29,name=marko}gremling.v(1).outE==e[7][1-knows-2]==e[8][1-knows-4]==e[9][1-created-3]gremling.v(1).outE('knows')==e[7][1-knows-2]==e[8][1-knows-4]gremling.v(1).outE('knows').map=={weight=0.5}=={weight=1.0} 41. Tinkerpop - Gremlin//getverticiesknownbymarkogremling.v(1).outE('knows').inV==v[2]==v[4]//getpropertiesofverticiesknownbymarkogremling.v(1).outE('knows').inV.map=={age=27,name=vadas}=={age=32,name=josh}//filterbythoseolderthan30gremling.v(1).outE('knows').inV.filter{it.age30}.map=={age=32,name=josh}//justgetnamegremling.v(1).outE('knows').inV.filter{it.age30}.name==josh//findnodeswhoknowsomeoneolderthan30gremling.V.as('x').outE('knows').inV.has('age',T.gt,30).back('x').map=={age=29,name=marko} 42. Tinkerpop - Gremlin//findedgeswithweight.5gremling.E.filter{it.weight0.5}==e[10][4-created-5]==e[8][1-knows-4]//findedgesw/weight.5frommarkogremling.E.filter{it.weight0.5}.as('x').outV.has('name',T.eq,'marko').back('x')==e[8][1-knows-4]//findnodescreatedbyothernodesgremling.V.as('x').inE('created').back('x').map=={name=lop,lang=java}=={name=ripple,lang=java}gremling.E.filter{it.label=='created'}.inV.dedup().map=={name=lop,lang=java}=={name=ripple,lang=java}//findnodescreatedbymorethan1nodegremling.E.filter{it.label=='created'}.inV.groupCount().cap()=={v[3]=3,v[5]=1}//findnodescreatedbymarkosfriendsgremling.v(1).outE('knows').inV.outE('created').inV.map=={name=ripple,lang=java}=={name=lop,lang=java} 43. Tinkerpop - Gremlin//addsomenewnodesgremling.addVertex([name:'bob',age:'60'])==v[0]gremling.addVertex([name:'eve',age:'40'])==v[7]gremling.addVertex([name:'timmy',age:'5'])==v[8]//addsomeedgesgremling.addEdge(g.v(0),g.v(7),'friend)==e[13][0-friend-7]gremling.addEdge(g.v(0),g.v(8),'child')==e[14][0-child-8]gremling.V.filter{it.name=='bob'}.outE('child').as('x').inV.filter{it.name=='timmy'}.back('x')==e[14][0-child-8]gremling.removeEdge(g.e(14))==nullgremling.V.filter{it.name=='bob'}.outE('child').as('x').inV.filter{it.name=='timmy'}.back('x')//noresults 44. Tinkerpop - Gremlin//previouslygremling.addVertex([name:'bob',age:'60'])==v[0]gremling.addVertex([name:'eve',age:'40'])==v[7]gremling.addEdge(g.v(0),g.v(7),'friend')==e[13][0-friend-7]//queryforedgegremling.v(0).outE==e[13][0-friend-7]//removevertex(autoremovesorphanededge)gremling.removeVertex(g.v(7))==nullgremling.v(0).outE//noresultsgremling.e(13)==null 45. TITANA Distributed Graph Database 46. Titan Graph Database Optimized to work against billions of nodesand edges Theoretical limitation of 2^60 edges and 1^60 nodes Works with several different distributed DBsincluding Cassandra and HBase Supports many concurrent users doingcomplex graph traversals simultaneously Native integration with Tinkerpop stack Supports integration with searchtechnologies such as Lucene andElasticsearch Created by Thinkaurelius(http://thinkaurelius.com/) 47. Titan Distributed Architecture TitanDB can integrate with distributed architectures in afew different waysNative Remote Embedded Put Rexter in front toallow RESTful access Connects remotely tocluster Can scale size as faras cluster can Possible processingbottleneck TitanDB and Rexter run oneach node in the cluster Can run on same JVM Considerableperformance/scalabilityimprovement Connects remotelyto cluster (or local) Can scale size asfar as cluster can Native Titan API Possibleprocessingbottleneck 48. Titan Indexing Standard index Internal to Titan Very fast but only supports exact matches External index Use indexing engine external to Titan (Lucene or Elasticsearch) Supports range queries Lucene Limited to only one machine (small-sized datasets) Also as richer set of search features (than Elasticsearch) Elasticsearch Distributed Not as feature-filled as Lucene 49. Distributed Titan Limitations/Gotchas Limitations which are present but which are scheduled tobe remedied Property indexes must be created before property is ever used Unable to drop indices Types cannot be changed once created Gotchas Multiple graphs on same backend requires specific configurationsper graph Ghost vertices certain concurrency circumstances can leavetraces of vertices. Recommendation is to allow this and periodicallyclean them up 50. Titan Graph Database - Gremlingraph vertices edges propertiesG = (V , E , ) 51. Titan Graph Database - Gremlingraph vertices edges propertiesG = (V , E , ) 52. Titan Graph Database - Gremlingraph vertices edges propertiesG = (V , E , )Application 53. Titan Graph Database - Gremlingraph vertices edges propertiesG = (V , E , )Application 54. Titan Graph Database - Gremlingraph vertices edges propertiesG = (V , E , )Application 55. DATA MODELINGEXAMPLEA Blogging Application 56. Bloggie Blog Requirements Create users, posts, and comments Retrieve all posts for a user Retrieve posts by time range Retrieve all comments for a user Retrieve all comments for a post, sorted by vote Retrieve the top N posts, sorted by vote User can only vote *once* on a post or comment 57. Get CassandraTitan https://github.com/thinkaurelius/titan/wiki/Downloads (0.3.2 stable)$$TITAN_LOCATION/bin/gremlin.sh,,,/(oo)-----oOOo-(_)-oOOo-----gremling=newTinkerGraph();==tinkergraph[vertices:0edges:0]gremlin 58. Modeling Entities (User, Post, Comment) Theres no one way to model this. General rules to follow: 1-N relationships can be modeled as one node with N edges pointing toother nodes 1-1 relationships can be modeled as a simple edge between two nodes M-N relationships are just more edges It is important to categorize the different types of edges since manydifferent types of edges will connect to a single node Dont shy away from attaching properties to edges. Remember that edgesare just a query-able as nodes. A common practice is to tend to model actions as edges andactors/artifacts as nodes Denormalize to minimize traversals 59. Users, Posts, Comments 60. Retrieve Users Posts Lets create a user and post Link them together Retrieve the user and their postsgremling.addVertex([type:'user',email:'bob@test.com',name:'Robert',password:'asdf'])==v[0]gremling.addVertex([type:'post',guid:'21EC2020-3AEA-1069-A2DD-08002B30309D',title:'HelloWorld',text:'Myfirstpost!',userDisplayName:'Bob'])==v[1]gremling.addEdge(g.v(0),g.v(1),'postAuthor')==e[3][0-postAuthor-1]gremling.V.has('type','post').as('posts').inE('postAuthor').outV.has('email','bob@test.com').back('posts').map()=={guid=21EC2020-3AEA-1069-A2DD-08002B30309D,text=Myfirstpost!,title=HelloWorld,userDisplayName=Bob,type=post} 61. Retrieve Posts by Time Range Add timestamp property to post Query by rangegremling.V.has('guid','21EC2020-3AEA-1069-A2DD-08002B30309D').has('type','post').sideEffect({it.createTimestamp=1383726500});==v[1]gremling.V.has('createTimestamp',T.gt,1383726400).has('createTimestamp',T.lt,1383726600).map()=={guid=21EC2020-3AEA-1069-A2DD-08002B30309D,createTimestamp=1383726500,text=Myfirstpost!,title=HelloWorld,userDisplayName=Bob,type=post} 62. Retrieve All Users Comments Add comment Link to author and to postgremling.addVertex([type:'comment',guid:'3F2504E0-4F89-11D3-9A0C-0305E82C3301',text:'Ilikeit!',userDisplayName:'Sally',createTimestamp:1383736500])==v[4]gremling.addEdge(g.v(1),g.v(4),'postComment')==e[5][1-postComment-4]gremling.addVertex([type:'user',email:'sally@test.com',name:'Sally',password:'qwerty'])==v[6]gremling.addEdge(g.v(6),g.v(4),'commentAuthor')==e[7][6-commentAuthor-4]gremling.V.has('type','comment').as('comments').inE('commentAuthor').outV.has('email','sally@test.com').back('comments').map()=={guid=3F2504E0-4F89-11D3-9A0C-0305E82C3301,createTimestamp=1383736500,text=Ilikeit!,userDisplayName=Sally,type=comment} 63. Retrieve top N posts by vote Create postVote edge andaggregated votes count in post Query and sort by votesgremling.addEdge(g.v(6),g.v(1),'postVote',[date:1383726600])==e[8][6-postVote-1]gremling.V.has('type','post').has('guid','21EC2020-3AEA-1069-A2DD-08002B30309D').sideEffect({it.votes=1})==v[1]gremling.addVertex([type:'post',guid:'21EC2020-3AEA-1069-A2DD-08002B30309E',createTimestamp:1383726600,title:'LearningGremlin',text:'Gremlinisneat.',userDisplayName:'Bob',votes:2])==v[9]gremling.V('type','post').order({it.b.getProperty('votes')=it.a.getProperty('votes')}).transform({['title':it.getProperty('title'),'votes':it.getProperty('votes')]})[0..5]=={title=LearningGremlin,votes=2}=={title=HelloWorld,votes=1} 64. Retrieve Post Comments Sorted by Vote Similar to post votesgremling.addEdge(g.v(0),g.v(4),'commentVote',[date:1383726700])==e[10][0-commentVote-4]gremling.V.has('type','comment').has('guid','3F2504E0-4F89-11D3-9A0C-0305E82C3301').sideEffect({it.votes=1})==v[4]gremling.addVertex([type:'comment',guid:'3F2504E0-4F89-11D3-9A0C-0305E82C3302',text:'Thanks.',userDisplayName:'Bob',createTimestamp:1383736500])==v[11]gremling.addEdge(g.v(1),g.v(11),'postComment')gremling.addEdge(g.v(0),g.v(11),'commentAuthor')gremling.v(1).outE('postComment').inV.order({it.b.getProperty('votes')=it.a.getProperty('votes')}).map()=={guid=3F2504E0-4F89-11D3-9A0C-0305E82C3301,createTimestamp=1383736500,text=Ilikeit!,votes=1,userDisplayName=Sally,type=comment}=={guid=3F2504E0-4F89-11D3-9A0C-0305E82C3302,createTimestamp=1383736500,text=Thanks.,userDisplayName=Bob,type=comment} 65. User Can Only Vote Once Could enforce using externalunique indexes Or do 2-step incrementing ingremlin (small chance of dups)gremlinuser=g.v(0);post=g.v(1);if(post.inE('postVote').outV.has('email',user.email).count()==0){g.addEdge(user,post,'postVote',[date:newDate().getTime()]);if(post.getProperty('votes')!=null){post.votes++;}else{post.votes=1;}}==1gremlin//samecommandabove==null 66. Graph Visualization 67. Areas Not Covered Map/Reduce Gremlin has its own built-in M/R API Indexing Titan currently has limitation requiring all indexes are created up-front Integration with other backends HBase, Oracle Berkeley DB, Hazelcast, Persistit Detailed full-text search through external indexes Graph analytics engine (Faunus) Deep dive into gremlin query language andGroovy Seriously, theres a TON there. 68. Referenceshttp://sql2gremlin.com/http://www.tinkerpopbook.com/ - http://www.tinkerpop.com/https://github.com/thinkaurelius/titan/wiki/Getting-Startedhttps://groups.google.com/forum/#!forum/gremlin-usershttps://groups.google.com/forum/#!forum/aureliusgraphshttp://thinkaurelius.com/ 69. THANK YOU{email : calebjones@gmail.com,website : http://calebjones.info,twitter : @JonesWCaleb}