The Power of Machine Learning and Graphs
-
Upload
franz-inc-allegrograph -
Category
Technology
-
view
18 -
download
0
Transcript of The Power of Machine Learning and Graphs
![Page 1: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/1.jpg)
The power of
machine learning
and graphs
March 2017, Webinar
Jans Aasman
allegrograph.com
![Page 2: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/2.jpg)
Contents
• Cognitive computing overview
• Why we put the results of analytics and machine learning
back in the graph
• Some examples first
• The main loop for data science with graphs
• Two AG open source packages for data science
• AllegRo: an R interface to AllegroGraph
• Python-Agraph: a Python interface to AllegroGraph (installable with
Anaconda)
• Future work
![Page 3: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/3.jpg)
10 years ago
Structured Data
![Page 4: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/4.jpg)
7 years ago
Structured Data Unstructured Data
![Page 5: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/5.jpg)
7 years ago
Structured DataUnstructured Data
NLP, Key/Value stores, NoSQL, Big Data, Hadoop, IoT
![Page 6: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/6.jpg)
4 to 5 years ago
Structured DataUnstructured Data and IOT
KnowledgeDomain knowledge
Linked Open Data
Vocabularies
Taxonomies/Ontologies
![Page 7: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/7.jpg)
New #1: Learning. Feed output of data
science back into data infrastructure
Structured
Data
Unstructured
Data and IOT
KnowledgeDomain knowledge
Linked Open Data
Vocabularies
Taxonomies/Ontolo
gies
Probabilistic
Inferences.
![Page 8: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/8.jpg)
New # 2: Everything in one (distributed)
Semantic Graph
Structured
Data
Unstructured
Data
KnowledgeDomain
knowledge
Linked Open Data
Vocabularies
Taxonomies/Ontol
ogies
Probabilistic
Inferences.Unstructured
Data and IOT
KnowledgeDomain
knowledge
Linked Open Data
Vocabularies
Taxonomies/Ontol
ogies
![Page 9: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/9.jpg)
AKA: Cognitive Computing
Structured
Data
Unstructured
Data
KnowledgeDomain
knowledge
Linked Open Data
Vocabularies
Taxonomies/Ontol
ogies
Probabilistic
Inferences.Unstructured
Data and IOT
KnowledgeDomain
knowledge
Linked Open Data
Vocabularies
Taxonomies/Ontol
ogies
![Page 10: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/10.jpg)
Current state of analytics
Usually the output of data science results in reports and publications but
• No formal trace where the data came from
• No formal link to the actual methods you used,
or who did it, or when you did it
• Cannot be compared to earlier results
• Cannot be used as building blocks for further research
• In general : the output is not queryable and discoverable
![Page 11: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/11.jpg)
Enriching the graph with analytics
True Machine Learning
• results become data
• build layers of analytics
• Formal provenance for all results. Links to the actual data and methods
you used, or who did it, or when you did it, or even why you did it.
• Important for compliance and auditability
• Important for explaining why you took certain actions
• Historical analysis
• Results become queryable and discoverable
• The analytics fits in the total infrastructure of structured/unstructured and
knowledge.
![Page 12: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/12.jpg)
Odds ratio
![Page 13: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/13.jpg)
![Page 14: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/14.jpg)
Association rules
![Page 15: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/15.jpg)
![Page 16: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/16.jpg)
K-means clustering
![Page 17: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/17.jpg)
![Page 18: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/18.jpg)
![Page 19: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/19.jpg)
In the ecommerce world: find similar objects based on > 10 criteria, including description, product codes, pictures, etc
![Page 20: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/20.jpg)
The main loop for data science with graphs
AllegroGraph
SPARQL
dataframe
R Python SPARK
results
![Page 21: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/21.jpg)
AllegRo: work with AG directly from R
• Line1
![Page 22: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/22.jpg)
AllegRo: work with AG directly from R
• The entire AllegroGraph API directly available from R
• Create/open databases, add/delete/query, SPARQL 1.1
• Create data frames directly from getStatement or SPARQL queries
• Will work with free AllegroGraph trial version
![Page 23: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/23.jpg)
Quick tutorial demo
![Page 24: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/24.jpg)
![Page 25: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/25.jpg)
Agraph-python (available on github but please use Anaconda to install)
• Line1
![Page 26: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/26.jpg)
Title
• Line1
![Page 27: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/27.jpg)
Quick tutorial demo Python & Anaconda
create an environment in Anaconda2
• conda create -name testenv -c franzinc agraph-python numpy pandas matplotlib
activate environment
• source activate testenv
if you want to install a particular version of agraph-pythonif you want to install a particular version of agraph-python
• conda install -c franzinc agraph-python=6.2.0
or, if an older version is already installed:
• conda update -c franzinc agraph-python
![Page 28: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/28.jpg)
IRIS example
• Classify 3 types of Irises: Setosa, Virginica, Versicolour
• based on petal length, petal width,
sepal length, sepal width.
![Page 29: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/29.jpg)
Future Webcasts
• Formal ontologies to represent analytic output
• Using Knime as a data science framework
• Distributed AllegroGraph & SPARK
![Page 30: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/30.jpg)
![Page 31: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/31.jpg)
SDL Super-
Learner
![Page 32: The Power of Machine Learning and Graphs](https://reader035.fdocuments.us/reader035/viewer/2022062412/58ede1eb1a28ab6a498b4685/html5/thumbnails/32.jpg)
Conclusion
• Adding data science results back to the graph is an valuable new
paradigm
• We make it really straight forward to do data science with AllegroGraph
• Try it