Post on 08-Jul-2015
description
What Does Big Data Really Mean for your Business?
Leslie Hawthorn Director of Developer Relations
leslie.hawthorn@elasticsearch.com @lhawthorn
Even our small data has “Big Data”
Every day we create more…
User Generated Machine Generated
• Tweets • Blogs • Books • Notes • Emails • Comments
• Logs • Events • Statistics • Alerts • Metrics
And it’s growing…
0
1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
2010 2016
Exab
ytes
Year
machine social voip enterprise
But there is a gap
Storing Data
Actionable Insight
Data From Any Source
Instantly Analyze
Actionable Insights
The promise of “Big Data”
Massive effort in cleaning the data.
Takes weeks to analyze.
Insights come too
late.
The reality
“I’m a data janitor.” Senior Director of Data Science, major Apache Hadoop distro
Why?
Balancing Structure and Scale
SQL Rela)onships Structure Can it Scale?
SQL Yes Yes No
NoSQL No Yes Yes
Hadoop No No Yes
We got excited about Scale, but now suffer for lack of Structure.
Which means: We can store a lot of data, but it takes a ton of work to
search, analyze and get insights out of it.
Hadoop Also Brings: Intensive Engineering and a Large Upfront Investment
Where we’re going
Your Big Data pipeline
• Incredibly crowded landscape in both the open source and proprietary worlds
• Most big data tools were built for people with Massive Data
• Tools now focus on making working with both structured & unstructured data easier
Why open source? http://w
ww.flickr.com
/photos/16462767@N00/3984507322/
flexibility
Data From Any Source
Instantly Store, Search
& Analyze
Actionable Insights
Elasticsearch: Search, Analytics, Logging & Visualization
Logstash Elasticsearch Kibana
The ELK Stack
Data Democratization
Big Data for our business
• Measuring our community health metrics
Demo time….
[see appendix for screenshots from demo]
Be a data superhero, not a janitor!
Questions?
Leslie Hawthorn Director of Developer Relations
leslie.hawthorn@elasticsearch.com @lhawthorn
Appendix – Demo Slides