Building a Data Driven Organization
-
Upload
it-weekend -
Category
Data & Analytics
-
view
123 -
download
1
description
Transcript of Building a Data Driven Organization
@AbeStanway
BUILDING A DATA DRIVEN ORGANIZATION
1. why 2. how
1. why 2. how
“DATA IS THE NEW GOLD”
Predict the future!
Retain Customers!
Grow the business!
Recommend content!
Drive Engagement!
unclear paths to $$$
IN IT, It’s clear.
Data are Dollars
. IT Working = +$$$ IT Not Working = -$$$ . .
How do you know if your IT is working
right now?
How do you know if you are earning money
right now?
KPIs. What are they?
Etsy: Literally a Money per
second Graph
Planet Labs: Literally an Images
per day graph
What are the Kpis for kips?
$ per second
items bought per
second page requests
per second
database queries per
second memcache hits per second
fread() per second
If you do not have the data about your infrastructure,
it is already broken.
LEsson:
Test driven development ->
data driven development
Without data, you are flying blind
How do you know you’re hitting your goals?
How do you know if You’re making the right ones in the
first place?
How do you know if you’re still in
business?
How do you even know what planet
you live on?
Assumptions are death
You need data, yo.
1. why 2. how
1. collect 2. analyze 3. ??? 4. Profit!
1. collect 2. analyze 3. ACT 4. Profit!
data that cannot be acted upon should
not be analyzed.
You are running a business, not an
art museum
You are Trying to Win the market, not
a fields medal
This can be disappointing
Data SCientist?
Data Scientist? Realist.
Find a way to Align your employees intellectual curiosity
With your Real business needs.
LEssoN:
Train your organization
you need a data culture.
“It’s not shipped until it’s monitored”
“If you are not looking at Dashboards, you are
not doing your job”
Building instrumentation and watching dashboards
are hard And Time consuming
App code -> statsD -> Graphite -> Dashboards -> Insightsby hand by hand by hand by hand by hand
Developers just want to code
Let’s automate
Which is easier to automate?
Insights or data collection?
Insights are sexy and fun
Collection is hard And unsexy
Collection is hard And Boring
Collection is hard And unsexy
We’re on track to have excellent
automated insights
anomaly detection
App code -> statsD -> Graphite -> Dashboards -> Insightsby hand by hand by hand AUTOMATIC!AUTOMATIC!
(…if only we had the data)
How do we automate data collection?
currently have ganglia, New relic,
collectD, etc
NOT WHAT WE NEED
they provide data about your raw machines, not your
CUSTOM DEVELOPED TECHNOLOGY And Application level logic
Healthy servers don’t make you money. Healthy services do.
enter LARIMAR
Full disclosure: this is my new PROJECT
!
we’re going to talk about it because i’m pretty excited and the
beta is opening up soon.
LARIMAR uses raw machine metrics to infer App level architecture and inform developers about problems
A service:cpu resources
disk io PCAP data
ports Used syscalls
A service:cpu resources
disk io PCAP data
ports Used syscallsservice fingerprint
MACHINE LEARNING
A service:cpu resources
disk io PCAP data
ports Used syscallsABNORMAL BEHAVIOR
MACHINE LEARNING
a system:service service service service service
MACHINE LEARNING
graphical system fingerprint
a system:service service service service service
MACHINE LEARNING
Abnormal, holistic system behavior
Larimar automates both analysis And
Relevant data collection
so your developers can focus on coding
and acting on insights
No configuration!
App code -> statsD -> Graphite -> Dashboards -> Insightsby hand AUTOMATIC!AUTOMATIC!AUTOMATIC! AUTOMATIC!
1. collect 2. analyze 3. ACT 4. Profit!
organizational shifts are still
needed to inspire ACTION on Data
but ACTION is easier to inspire when there is lots of data and lots
of insight everywhere
Create a culture where your developers create
these kinds of tools
When a data driven mindset is the default,
tools will build themselves.
Thanks! @abestanway
!
!
larimar.io @larimarhq