Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS...
Transcript of Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS...
![Page 1: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/1.jpg)
Ben Porterfield Founder, VP Engineering
Business Analytics: Asking the Right
Questions
![Page 2: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/2.jpg)
B U S I N E S S I N T E L L I G E N C E
Operational Control How many sales did I do today?
Understand & Improve Experience Are users engaging? Do they like the new features?
Make business decisions Should we start delivering in a new city?
![Page 3: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/3.jpg)
—ANDREW LEONARD Salon
“Data indicated that the same subscribers who loved the original BBC production also gobbled down movies starring Kevin Spacey or directed by David Fincher”
![Page 4: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/4.jpg)
1 1 Tracking Data
2 Storing Data
3 Merging Data (ETL)
4 Retrieving Data
5 Analysis & Decision Making
The Analytical Process
![Page 5: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/5.jpg)
Tracking Data
![Page 6: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/6.jpg)
What To Track?
Views Clicks
In-app actions
E v e n t
Users Orders
Inventory
T r a n s a c t i o n a l
![Page 7: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/7.jpg)
Embed in product process
Server-side too
Taxonomy matters
Tracking - Event Data
Every new feature should come with events
Lots of non-transactional events happen on server
Big flat event space becomes unwieldy
![Page 8: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/8.jpg)
Storing Data
![Page 9: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/9.jpg)
Go with SQL
Store all states
Keep it clean
Storing - Transactional Data
NoSQL could be a burden long-term
Even offline processes Messy schema = complicated analytics
![Page 10: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/10.jpg)
—MICHAEL ERASMUS Back-end Engineer, Buffer
“We were relying on MongoDB…while it was easier for developers to play with the data, it became a hurdle for other team members.”
![Page 11: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/11.jpg)
Own it Use eco-system too
Store all the IDs
Storing - Event Data
Or, at a minimum, be able to get it
Lots of great SaaS event platforms
Need to be able to correlate events to transactions
![Page 12: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/12.jpg)
Merging Data
![Page 13: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/13.jpg)
O t h e r D a ta?
Tra n s a c t i o n a l D a ta E ve n t D a ta
Ra w Q u e r i e s B i z- Us e r To o l s
You should combine transaction and event data, +more
Use an analytical database
Redshift is current leader
Difficult - data is heavy
A p p l i ca t i o n
WITH user_order_activity AS ( SELECT user_id, age FROM ORDERS GROUP BY user_id) SELECT AVG(users.age) as average_age_of_purchaser FROM user_order_activity LEFT JOIN users ON user_order_activity.user_id = users.user_id
![Page 14: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/14.jpg)
SUMMARY
Traditional Approach
OLAP / Data Summaries S I L O E D
Restricted Q&A L I M I T E D
I
G
M N
L
D
Q
B A
P
R
S
Q
D I F F I C U LT & C O M B E R S O M E ETL - Heavy Transformation
E N D US E R B I T E A M E T L T E A M E D W T E A M
W A N T T O A S K N E W Q U E S T I O N S ?
A B
? C F
X
E B
A EVENT DATA
TRANSACTIONAL DATA
![Page 15: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/15.jpg)
Modern Approach
3 R D PA RTY A P P
A P I
A N Y D E V I C E
Transformation at Query F L E X I B L E
Anywhere for Anyone A C C E S S I B L E C O N S O L I D A T E D
Simple Extract & Load
I
G
M N
D
Q
A
P
R
S
Q
T U W
X
G Q
U
S
A
Z
Data Modeling Layer A G I L E
D A T A T E A M E N D U S E R S
Data Model
- name: first_purchasers type: single_value base_view: orders measures: [orders.first_purchase_count] listen: - name: orders_by_day_and_category title: "Orders by Day and Category" type: looker_area base_view: order_items
I N N O V A T I O N
TRANSACTIONAL DATA
EVENT DATA
Z
B
Q A A Z
M P P | R E D S H I F T | I M P A L A
![Page 16: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/16.jpg)
Asana’s Data Infrastructure
![Page 17: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/17.jpg)
Retrieving Data
![Page 18: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/18.jpg)
—TODD LEHR SVP Engineering, Dollar Shave Club
“We have a developer name Juan and any reports we needed would flow through him.”
![Page 19: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/19.jpg)
—TODD LEHR SVP Engineering, Dollar Shave Club
“When he got backlogged, our team didn’t have access to the data immediately.”
![Page 20: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/20.jpg)
—ANNIE CORBETT Business Intelligence Analyst, Venmo
“Initially whenever we were asked for data, we would write a custom script…”
![Page 21: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/21.jpg)
—ANNIE CORBETT Business Intelligence Analyst, Venmo
“..and then repeat this process whenever the product team wanted to extend the timeframe.”
![Page 22: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/22.jpg)
What’s selling? What colors and sizes is it selling in?
What’s getting returned? Is there a particular size/color?
Is there a product people buy first that increases their likelihood of becoming a repeat customer?
Questions from a retail buyer at e-commerce store:
![Page 23: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/23.jpg)
Get them the tool
Decisions vs. data science
Game-changing insights
Self-Service is Key
People with questions are running the businesses.
“Should we open a new market in Maine?”
Don’t only come from analyst group
![Page 24: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/24.jpg)
Analysis and Decision Making
![Page 25: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/25.jpg)
1 1 Clearly define success metrics
2 Look for low-hanging fruit
3 Go one level deeper
Analysis and Decision Making
![Page 26: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/26.jpg)
Analysis and Decision Making: Success Metrics
![Page 27: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/27.jpg)
Focus on desired outcome What do you want users to experience?
Measure Engagement In most cases this is first-line business analytics
Measure Retention Are people coming back?
S U C C E S S M E T R I C S
![Page 28: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/28.jpg)
H O W T O T R A C K E N G A G E M E N T ?
Not with page views Usually not even with time on page
Upworthy’s attention minutes Lots of indicators (mouse, video, etc)
Looker’s approximate usage Any event in 2 minute window
![Page 29: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/29.jpg)
Deriving Approximate Usage
SELECT event.created_at AS created_date, event.user_id as user_id, COUNT(*) AS count, COUNT(DISTINCT CONCAT( CONCAT(event.user_id,'|',event.user_browser_id), FLOOR(UNIX_TIMESTAMP(event.created_at)/(60*2)) ) )*2 AS approximate_usage_in_minutes FROM event GROUP BY created_date, user_id
created_date user_id count approximate_usage
1/10 1 123 100 minutes
1/10 2 228 50 minutes
1/10 3 45 80 minutes
![Page 30: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/30.jpg)
Derived Tables
SELECT orders.user_id as user_id COUNT(*) as lifetime_orders MIN(orders.created_at) as first_order MAX(orders.created_at) as latest_order COUNT(DISTINCT DATE_TRUNC('month’)) as
distinct_months_with_orders FROM orders GROUP BY user_id
Transactional
Event
Analytical
Derived Table
Insights
![Page 31: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/31.jpg)
Start simple
Most useful at row level
Great for cohorts and sessionization
Derived Tables
Subselects until slow, SQL on cron works surprisingly well
Don’t roll up data, pre-compute facts
Tiered derived dimension vs. some other metric
![Page 32: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/32.jpg)
Derived Table - User Order Facts
SELECT orders.user_id as user_id COUNT(*) as lifetime_orders MIN(orders.created_at) as first_order MAX(orders.created_at) as latest_order COUNT(DISTINCT DATE_TRUNC('month’)) as
distinct_months_with_orders FROM orders GROUP BY user_id
user_id lifetime_orders first_order latest_order distinct_months_with_orders
1 10 1/10/15 2/14/15 2
![Page 33: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/33.jpg)
Derived Table + Sourcing
![Page 34: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/34.jpg)
Derived Table + Sourcing
![Page 35: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/35.jpg)
Churn Users that will likely never do X again
Usage How likely to purchase if they do X
Time to transaction How long till first X
Retention Are users coming back
??? Invent a metric
Repeat buyers What’s different about them
![Page 36: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/36.jpg)
Pay/Charge Mistake.
![Page 37: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/37.jpg)
It was clear some users were
accidentally paying instead of charging, but it wasn't clear
how widespread the problem was and
whether it was worth prioritizing a fix
Inventing Metrics
![Page 38: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/38.jpg)
Identify behavior
Measure % of population
Experiment
Inventing Metrics
Can be good or bad – just something possibly significant
Who is doing this thing? Ability to play with numbers is crucial
![Page 39: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/39.jpg)
Analysis and Decision Making: Low-hanging Fruit
![Page 40: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/40.jpg)
This is the kind of very visual, very
data‑driven piece of analysis that
helps us think, "Is opening the sale at
noon the right decision?”
???
Low-hanging Fruit
![Page 41: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/41.jpg)
Out of stocks are huge detractors from
the customer experience - it sucks ordering something and then not getting
it - as well as revenue we failed to
capture
Low-hanging Fruit
![Page 42: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/42.jpg)
Analysis and Decision Making: One Level Deeper
![Page 43: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/43.jpg)
While this immediate insight might have led us to focus on small groups, this didn’t match our expectations of
people planning an outing on a Friday
night, prompting us to look further.
One Level Deeper
2 3 4
Time To Book
2 3 4
Group Size
![Page 44: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/44.jpg)
We analyze all the platform data
available - When someone attempts to sign, completes the signup, pushes an app, has spend,
etc
One Level Deeper
![Page 45: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/45.jpg)
Even though it looks like we were
having nice incremental
growth, looking into the details we see
some things to look into further
One Level Deeper
![Page 46: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/46.jpg)
Don’t confuse an increase in a metric with success.
![Page 47: Business Analytics: Asking the Right Questions · Data Modeling Layer AGILE DATA TEAM END USERS Data Model - name: first_purchasers type: single_value base_view: orders measures:](https://reader034.fdocuments.us/reader034/viewer/2022042223/5ec98b51b83f5f77ec2d49ed/html5/thumbnails/47.jpg)
Put data in analytical database
Give business users tool
Define success metrics
Takeaways
Make sure it’s fast and speaks SQL
Empower them to answer their own questions
Focus on engagement and retention