1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris

Product Analytics 101A introductory view on analytics for

product design, development and management

George @ Voulgaris .net @gevou

George Voulgaris, Ph.D. - Product Analytics 101 1st Prod.Active MeetUp, Athens, Dec 10th, 2014

How many products will you be selling in 12 months from today?


Short tale instead of intro “Sales Forecast: The Quest for Holy Grail” - or - “the 50 shades of finance”


1. Tools


Advertising

Which ad is driving the most sign ups?Advertising tools allow you to add event tracking and ad retargeting to better understand your site visitors, where they came from, and if your campaigns are working.

Analytics

What are customers doing before they churn?Analytics tools allow you to visualize your data, see how customers use your product, and identify trends, so you can make better business decisions.

Marketing Automation

Did that newsletter send traffic to your site?Email marketing tools make it easy to send newsletters and other marketing materials to your customers. Integrating with these tools makes it easy to setup drip campaigns for your product.

Sales & Business

What sales leads and customers need the most attention today?Sales & business tools help you track leads throughout the sales funnel, manage customer relationships, and run outbound campaigns.

Customer Support

How can I answer any questions my customer have?Customer support tools help you communicate with current customers to answer their questions, provide support, and identify recurring issues.

Developer Tools

Why couldn’t new users sign up for accounts yesterday and how much money did we lose?Performance and error reporting tools help you improve the health and stability of your application by automatically logging and tracking errors and alerting you when there is any downtime.

User Testing

What is the best homepage copy for driving sales?Testing tools give you insight into how people are using your product, so you can optimize your design. Experiment with variations using A/B tests and understand behavior with heat maps to see what version works best.

source: segment.com

http://segment.com


Advertising

Which ad is driving the most sign ups?Advertising tools allow you to add event tracking and ad retargeting to better understand your site visitors, where they came from, and if your campaigns are working.

Analytics

What are customers doing before they churn? Analytics tools allow you to visualize your data, see how customers use your product, and identify trends, so you can make better business decisions.


Did that newsletter send traffic to your site? Email marketing tools make it easy to send newsletters and other marketing materials to your customers. Integrating with these tools makes it easy to setup drip campaigns for your product.

Sales & Business

What sales leads and customers need the most attention today? Sales & business tools help you track leads throughout the sales funnel, manage customer relationships, and run outbound campaigns.

Customer Support

How can I answer any questions my customer have? Customer support tools help you communicate with current customers to answer their questions, provide support, and identify recurring issues.

Developer Tools

Why couldn’t new users sign up for accounts yesterday and how much money did we lose? Performance and error reporting tools help you improve the health and stability of your application by automatically logging and tracking errors and alerting you when there is any downtime.

User Testing

What is the best homepage copy for driving sales? Testing tools give you insight into how people are using your product, so you can optimize your design. Experiment with variations using A/B tests and understand behavior with heat maps to see what version works best.

source: segment.com

http://segment.com


few examples


inspectlet.com


inspectlet.com

Replay user activity* for the ultimate insight on how users actually use your app

*be careful with your T&Cs and the possibility of sensitive data


mixpanel.com


Add Push notifications, Surveys, Revenue analysis…

mixpanel.com


… + pretty much any data you want to send from your App

mixpanel.com


Advanced Segmentation, In-depth profiling, Direct Engagement / Communication

intercom.io


… combined with automated messaging.

intercom.io


“Q: Is this the:

a) best b) recommended c) endorsedd) cutest

set of tools?”

A: These are just examples. I do not endorse any - I just happen to use them at the moment. Do your own research for best fitness to your case.


With all those beautiful apps, tools, analyses, graphs, functions, automations you should be able to build the perfect products and businesses! Right?


What tools do you use? (or fooling around with) How and Why? What are your challenges at the moment?


2. What matters? (because observation is not a business by itself… mostly)


why


There is a great difference in focus for startups and established businesses. What is it?


There is a massive difference in focus between Customer Discovery and Customer Development stages. What is it?


where


Inbound Marketing

website & purchase funnel in product

UX

Customer Support

Customer engagement inc. churn, retain, upsell

Company

The Long Funnel gets even longer


Inbound Marketing


UX

Customer Support


Company


Layered Inbound Marketing diagram source: Inbound Marketing Handbook - MarketingSherpa


Inbound Marketing


UX

Customer Support


Company

Advertising

Analytics


Sales & Business

Customer SupporDeveloper Tools

User Testing

BI


Layered Inbound Marketing diagram source: Inbound Marketing Handbook - MarketingSherpa


when


Which stages of the entire journey / funnel are you currently monitoring in your business? How?


What matters most depends on the product / business maturity stage and the user engagement stage.

There are few frameworks that can help.


How do your users become aware of you? SEO, SEM, widgets, email, PR, campaigns, blogs ...

Do drive-by visitors subscribe, use, etc? Features, design, tone, compensation, affirmation ...

Does a one-time user become engaged? Notifications, alerts, reminders, emails, updates...

Do you make money from user activity? Transactions, clicks, subscriptions, DLC, analytics...

Do users promote your product?Email, widgets, campaigns, likes, RTs, affiliates...

Acquisition Activation Retention

Revenue Referral

Dave McClure’s Pirate Metrics

Source: http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version

http://www.slideshare.net/dmc500hats/startup-metrics-for-pirates-long-version


Lean Analytics (A.Croll & B. Yoskovitz recommended reading)

Stage

EMPATHY I’ve found a real, poorly-met need that a reachable market faces.

STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for.

VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives.

REVENUE The users and features fuel growth organically and artificially.

SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.

Gate

The

five

stag

es


Eric Rees’ (The Lean Startup) Sustainable growth is a requirement for any business.

“New customers come from the actions of past customers”

The three growth engines: Virality Make people invite others as a necessary side effectRate of growing StickinessKeep people coming back / addictive / network effects / lock-in Rate of churn (compound)Price Reinvest revenue of past to get new customers. Rate of expenditure

Both AARRR & Lean Analytics frameworks resonate with…


how


Lean Analytics (Adapted version from the book by A.Croll & B. Yoskovitz recommended reading)

Pick a KPI

Draw a line in the sand

Identify ways to improve this KPI

Make a guess Find relevant patterns in your

user data

Design experiments

(e.g. A/B)

Implement what seems to work

Find correlations between those

patterns & users

data available

no data

Analyze the behavior of the KPI

Line-in-the-sand was crossed

Line-in-the-sand was NOT crossed

Give up or pivot Go back to Discovery.

Talk to customers

Try again


Lean Analytics (Adapted version from the book by A.Croll & B. Yoskovitz recommended reading)

Pick a KPI

Draw a line in the sand

Identify ways to improve this KPI

Make a guess Find relevant patterns in your

user data

Design experiments

(e.g. A/B)

Implement what seems to work

Find correlations between those

patterns & users

data available

no data

Analyze the behavior of the KPI

Line-in-the-sand was crossed

Line-in-the-sand was NOT crossed

Give up or pivot Go back to Discovery.

Talk to customers

Try again

Identify business objectives & decision processes


what


Good metrics are:

• Actionable Accessible Auditable

• Metrics need to change a behavior - or need to hurt

• Use rates

• One Metric That Matters: based on model and stage


Some terms to familiarise yourself with:

• Qualitative versus quantitative metrics Qualitative metrics are unstructured, anecdotal, revealing, and hard to aggregate; quantitative metrics involve numbers and statistics, and provide hard numbers but less insight.

• Vanity versus actionable metrics Vanity metrics might make you feel good, but they don’t change how you act. Actionable metrics change your behavior by helping you pick a course of action.

• Exploratory versus reporting metrics Exploratory metrics are speculative and try to find unknown insights to give you the upper hand, while reporting metrics keep you abreast of normal, managerial, day-to-day operations.

• Leading versus lagging metrics Leading metrics give you a predictive understanding of the future; lagging metrics explain the past. Leading metrics are better because you still have time to act on them.

• Correlated versus causal metrics If two metrics change together, they’re correlated, but if one metric causes another metric to change, they’re causal. If you find a causal relationship between something you want (like revenue) and something you can control (like which ad you show), then you can change the future.


Typically your focus will go through the fives stages of product/business maturity

Stage

EMPATHY I’ve found a real, poorly-met need that a reachable market faces.

STICKINESS I’ve figured out how to solve the problem in a way they will keep using and pay for.

VIRALITY I’ve found ways to get them to tell their friends, either intrinsically or through incentives.

REVENUE The users and features fuel growth organically and artificially.

SCALE I’ve found a sustainable, scalable business with the right margins in a healthy ecosystem.

Gate

The

five

stag

es


More terms to familiarise with:

Average Revenue Per User (ARPU): Revenues / Users

Churn: users abandoned (month)

Lifetime: 1 / churn

Lifetime Value (LTV): ARPU * Lifetime

Customer Acquisition Cost (CAC): Total monthly acq costs / new monthly customers (e.g. 3xCAC < LTV)


Even more terms to familiarise yourself with:

Invitation Rate = Invited / inviters

Acceptance Rate = Signed-up / Invited total

Viral Coefficient = Invitation Rate * Acceptance Rate (e.g. Users * VC - churn > 1)


Empathy

Stickiness

Virality

Revenue

Scale

E-commerce SaaS MediaMobile

appUser-gencontent

2-sidedmarket

Interviews; qualitative results; quantitative scoring; surveys

Loyalty, conversion

CAC, shares, reactivation

Transaction, CLV

Affiliates, white-label

Engagement, churn

Inherent virality, CAC

Upselling, CAC, CLV

API, magic #, mktplace

Content, spam

Invites, sharing

Ads, donations

Analytics, user data

Inventory, listings

SEM, sharing

Transactions, commission

Other verticals

(Money from transactions)

Downloads, churn, virality

WoM, app ratings, CAC

CLV, ARPDAU

Spinoffs, publishers

(Money from active users)

Traffic, visits, returns

Content virality, SEM

CPE, affiliate %, eyeballs

Syndication, licenses

(Money from ad clicks)

Lean Analytics Important: drawing base-lines (in the sand)

A considerably more information-rich version is available at: bit.ly/BigLeanTable


What KPIs do you use at the moment? What is the example of the impact of these in your business?


3. What matters? Money-talks


TL;DR

1.Make more money than what you spend *


TL;DR

1. Make more money than what you spend *

2. The (dream of) Hockey-stick growth includes a long painful period with resource stagnation (you are working on high growth ventures right?)


8

THE STARTUP FUNDING SPECTRUM

+

NET

CASH

PO

SITI

ON

Closed

ALPHA

Public

BETA

SCALE - STAGE

1st Revenue

Redistributed for educational purposes with permission from Maria Dramalioti Taylor, Managing Partner, x.Million Capital Ventures


TL;DR


2. The (dream of) Hockey-stick growth includes a long painful period with resource stagnation

3. Finance people want numbers, charts, proof; be it investors or CFOs

4. Planning and playing with other people’s money involves accountability. Be accountable for the right thing


The “Business Plan” Approach (top-bottom)

• Do background / buy research

• Check macro - trends

• Make projections and forecasts Initially to convince… and find yourself accountable 12 months later…

The Data Driven Approach (bottom-up)

Deductively and quantitatively prove that:

• (Startups *) …you are narrowing-in a valid Business Model

• (Existing businesses / products ) …your key growth assumptions still hold-true; optimize


What planning, forecasting and optimization processes do you use at the moment? What are your challenges with these at the moment?


Hopefully by now we all agree that:


2. The (dream of) Hockey-stick growth includes a long painful period with resource stagnation

3. Playing with other people’s money involves accountability. Be accountable for the right thing

4.Data-driven planning & decisions (*) reduce risk


4. Perfect


A few common problems when implementing a data-driven business

1. The asterisk in decisions was on purpose. data-driven decisions need analytics AND decision processes

2. The 2nd asterisk in decisions. What’s hard about pivoting? alignment, transparency, knowledge

3. Vanity metrics … and politics. Someone will look ugly.

4. Resourcinginability to explain / understand the importance

5. Culture (I know all) / Ethics (Morale in changing situations) Change is tough

George @ Voulgaris .net @gevou

fallingelephant.com visionmobile.com techtalkscentral.com

1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris

Presentations & Public Speaking

Transcript of 1st Prod.Active Meetup: "Product analytics 101" by George Voulgaris