@NYUEntrepreneur

Startup School:Metrics that Matter

Andy MossNYU Entrepreneurial InstituteEntrepreneur-in-ResidenceDirector, Blackstone LaunchPad at NYU

@NYUEntrepreneur

If you don’t know where you’re going, any road will get you there. – George Harrison

@NYUEntrepreneur

Agenda

u Why Monitor Metrics?u Metrics that Matter to Startupsu Metrics that Matter to Investorsu Resourcesu Discussion

@NYUEntrepreneur

Various types of Metrics

@NYUEntrepreneur

Why Monitor Metrics?

u Benefits u Allows objective tracking of progress (or lack

thereof)u Aligns team around identifiable goalsu Enables clarity of making decisionsu Distinguishes signal from noise

u Concernsu Management may focus of the metric and not real

business u Beware of “vanity metrics” that have no real value

@NYUEntrepreneur

@NYUEntrepreneur

ARC Method

Alignment | Results | Communication

u Alignment with…u Investorsu Employees u Customers

u Results – identify, track, measure results that matter…

u Communication - open communication tends to engender trust. Regular reporting shows realistic and follow through and creates accountability.

@NYUEntrepreneur

Metrics that Matter to Startups

Processu What really affects your business changes over timeu Determining the right metrics and monitoring techniques

that fit your business may take experimentationu Change/evolve during various stages of life cycle

Motivationu Help improve business decisionsu On the margin make investors happy

@NYUEntrepreneur

Metrics at Various Stages

@NYUEntrepreneur

One Metric that Matters (OMTM)Ben Yoskovitz, Lean Analytics

A single number that you care the most about at the current stage of your startup

1. Answers the most important question you have2. Forces you to draw a line in sand 3. Clearly state goals4. Focuses the entire company5. Inspires a culture of experimentation

OMTM will change. It’s not a single number that matters throughout your startup’s existence

@NYUEntrepreneur

Does it support the big picture?

All areas of a business should integrate up to the core goals.

Two most common core goals1. Solve the customers problem2. Be a viable business (make a profit,

eventually)

@NYUEntrepreneur

@NYUEntrepreneur

One Metric that Matters (OMTM)

@NYUEntrepreneur

What Makes a Good Metric?

1. Rate or ratio is better than an absolute or cumulative value. “New users per day” is better than “Total users.” Absolute numbers tend to be vanity metrics

2. Comparative to other time periods, sites, or segmentsKey is Cohort Analysis where you track a metric over different groups of people, typically over different periods of time

3. Simple and easy to understandOtherwise, people won’t remember it and discuss it

4. “Accounting” metrics – makes predictions more accurateFor board, investors, media, employees. Most of these are wildly inaccurate and aspirational. People want to see you be reasonable.

5. “Experimental” metrics – will significantly change behaviorUsed to optimize the product, pricing, or market

@NYUEntrepreneur

How to Pick OMTM

1. Type of Businessu Transactional: transaction funnel metricsu Collaborative: engagement funnel metricsu SaaS: LTV, CAC, Magic Numberu Media: page views, CTR, Conversionu Games: Avg Revenue Per User (ARPU), upsellu Apps: Daily Usage (DAU), Monthly Usage (MAU), Uninstalls

2. Stage of Businessu Seed: more qualitative than quantitative

u If you’re going to be qualitative; have a plan and show people you know how to transform this into something more quantitative.

u Series A and Beyond: more quantitative than qualitative

@NYUEntrepreneur

Example: Hospital ReadmissionsHospital Readmissions: If a patient is readmitted to a hospital within 30 days of being discharged it is typically very costly, and historically we know the outcomes are much worse.

If you have a digital health company that proposes to impact this, what are some things you might measure?• Number of readmissions before using the solution• Number of readmissions while using the solution• Cost of readmissions• Diagnosis of the patient• Medical complications of the patient• Measures of usage on the solution itself (length, type, etc.)• Engagement scores of people who use it

• Patients• Care providers

@NYUEntrepreneur

Metrics that Matter to Investors

u “Directional confirmation” of core hypotheses and risksu Engagement: u Unit Economicsu Go to Market Strategyu Distribution Model

u In hyper-competitive, noisy Seed universe, Investors want early signs of breakout potential and excitement

u VCs monitor these to inform next investment decisionu “Killing It:” Preempt the Series A before anyone else

finds outu “Good+/-:” Wait and see more cards, positive biasu “Poor:” Wait to see if something changes, help instigate

changes

@NYUEntrepreneur

Metrics that Matter to Investors

Invest to show breakout potential

Invest when have broken out

Focusonkeymilestonesandrelevantgrowthmetrics

ExampleSeedgraduates

RelevantMetric(monthlygrowthrate)

Crashlytics 47%

SeedA 37%

SeedB 32%

Seed C 38%

Track growthrelativetopeersandinternaldataset

ExampleSeriesA/Bcompanies

RelevantMetric(monthlygrowthrate)

CompanyA 38%

Company B 82%

CompanyC 18%

Stackdriver 54%

MarketPotentialTeam BusinessModel

IPPosition HowDisruptiveFinancing NetworkEffects

@NYUEntrepreneur

Metrics that Matter to Investors

@NYUEntrepreneur

Metrics that Matter to Investors

Sep-11 Dec-11 Mar-12 Jun-12 Sep-12

@NYUEntrepreneur

Resources

u CAC & LTV – David Skoku The Magic Number – Will Priceu AARRR!!! -- Dave McClure

@NYUEntrepreneur

CAC & LTV

u The common interpretation of CAC and LTV ratio:< 3.0 step back and look at your marketing spend> 3.0 your business is healthy

u One arrow, but coarse indication of “good business”u Other reasons to monitor it though – looking ahead:

uBuilding for profits considered more important than delivering todayuTraditional models penalize SaaS companies for near term growth

uArgues for companies to drive LTV higher (i.e. increase price and lower churn) while containing CAC and to strive for adding customers as quickly as practical.

@NYUEntrepreneur

CAC & LTV

@NYUEntrepreneur

The Magic Number

u Provides insight into the effectiveness of previous quarter Sales and Marketing spend on MRR growth

u To calculate:QRev[X] = Quarterly Recurring Revenue (QRR) for period XQRev[X-1] = QRR for period preceding XExpSM[X-1] = Sales & Marketing Expense for period preceding X

Magic Number = (QRev[X] – Qrev[X-1])*4/ExpSM[X-1]

@NYUEntrepreneur

The Magic Number

u Example:

u James indicates the following in interpreting the MN:< 0.75 step back and look at your business > 0.75 start stepping on the gas because your business is primed

to leverage spend into growth. > 1.5 call him immediately

Q1 Q2 Q3Revenue ($K) 1,000 1,200 1,500 S&M Exp ($K) 800 900

Magic Number 1.00 1.33

@NYUEntrepreneur

AARRR – Get, Keep, Grow…

• Acquisition: users come to the site from various channels

• Activation: users enjoy 1st visit: "happy" user experience

• Retention: users come back, visit site multiple times

• Referral: users like product enough to refer others

• Revenue: users conduct some monetization behaviorAARRR!

@NYUEntrepreneur

AARRR

@NYUEntrepreneur

AARRR

1. ACQUISITION

SEOSEM

Apps & Widgets

Affiliates

Email

PR Biz Dev

Campaigns, Contests

Direct, Tel, TV

Social Networks

Blogs

Domains

Website.com

Emails & widgets

Campaigns, Contests

Biz DevAds, Lead Gen, Subscriptions, etc

Homepage / Landing Page

Product Features

Emails & Alerts

Blogs, Content

System Events & Time-based Features

Customer Lifecycle / Conversion Behavior

@NYUEntrepreneur

AARRR Executed

@NYUEntrepreneur

AARRR

Category User Status Conv % Est. Value

Acquisition Visit Site(or landing page, or external widget)

100% $.01

Acquisition Doesn't Abandon (views 2+ pages, stays 10+ sec, 2+

clicks)

70% $.05

Activation Happy 1st Visit(views X pages, stays Y sec, Z clicks)

30% $.25

Activation Email/Blog/RSS/Widget Signup(anything that could lead to repeat visit)

5% $1

Activation Acct Signup(includes profile data)

2% $3

Retention Email Open / RSS view -> Clickthru 3% $2Retention Repeat Visitor

(3+ visits in first 30 days)2% $5

Referral Refer 1+ users who visit site 2% $3Referral Refer 1+ users who activate 1% $10Revenue User generates minimum revenue 2% $5Revenue User generates break-even revenue 1% $25

@NYUEntrepreneur

Example: Cognuse & Venture CatalystA tablet based system for educating and empowering patients and families, coordinating care, and improving outcomes in Intensive Care Units and beyond.

u How do we work with this company to maximize their odds of success and do this in practice?

u What we focus on with Cognuse:u Partnerships with academic centers: finding good partners really matters, a

lot.u Jointly doing the research to prove the tech works!

u These are jointly created and shared metrics that track results and provide concrete evidence

u Trade show participation, where customers who are already primed for this are convening.

u Referral and incentive programs. If they are early partners or customers, make it worthwhile to them, and turn them into promoters. Incentivize them..

@NYUEntrepreneur

Example: Cognuse & Venture CatalystI can’t do everything at once….

• Keep a prioritized list of things you • Know you need to do at some point and when you

think you’ll need to do it.• Want to do at some point and at what point it might

make more sense to do it.

Make metrics that collect data and try to find ways to efficiently collect it. It’s ok to be a little inefficient at first, just write out what it would take to be more efficient and hold onto that for later.

@NYUEntrepreneur

Summary

@NYUEntrepreneur

Discussion??