Measure Right the First Time - Infusionsoft Partnercon
Transcript of Measure Right the First Time - Infusionsoft Partnercon
MEASURE RIGHT THE FIRST TIME
Infusionsoft Partnercon 2012
vuurr.com
WHAT WE’LL COVER
• What you need• How to track• How to design experiments• What to experiment on• Putting it all together
TOOLS
TOOLS YOU MUST HAVE
• Google Analytics (or SiteCatalyst, or similar)• Optimizely (or similar, but you need killer
statistics)• Call tracking => analytics platform of choice• Link builder appropriate for your analytics
platform of choice• AdWords/AdCenter/Facebook properly
configured (if advertising)
DETERMINE YOUR MODEL
CPA ROI MARGIN
Minimize the cost of
each customer
acquisition
Maximize revenue for every dollar
spent
Minimize ad spend as a percent of revenue
DETERMINE YOUR MODEL
CPA ROI
Ad Spend ($)Conversion (qty)
Revenue - SpendSpend
all in dollars ($), answer is %
Revenue - SpendRevenue
all in dollars ($), answer is %answer is in dollars ($) per
conversion
MARGIN
DETERMINE YOUR MODEL - FINDING ROI & MARGIN
MARGIN
ROI
DETERMINE YOUR MODEL - FINDING CPA (MANUALLY)
Completed Conversions
Total Cost
Advertising => AdWords => Campaigns => Site Usage
Advertising => AdWords => Campaigns => Clicks
DETERMINE YOUR MODEL - FINDING CPA (AUTOMATICALLY)
• Use the Google Analytics API to query for total number of conversion by source & medium, and then properly divide them into the total marketing expense for each source & medium.
• Use a third party tool
TRACKING
INSTALLING ANALYTICS
• Follow your vendor’s instructions
• Don’t just dump the tracking code in and “go”
• Setup proper funnels, events, goals, and filters
• Know the difference between accounts, properties, and profiles
• Attach GWT and AdWords (if applicable)
• Track ALL OF THE THINGS
GOALS & ECOMMERCE
• Assign relevant goal values. A video view != purchase
• Track eCommerce as accurately as possible
• Explicit funnels are only useful if the supporting steps are relevant and provide information - use sparingly
TRACKING CAMPAIGNS
Campaign
Name/Content
Source Medium
• Every click is valuable
• Every traffic source matters
• Proper campaign tracking is crucial to adequately gauging traffic value
PROPER URL TAGGING
TRACKING RICH MEDIA: VIDEOS
• Instantiate video with YouTube/Vendor JavaScript API
• Track state changes, namely Plays and Completes
• On these changes, push GA events
TRACKING RICH MEDIA: SOCIAL BUTTONS
_gaq.push(['_trackSocial', network, socialAction, opt_target, opt_pagePath]);
Instantiate by subscribing to edge.create and message.send
Google Analytics handles for you
Instantiate by subscribing to the ‘tweet’ intent event
TRACKING RICH MEDIA: FORM ABANDONMENT
• Track full and empty exits as events
• Determine fields causing abandonment
TRACKING OFFLINE: PHONE CALLS
• Build GA gif request and HTTP GET on phone call
• Push calls as pageviews ( /call/+14805551234/abcdefg )
• Use Twilio & Twimlbin or another 3rd party service
• Consider filtering out those pageviews from main profile
• Setup goals in Google Analytics to fire on calls
• Give each ad, landing page, social source, different number
TRACKING PAID CLICKS
• Properly configure AdWords to push to Analytics
• Verify all cost data is applied to Analytics account
• Put MSN AdCenter {AdId} & {Keyword} in destination URL, then use AdCenter API to pull the cost and associate with relevant clicks from Analytics API
EXPERIMENT DESIGN
INTERNET MARKETING IS ABOUT EXPERIMENTATION
y = f( x1, x2, x3, . . . . xn )
y CTRCTR
CRCR
GoalsGoals
X1= X2= X3= Xn=
Page or Ad CopyPage or Ad Copy
Page LayoutPage Layout
Button Color & SizeButton Color & Size
Other FactorsOther Factors
DESIGNING EXPERIMENTS - ONE FACTOR AT A TIME
Let’s run an experiment where we test two different button colors.
Run Button Color ( x1 ) CTR ( y )
1 Green 5.2%
2 Orange 7.7%
Factors are tested in series, one at a time. This is extremely time intensive, requiring many tests for very gradual improvement.
DESIGNING EXPERIMENTS - FULL FACTORIAL DESIGN
Let’s run an experiment where we test two different button colors, button positions, and button labels simultaneously.
This tests multiple changes in parallel to find optimal combination of factors (x variables) to optimize results (y variable).
3 FACTORS WITH 2 VARIATIONS EACH
Number of Unique Tests = 23 = 8
DESIGNING EXPERIMENTS:FULL FACTORIAL DESIGN
Run Color (x1) Position (x2) Text (x3) CTR (y)
1 Green Left Try Now 5.2%
2 Orange Left Try Now 7.7%
3 Green Right Try Now 9.3%
4 Orange Right Try Now 12.7%
5 Green Left Buy Today 21.2%
6 Orange Left Buy Today 19.3%
7 Green Right Buy Today 14.9%
8 Orange Right Buy Today 17.7%
DESIGNING EXPERIMENTS: OFAT VS. FULL FACTORIAL
ONE FACTOR AT A TIME
IterativeSimple to Setup
Easy to Measure Results
SLOW & GRADUALIMPROVEMENT
VS.FULL FACTORIAL
More Complex to SetupRequires more visitor
data to test all combinations
QUICK & DRASTIC IMPROVEMENT
DESIGNING EXPERIMENTS:STATISTICAL SIGNIFICANCE
Preliminary results - is orange better?
Button Color ( x1 )
Visitors Clicks CTR ( y )
Green 38 2 5.2%
Orange 39 3 7.7%
Too small of a sample size leads to incorrect conclusions.
DESIGNING EXPERIMENTS: STATISTICAL SIGNIFICANCE
It turns out our original conclusion of CTR was incorrect.
Button Color ( x1 )
Visitors Clicks CTR ( y )
Green 8,238 486 5.9%
Orange 7,893 734 9.3%
Too large of a sample size wastes time, effort, & money.
SO WHAT QUANTITY OF VISITORS IS ENOUGH?
DETERMINING MINIMUM SAMPLE SIZE
x = Sample mean
μ = Population mean
Z = Number of standard deviations above mean
α = Confidence distance from 100%σ = Standard deviation
n = Minimum sample size
E = x - μ
E = Z α/2 σ
√n
DETERMINING MINIMUM SAMPLE SIZE
α/2 - α/2 Z
DETERMINING MINIMUM SAMPLE SIZE (CONTINUED)
E = Z α/2 σ√n
Standard Deviation
Minimum Sample Size
n = [Z α/2 σE
]
A/B TESTING SOFTWARE
TESTING
EVERY SINGLE CLICK HAS A COST ASSOCIATED WITH IT
KNOW YOUR LEVERS
• Product Pricing• Organic Rankings• Ad bids, placement,
content• Email subject lines &
content• Social content• Site Content
FUNNEL ANALYSIS & OPTIMIZATION
• Improve Quality Score• Split Test Ad Copy• Adjust Keyword Bids• Increase Organic Rankings• Increase Social Reach
• Split Test Landing Page Copy
• Improve Call to Action• Experiment with Deep
Linking• Look at Bounce Rates• Look at Cart Abandonment
IMPRESSIONSIMPRESSIONS
CONVERSIONCONVERSIONSS
(Sales/Leads)(Sales/Leads)
CLICKS(Site Visits)
Traffic Sources drive impressions & clicks
Optimizing those sources get more clicks
Optimized landing pages & site content increase conversionsCR
CTR
• SEM, SEO, Social, Email, & More
FUNNEL ANALYSIS & OPTIMIZATION
EMAILEMAIL
PPCPPC
ORGANIC ORGANIC SEARCHSEARCH
SOCIALSOCIAL
OFFLINE OFFLINE MARKETINGMARKETING
PHONE PHONE CALLSCALLS
AFFILIATESAFFILIATES
ALL OF THESE SOURCES HAVE ALL OF THESE SOURCES HAVE LEVERS THAT MANIPULATE THEIR LEVERS THAT MANIPULATE THEIR
QUALITY & LIKELIHOOD QUALITY & LIKELIHOOD TO BECOME TO BECOME
BUYERSBUYERS
EMAIL OPTIMIZATION LEVERS
EMAILS SENTEMAILS SENT
OPENED OPENED EMAILSEMAILS
LINKS LINKS CLICKEDCLICKED
LEVER 1: List Size
• Completely dependent on conversion rate of subscription form
LEVER 2: Open Rate
LEVER 3: CTR
Increase List Size
Test Subject Line
Test Email Copy
• Optimize CR on subscription page
• Increase sales to lure repeat buyers
EMAIL OPTIMIZATION LEVERS
EMAILS SENTEMAILS SENT
OPENED OPENED EMAILSEMAILS
LINKS LINKS CLICKEDCLICKED
LEVER 1: List Size
LEVER 2: Open Rate
Quantity Opened
LEVER 3: CTR
Quantity Sent
OPEN RATE =
Increase List Size
Test Subject Line
Test Email Copy
• Use an email application with A/B testing
• Call to action is important
EMAIL OPTIMIZATION LEVERS
EMAILS EMAILS SENTSENT
LINKS LINKS CLICKEDCLICKED
LEVER 1: List Size
LEVER 2: Open Rate
LEVER 3: CTR
Clicks from Email
Quantity Opened
CTR =
Increase List Size
Test Subject Line
Test Email Copy
• Treat as a landing page
• Test layout, buttons, content, etc.
EMAILS SENTEMAILS SENT
OPENED OPENED EMAILSEMAILS
PPC OPTIMIZATION LEVERS
AD IMPRESSIONSAD IMPRESSIONS
AD CLICKSAD CLICKS
LEVER 1: Budget
LEVER 2: Bids
LEVER 3: Quality Score
Increase Budget
Optimize for KPI
Maximize QS
LEVER 4: CTR Test Ad Copy
• Directly drive conversions factoring in conversion rate & CPC
• Optimize for maximum conversions at current CPC
PPC OPTIMIZATION LEVERS
AD IMPRESSIONSAD IMPRESSIONS
AD CLICKSAD CLICKS
LEVER 1: Budget
LEVER 2: Bids
LEVER 3: Quality Score
Increase Budget
Optimize for KPI
Maximize QS
LEVER 4: CTR Test Ad Copy
Ranking Score of Position Below
Quality Score of Your Ad
+Actual Cost $0.01= +
PPC OPTIMIZATION LEVERS
AD IMPRESSIONSAD IMPRESSIONS
AD CLICKSAD CLICKS
LEVER 1: Budget
LEVER 2: Bids
LEVER 3: Quality Score
Increase Budget
Optimize for KPI
Maximize QS
LEVER 4: CTR Test Ad Copy
• Function of ad content, landing page content, historical CTR, & more
• Relevant ad• Relevant landing page• Display URL
PPC OPTIMIZATION LEVERS
AD IMPRESSIONSAD IMPRESSIONS
AD CLICKSAD CLICKS
LEVER 1: Budget
LEVER 2: Bids
LEVER 3: Quality Score
Increase Budget
Optimize for KPI
Maximize QS
LEVER 4: CTR Test Ad Copy
• Continuously A/B test ad
• Keep highest CTR ad• Delete lower CTR ads• Repeat until returns
diminish
Clicks from AdAd Impressions
CTR =
SOCIAL CONTENTOPTIMIZATION LEVERS
SOCIAL SOCIAL IMPRESSIONSIMPRESSIONS
SOCIAL CONTENT SOCIAL CONTENT CLICKSCLICKS
LEVER 1: Network
LEVER 2: Content
LEVER 3: Interactions
Add Networks
Quantity/Quality
Followers/Frequency
• Pinterest• Reddit• Others
• Be there in the first place
SOCIAL CONTENTOPTIMIZATION LEVERS
SOCIAL SOCIAL IMPRESSIONSIMPRESSIONS
SOCIAL CONTENT SOCIAL CONTENT CLICKSCLICKS
LEVER 1: Network
LEVER 2: Content
LEVER 3: Interactions
Add Networks
Quantity/Quality
Followers/Frequency
• Frequency• Better content drives
more shares/retweets
• Links to deep content• Polite, relevant posts
SOCIAL CONTENTOPTIMIZATION LEVERS
SOCIAL SOCIAL IMPRESSIONSIMPRESSIONS
SOCIAL CONTENT SOCIAL CONTENT CLICKSCLICKS
LEVER 1: Network
LEVER 2: Content
LEVER 3: Interactions
Add Networks
Quantity/Quality
Followers/Frequency
• Increase quantity and value of followers
• Don’t “buy” followers
• More interactions drives more clicks
WEBSITE LAYOUT & CONTENT
VISITSVISITS
CONVERSIONSCONVERSIONS
LEVER 1: Layout
LEVER 2: Content
LEVER 3: Click Areas
Forms, Phone #s
Quantity, Quality
Size, Color, Position
LEVER X: etc.....
EXECUTION
THE WEB IS A SYSTEM
• Your marketing strategy is a system of equations• Some levers are independent of others• Some levers manipulate others• Some levers have a larger impact on KPI than
others
Ad Bid SEO Spend
Email Subject
Social Effort
THE WEB IS A SYSTEM
• Think about moving several levers at once• Always consider what a single and multiple-lever move
will do to the system, and accordingly, your end result• Design a good experiment, continue to measure, and
repeat
y = f( x1, x2, x3, . . . . xn )
EXAMPLE SCENARIO
• Current Budget: $7,500 / month ( $250 / day )• Conversions: 48 / month ( 1.6 / day )• Average CPA = $158.17• Average CPC = $2.29• Landing Page CR = 1.45%
New Target: 10 Conversions / Day AND Lower CPA
SCENARIO ONE: ADJUSTING BUDGET
• Average CPA at $158.17 means 10 conversions per day is $1581.70
• $47,000 per month• Easy, but too expensive
SCENARIO TWO: ADJUSTING CPC
• Budget stays at $250 / day, conversion rate steady at 1.45%
• Need 689 clicks to get 10 conversions• Average CPC must be $0.36• Improbable to bring $2.29 down to $0.36 in
this space while maintaining volume
SCENARIO THREE: ADJUST CONVERSION RATE
• Budget stays at $250 / day, CPC at $2.29• Can purchase 110 clicks for $250• To get 10 conversions, landing page needs to
convert at 9.1%• Improbable to quickly jump from 1.45% to
9.1%
FINAL SCENARIO: MOVE ALL THREE LEVERS
• Double the budget to $15,000• Reduce average CPC by 50% ( $2.29 to $1.15 ) via
QS & Keywords• Improve landing page CR from 1.45% to 2.3% via
A/B testing• ( $15k / $1.15 per click ) at 2.3% drives 300
conversions monthly at a CPA of $50• Used 3 levers simultaneously to improve
conversions 6x and reduce average CPA by 68%
ITERATING
CREATE A CULTURE OF CONTINUOUS IMPROVEMENT
• Define success• Measure current performance and compare to
targets• Determine biggest levers and how they contribute
to KPIs• Design an experiment• Test repeatedly• Finalize improvements based on adequate data• Repeat
Questions?
Scott Yacko - [email protected] - @scottmyacko
Jonathan Kressaty - [email protected] - @kressaty