Post on 24-Jun-2015
description
The Sensationalisation of A/B TestingAnd why it doesn’t work for you
#MEASUREFEST
8th October 2014
Manuel da Costa
@digital_tonic
DIGITAL TONIC
We have a problem with A/B testing
#MEASUREFEST
Let’s see how good you are at A/B testing
#MEASUREFEST
#MEASUREFEST
CONTROL
#MEASUREFEST
VARIATION
#MEASUREFEST
VARIATION
25% increase in conversions
#MEASUREFEST
#MEASUREFEST
#MEASUREFEST
Clickthroughs 606% and Purchases 11%
#MEASUREFEST
#MEASUREFEST
What we don’t see
What was the sample size?
How long did they run the test for?
Significance of results?
What was the target segment for the test?
Did they re-test ?
THROWINGcrap against the wall hoping it sticks
#MEASUREFEST
#MEASUREFEST
AB/ TESTING
You need aFramework
#MEASUREFEST
#MEASUREFEST
Observe
#MEASUREFEST
Start here
Your customers
Identify the leaks in your funnel
Get customer insights
Brainstorm
#MEASUREFEST
We believe that changing X for Y customers will result in ZX = variables you want to test
Y= target segment you want to run the test on Z = the outcome i.e. change conversions or user behaviour
Prioritise your test ideas
Ease of testingImpact on bottom line
#MEASUREFEST
AB/ TESTING
Select your sample size
Set your test duration
QA your test
Integrate with analytics
Don’t stop your test too early
Review & Iterate
#MEASUREFEST
Segment your data!!!
Statistical significance http://abtestguide.com/calc/
Re-test
What did you learn?
A B/ Testing
ObserveYour customers
BrainstormReview & Iterate
#MEASUREFEST
Need inspiration for your A/B Testing?
LOOK AT YOUR DATA
#MEASUREFEST
http://bit.ly/digitaltonicmeasurefest
Interested in more Conversion Optimisation stuff?Get the slides, resources and more
#MEASUREFEST
Follow me on twitter : @digital_tonic Drop me a line : manuel@digital-tonic.co.uk