A/B Testing: You Might be Driving in the Wrong Direction
35
A/B Testing You Might Be Driving in the Wrong Direction TOMASZ BORYS
-
Upload
kissmetrics-on-slideshare -
Category
Marketing
-
view
5.714 -
download
0
Transcript of A/B Testing: You Might be Driving in the Wrong Direction
A/B Testing You Might Be Driving in the Wrong Direction
TOMASZ BORYS
@NEILPATEL@Kissmetrics
#KissWebinar
@thuelmadsen
Thue is the Kissmetrics Webinar Wizard and Marketing Ops Manager. Before joining forces with
Kissmetrics, he was a Lyft driver in SF, which is also how he ended up as a KISSmetrics marketer.
Whenever Thue is not trying to automate everything around him, you can find him hiking in the Sierras.
THUE MADSEN Marketing Operations Manager, Kissmetrics
@ThueLMadsen
Tomasz loves dipping his feet in the river while fishing, injuring his thumb while gaming, hacking
away at a golf club, and driving demand at Kissmetrics.
He’s also the biggest fan of gummy bears.
TOMASZ BORYS Director of Marketing, Kissmetrics
@tbcali
@NEILPATEL@Kissmetrics
#KissWebinar
@tbcali
1 The optimum strategies for A/B testing
2 How we A/B Test at KISSmetrics
3 A/B Testing beyond click conversions - having a pulse on the entire funnel and why it’s important
TABLE OF CONTENTS
4 The influence social traffic has in the funnel
WATCH WEBINAR RECORDING NOW
• Set a Goal
• Baby Steps
• Aim for Statistical Significance
• Never Lose Sight
THE OPTIMUM STRATEGIES FOR A/B TESTING
Set a Goal
Baby StepsPeople want to move the needle and see results fast, by implementing multiple changes at once.
But how can you be sure what element had an impact?
Aim for Statistical Significance SET A GOAL
Aimfor Statistical Significance
getdatadriven.com
Never Lose SightWe can caught up on what we think is best to optimize conversions or traffic…but data doesn’t lie
• 4,000 and above sample size
• 99% Statistical Confidence
• Blind Eye for 1 Week
HOW WE A/B TEST AT KISSMETRICS
4,000 and Above Sample Size Significant or decent amount of data is needed when you’re testing beyond click conversions
4,000 and Above Sample Size
significant or decent amount of data is needed when you’re testing beyond click conversions
99% Statistical Confidence
Police
Blind Eye for 1 WeekData can be very erratic the first several days, so it’s easy to hit the panic button
A/B Testing Beyond Click Conversions
ORIGINAL
VARIANT
SIGNUPS
SIGNUPS
SIGNUPS
Signup Install JS Custom Data Opps
Signups
Installed JS
Signups
Install JS
Signups
Install JS
Custom Data
Custom Data
Custom Data
Custom Data
Opportunities Created
Custom Data
Opportunities Created
Custom Data
Opportunities Created
SOCIAL AD FUNNEL
LOG IN WITH GOOGLE
Start Your Free KISSmetrics Trial
TOMASZ BORYS Director of Marketing, Kissmetrics
@tbcali [email protected]
THUE MADSEN Marketing Operations Manager, Kissmetrics
@ThueLMadsen [email protected]
Questions?
