[WMD 2015] ConversionXL >> Peep Laja, "WARNING: 13 Ways You're Screwing Up Your A/B Tests"
-
Upload
500-startups -
Category
Technology
-
view
450 -
download
0
Transcript of [WMD 2015] ConversionXL >> Peep Laja, "WARNING: 13 Ways You're Screwing Up Your A/B Tests"
Peep Laja, ConversionXL @peeplaja
Typical scenario:
A company runs 100 tests
/ year. After 12 months
their conversion rate is still
the same.
Testing mistake #2:
You don’t have a data-
driven, learning-
oriented hypothesis
Peep Laja, ConversionXL @peeplaja
Learn more: http://conversionxl.com/how-to-come-up-with-more-
winning-tests-using-data/
Peep Laja, ConversionXL @peeplaja
I won’t even look at results until
there’s at least 250-350
conversions PER variation.
Way more if you want to segment the data
Peep Laja, ConversionXL @peeplaja
Learn more: http://conversionxl.com/how-to-do-conversion-
optimization-with-very-little-traffic/
Peep Laja, ConversionXL @peeplaja
Testing mistake #8:
Test data is not sent
to third-party
analytics
Peep Laja, ConversionXL @peeplaja
Testing mistake #9:
You give up after your
first test for a
hypothesis fails.
Peep Laja, ConversionXL @peeplaja
Full case study:
http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/
79.3% uplift in conversions
Peep Laja, ConversionXL @peeplaja
5% monthly increase in
your conversion rate will
result in 80% uplift for
the year. That’s how
math works.
Peep Laja, ConversionXL @peeplaja
Testing mistake #12:
You copy competitors
/ leaders / other
people’s tests
Peep Laja, ConversionXL @peeplaja