#thinkppcHOSTED BY:

The 11 13 Biggest A/B Testing Mistakes and How To

Avoid Them

Featuring Peep Laja

#thinkppc

Presenters

• Peep Laja– Guest Presenter– Founder of Conversion XL

– @peeplaja

• Roopa Carpenter– Moderator– CRO Manager at Hanapin Marketing

– @roopacarpenter

#thinkppc

Join the conversation

• Include the hashtag #thinkppc in your Twitter tweets.

Or use the webinar question box to send us questions.

#thinkppc

Live Poll Question #1

How long have you been doing CRO?#thinkppc

A. Less than 1 yearB. 1-3 yearsC. 3-5 yearsD. 5+ years

#thinkppc

Live Poll Question #2

How do you manage your account(s)?#thinkppc

a) I’m part of an in-house PPC team.b) I do all the marketing myself.c) I work with an agency.d) I’m a consultant.

#thinkppcPeep Laja, ConversionXL @peeplaja

Typical scenario: A company runs 100 tests / year. Lots of

wins. But no difference on the bank account.

#thinkppc

Precious time wasted on stupid

tests

Testing mistake #1:

#thinkppc

You don’t have a data-driven,

learning-oriented hypothesis

Testing mistake #2:

#thinkppc

Learn more: http://conversionxl.com/how-to-come-up-with-more-winning-tests-using-data/

#thinkppc

You think you know what will

work

Testing mistake #3:

#thinkppc

Your sample size is way too small

Testing mistake #4:

#thinkppc

#thinkppc

Calculate needed sample size in advance:

#thinkppc

I won’t even look at results until there’s at least 250-350

conversions PER variation.

Way more if you want to segment the data

#thinkppc

Your tests don’t run long enough

Testing mistake #5:

#thinkppc

#thinkppc

You don’t test full weeks at a time

Testing mistake #6:

#thinkppc

#thinkppc

You run tests when you have no

traffic

Testing mistake #7:

#thinkppc

•Learn more: http://conversionxl.com/how-to-do-conversion-optimization-with-very-little-traffic/

#thinkppc

Test data is not sent to third-party

analytics

Testing mistake #8:

#thinkppc

#thinkppc

(function(){ var experimentID = 2809164105; window.optimizely = window.optimizely || [];

if (typeof window.optimizely.data.experiments[experimentID] !== 'undefined') { var experimentName = window.optimizely.data.experiments[experimentID].name; var variationName = window.optimizely.data.state.variationNamesMap[experimentID]; window.ga = window.ga || function() {(window.ga.q = window.ga.q || []).push(arguments);}; window.ga.l =+ new Date(); window.ga('send', 'event', 'Optimizely', experimentID + ' ' + experimentName, variationName, {'nonInteraction': 1}); } })();

#thinkppcPeep Laja, ConversionXL @peeplaja

#thinkppc

You give up after your first test for

a hypothesis fails.

Testing mistake #9:

#thinkppc

#thinkppc

Full case study: http://conversionxl.com/case-study-how-we-improved-landing-page-conversion/

79.3% uplift in conversions

#thinkppc

#thinkppc

#thinkppc

#thinkppc

#thinkppc

#thinkppc

#thinkppc

You’re not aware of validity threats

Testing mistake #10:

#thinkppc

History effectInstrumentation

effectSelection effect

#thinkppc

You’re ignoring small gains

Testing mistake #11:

#thinkppc

5% monthly increase in your conversion rate will result in 80% uplift for the year. That’s how

math works.

#thinkppc

You copy competitors /

leaders / other people’s tests

Testing mistake #12:

#thinkppc

Which Test Won?

#thinkppc

WhichTestWon?

WhichTestWasNotRunProperly?

#thinkppc

Somebody else’s test results are

useless

#thinkppc

“Site shows stats from various A/B tests - Finally I've got evidence to show clients on a load of design decisions!”

#thinkppc

#thinkppc

The process, the method, the

analysis

#thinkppc

You’re not running tests at all times

Testing mistake #13:

#thinkppc

PPC Retainers

Need some assistance for your PPC? We offer on-going and one-time retainers.

Learn More:

http://www.hanapinmarketing.com/ppc-retainers/

#thinkppc

Live Q&A Time!

#thinkppc

Have more questions?

Thank you for attending our webinar! #thinkppc

• Contact us Directly:

» Hanapin Feedback: marketing@hanapinmarketing.com