Take Your Testing Program From Invisible To InvinciblePractical CRO tactics to boost your testing program’s success rate, visibility, and resources

1 Optimost.com

Uri Kogan | Zoltan Liu | David Gaskins

April 24, 2016

Optimost.com

Agenda

• Introduction

• Plan and focus on the right part of your site using data

• Prioritize the order of tests you run

• Develop smart hypotheses

• Continuously Optimize your tests

• Build a culture of testing in your organization

2

What would your testing program look like if you could have it all?

Optimost.com

You’d Run a Ton of Tests and Make a Killing

<0%

0% -10%

10% -25%

25% -50%

50% -100%

100% -200%

200% - 300%300% - 400% 400% - 500%

$0 -$250 K

$250 K-$1M

$1 M -$5 M

$5 M -$10 M

$10 M -$25 M

$25 M -$50 M

$50 M -$75 M

$75 M -$100 M

Data from an Optimost client that ran ~200 tests in 1 year

Revenue gains of all testsLift gains of all tests

Optimost.com5

If you’re doing hundreds of tests a year & making hundreds of millions of

dollars for your business, CONGRATULATIONS!

If not, this sessionIS FOR YOU

Optimost.com

Highest Paid Person’s Opinion

HIPPOs

They are responsible for the death of more meaningful tests each year than all other corporate roles combined.

7

“My wife was on the website last night and found the navigation confusing. We should fix this.”

An actual HIPPO quote:

– A Former CEO (yes, really)

Optimost.com

RandomActs ofTestingThey definitely cause substantial ROI losses, especially in developing business.

RAT Syndrome

A

or

B

CED

Optimost.com

What

9

gets in

yourway?

Optimost.com

Plan and Prioritize

Optimost.com

Overall Performance by Device Type: Desktop Leads

13

18,542,100Sessions

20%

Reservation Rate

$350Average Order Value

7,364,981Sessions

7%

Reservation Rate

$189Average Order Value

5,158,651Sessions

12%

Reservation Rate

$260Average Order Value

Optimost.com

But Users Have Different Priorities by Device

14

Reservation

Deals

Login

Reservation

Login

Deals

Reservation

Login

Deals

1

2

3

Optimost.com

Prioritize – Optimost’s SELECT Prioritization Framework

Based on the proposed experiment, what is the likelihood it will provide positive results?

Identifies the level of effort it will take to implement/execute the proposed experiment.

Estimated time to complete the proposed experiment.

Monthly revenue from 1% lift

Up to 3 custom attributes, such as customer satisfaction

SUCCESS POTENTIAL EASE OF

DEVELOPMENT LENGTH TO RUN CUSTOM TRAFFIC VALUE

Optimost.com

Smart Hypotheses

Optimost.com

Smart hypotheses

17

What does a Smart Hypothesis include? An easy to use format1

•Clearly defined testing objective: States how the test will likely generate lift, based on the specific testing goal or Key Performance Indicator (KPI)

•Strong rationale: Justifies why the intended element is being tested

•Specific target audience: Determines who the target audience is and builds off this visitor knowledge to inform the testing concept

•Knowledgeable approach: Uses insights garnered from other tests or best practices to build on the current test

Because we observed[data] and [feedback],

we believe that [doing]

for [people]

will result in [outcome].

Sources: Craig Sullivan, twitter.com/optimiseordie/status/633408692201852929, and Mark Fassbender, www.linkedin.com/in/flashbender

Optimost.com

Smart hypotheses: Example

• Test objectiveDetermine if reversing the order of the “Price Matrix” columns would increase the average order value without hurting conversions

• RationaleThe testing team believed viewers weren’t even reviewing the premium options. Would putting the premium coverage level on the left prompt customers to see all the options and select the premium options more frequently?

• Knowledgeable approachThe test was influenced by the work of Robert Cialdini. He found that presenting the highest price on the left worked well.

Optimost.com

Smart hypotheses: Example

Because we observed [customers weren’t interacting with all premium coverage levels available],

we believe that [moving the coverage order so it reads from premium to basic]

for [all users]

will result in [higher income and better customer satisfaction scores].

Optimost.com

Which one boosted conversions? Which one boosted revenue?

A

B

Optimost.com

Which one boosted conversions? Which one boosted revenue?

A

B

Result: Customers chose premium coverage 5.4% more frequently & conversions rose 13%.

Optimost.com

What’s your smart hypothesis?

Because we observed [data] and [feedback],we believe that [doing]for [people]will result in [outcome].

Bring your questions to our expert 1:1s

Continuous Optimization

Optimost.com

Test 1 – Message & CTA

433%INCREASE IN CLICKS

Optimost.com

Test 2 – Value Prop

4%INCREASE IN CLICKS

Optimost.com

Test 3 - MVT

71%INCREASE IN CLICKS

Optimost.com

Test 4 – Geo-targeting

+30.71%Geo-targeted images in San Francisco

+17.83%Geo-targeted images in New York increased

31%INCREASE IN CLICKS

Build a Culture of Testing

Optimost.com

Excite and Involve your Entire Organization!

Optimost.com

Wrong:“This test achieved a 9% boost in adds to cart.”

Better: “Every 1% boost in adds to cart equals $10 million in annual revenue. This test, annualized, could bring in $90 million in annual revenue”

Get Buy In!

Every

1%BOOST

In adds to cart

$10MILLION

Annual revenue

$90MILLION

Annual revenue

Use terms that matter to your leaders and help finance and your CEO understand. Help them understand why you test in terms of their KPIs.

Want to learn more?Meet Jim

optimost.com/tour

Thank You!

linkedin.com/in/kogan

@UriDXM

linkedin.com/in/zoltanliu linkedin.com/in/david-gaskins-0b08761