Iterative versus innovative testing: Exploiting the Full Spectrum of Testing Opportunities - Paul...

@paulrouke #elitecamp2015

FROM ITERATIVE TO INNOVATIVE TESTING

EXPLOITING THE FULL SPECTRUM

OF TESTING OPPORTUNITIES

PAUL ROUKEFounder & Director of Optimisation at PRWD


ITERATIVE

TESTINGINNOVATIVE

TESTING

OPPORTUNITY


OUT-GROW YOUR COMPETITION

By embracing the full spectrum of research & insight driven

testing opportunities


What is the difference?

What are the influencing factors?

Why choose each type?

When are you best choosing each type?

What business impact does each type deliver?

8 steps to out-grow your competition

FROM ITERATIVE

TO INNOVATIVE

TESTING


A full service conversion optimisation agency based in the UK

Optimising online experiences since 2004

Represents the UK in the Global Conversion Alliance, alongside WiderFunnel, WebArts & Conversionista

Conducted over 600 hours of in-depth user research across multiple sectors

540% ROI delivered on average through PRWD’s Six Step Optimisation Method

ABOUT PRWD








FROM ITERATIVE

TO INNOVATIVE

TESTING


WHAT IS THE DIFFERENCE?


ITERATIVE TESTING

Headline changes

Copy changes

Image changes

Button size changes

Button text changes

Content removal

Page layout changes

Removing or optimising form fields

Go on then if you must… button colour changes

IN SUMMARY

Almost all the testing that currently takes place


Control

ITERATIVE TESTING


PRWD Test Variation

ITERATIVE TESTING


Result

+ 4.7%Reply Conversion Rate

ITERATIVE TESTING


Control

ITERATIVE TESTING


PRWD Variation

ITERATIVE TESTING


Result

+ 20%Brochure Requests

CONSISTENTLY COMMUNICATE HOW GOOD YOU ARE

ITERATIVE TESTING


Ad-hoc testing with weak hypotheses will destroy buy-in

Intelligent, insight driven iterative testing will build your test & learn culture


INNOVATIVE TESTING

Full redesigns of homepage

Full redesigns of primary template pages

Landing page redesigns

Customer journey changes

Testing changes to a business proposition

Tests prior to changing back-end systems

Reducing steps within a flow ie. Checkout, application, registration

Even full website redesigns

IN SUMMARY

Very few businesses are bold enough to invest in this

type of testing


Control

INNOVATIVE TESTING


PRWD Test Variation

INNOVATIVE TESTING


Result

+ 78%New visitor sign-ups

BE BOLD & PROVIDE EARLY & EASY OPPORTUNITIES TO CONVERT

INNOVATIVE TESTING


Poorly planned innovative testing will waste valuable resources & damage testing momentum

Intelligent, insight driven innovative testing can help transform & grow your business








FROM ITERATIVE

TO INNOVATIVE

TESTING


WHAT ARE THE INFLUENCING FACTORS?


ITERATIVE TESTING


INNOVATIVE TESTING








FROM ITERATIVE

TO INNOVATIVE

TESTING


WHY CHOOSE EACH TYPE?


Quick wins

Start to learn

Use your tool

Limited resources

Political business

Build momentum

Demonstrate ROI

Develop a culture

IN SUMMARY

You want to walk before you start running with testing

WHY CHOOSE ITERATIVE TESTING?


Control

ITERATIVE TESTING


PRWD Test Variation

ITERATIVE TESTING


Result

+ 4.3%Sitewide Purchase Conversion Rate

Over £250k in additional revenue

ITERATIVE TESTING


Make significant increases

Really shift user behaviour

Quantifiable evidence for a major change

Prior to back-end system changes

Business proposition evaluation

Reached local maximum

Value proposition learning

Simplify a stepped process

Data driven website redesign

WHY CHOOSE INNOVATIVE TESTING?

IN SUMMARY

You want grow your business rather than just optimise your

website performance


BUSINESS CHALLENGE

Identify the 1 most influential feature why visitors will choose Premium over FREE


GET REVISING

FREE OR PREMIUM DECISION

Price doesn’t put me off, although the basic package is more than enough for me.

I think the site has already got everything I need.

Moderated user research participant



GET REVISING

SIGN-UP TABLE

+ 185%Premium Subscriptions

- 10%Free Sign-ups



Intelligent, innovative testing is essential in exploiting your growth potential








FROM ITERATIVE

TO INNOVATIVE

TESTING


WHEN ARE YOU BEST CHOOSING EACH TYPE?


Quick win test opportunities

Primary proposition headlines

Primary CTA’s in to sign-up process

When imagery is key

Sitewide value proposition messages

Landing pages with forms

Form tests

WHEN TO CHOOSE ITERATIVE TESTING

IN SUMMARY

When you have non-optimised primary pages in your user journey


When you have bold and/or innovative ideas built off previous test insights & deep user understanding

If you have run a range of iterative tests on a key page and are now struggling to impact page performance

After you have run tests aimed at understanding what USP’s most resonate with your target audience

When there is an opportunity to deliver a radically different user experience aimed at simplifying a process

ie. Registration, checkout, application

When there will be a considerable development cost to fully implementing a proposed solution

When you have quite low traffic and want to reduce your test length

When data driven optimisation is truly part of your company culture

WHEN TO CHOOSE INNOVATIVE TESTING

When you are ready to grow your business and not just optimise your website performance


Control

INNOVATIVE TESTING


PRWD Test

Variation

INNOVATIVE TESTING


Result + 18%Brochure Requests

INNOVATIVE TESTING


Control

INNOVATIVE TESTING


PRWD Test

Variation

INNOVATIVE TESTING


Result + 31%Brochure Requests

INNOVATIVE TESTING


What are the 2 biggest influencers to running successful innovative tests?

1) In-depth user understanding2) Meaningful insights from prior tests








FROM ITERATIVE

TO INNOVATIVE

TESTING


WHAT BUSINESS IMPACT DOES EACH TYPE HAVE?


BUSINESS IMPACT THROUGH ITERATIVE TESTING


BUSINESS IMPACT THROUGH INNOVATIVE TESTING


Innovative Testing

Iterative Testing

User Research

High Quality & IntelligentOptimisation Strategy =


Innovative Testing

Iterative Testing

User Research

=High Quality & IntelligentOptimisation Strategy








FROM ITERATIVE

TO INNOVATIVE

TESTING


8 STEPS TO OUT-GROW YOUR COMPETITION


Foundations - get the right foundations in place for optimisation

Quality – focus on quality of tests, then quantity

Intelligence - start with intelligent, research driven simple tests

Identification – use simple test insights & research to identify radical test opportunities

Momentum - build up testing momentum, buy-in and awareness of learnings

Business case – use ROI from simple/iterative tests to provide the business case for radical testing

Go innovative - begin planning and delivering bold, innovative tests alongside iterative tests

Growth - grow faster than your competition by exploiting the full spectrum of testing opportunities

1

2

3

4

5

6

7

8 STEPS TO EXPLOIT THE FULL SPECTRUM OF TESTING OPPORTUNITIES

8

PAUL ROUKE

Founder & Director of Optimisation

@paulrouke

THANK YOU FOR LISTENING

HERE’S MY FINAL CTA…

OVER 75 CRO ARTICLES

bit.ly/CROresources

Iterative versus innovative testing: Exploiting the Full Spectrum of Testing Opportunities - Paul...

Retail

Transcript of Iterative versus innovative testing: Exploiting the Full Spectrum of Testing Opportunities - Paul...