How to give your Optimisation Program Wings

Frans Poldervaart Web Analytics Manager

Tom Waterfall Director of Optimisation Solutions

3

- 75 country websites - 23 different languages

- 1100 pages per country -  Largest travel site in

Netherlands (Twinkle T100)

175.000  transac-ons  per  

month  

24%  online  

KLM.com: By the numbers

4

PEOPLE DRIVE SUCCESS

1. How to get started

dd-mm-2008 Name of presentation - Version 1.01

6

•  Increase share in online sales •  Use the most effective acquisition channel •  Increase flying blue members •  Make it easier for customers to navigate and purchase •  Increase ancillary revenue •  One stop shop •  Ensure customers’ experiences are more positive

1. How to get started: Identify Your Goals Upfront

dd-mm-2008 Name of presentation - Version 1.01

7

1. How to get started

1. How to get started

9

1 2

43

Control

Inspiration

Functional

Trust

1. EXAMPLE: How we took off

10

Inspiration

1. EXAMPLE: How we took off

+14.11% Conv.

+5,400 Bookings

+€3.8 Million

11

1. How to get started: Following Take Off

•  Use all the tools at your disposal •  Begin Brainstorming test ideas

dd-mm-2008 Name of presentation - Version 1.01

12

13

2. Getting buy-in: Start an illegal gambling ring

14

1. EXAMPLE: Starting an illegal gambling ring

15

2. Getting buy-in: Infiltrate & Cultivate

16

2. Getting buy-in: Shout about results & learning

17

2. EXAMPLE: Shout about results & learning

18

+23% Lift

2. EXAMPLE: Shout about results & learning

© 2011 Webtrends, All Rights Reserved. | 19

20

3. Planning & Prioritization: Companies struggle

21

Booking Details page tests Changes to structure, arrangement or functionality

Revenue Potential

Com

plex

Homepage tests Any display/messaging Related tests

Search results page functionality Tests with filters, modifying search

Strikethrough pricing Affecting display, but only doable during sales

Search results page display Promo codes, supplier logos, messaging

Booking Details page tests

Across the site Inclusions, messaging of any kind

Exceptions testing No results pages etc.

HIGH

HIGH

MID

MID

MID

?

?

HIGH

This chart is based on tentative information presented, and elements might move around based on new data available. Starbursts indicate revenue lift potential.

3. Planning: Focus on quick wins

Sim

ple

22

3. EXAMPLE: ROI Potential vs. Complexity

23

3. Planning: Prioritise a mix of simple & complex

+2.6% Lift

3. EXAMPLE: Mixing Simple and Complex

+39% Lift

3. EXAMPLE: Mixing Simple and Complex

27

Control

3. EXAMPLE: Mix & Match

28

Level 2 Level 1 (Control)

Level 3 Level 4

Level ‘Next step’ conversion rate

Lift / Chance to Beat Control

Booking Confirmation

Rate Lift /

Chance to Beat Control

1 33.23% --- 3.41% ---

2 34.91%

+5.05% (99.43% chance to beat

control)

3.21%

-5.84% (Not Significant)

3 35.05%

+5.48% (99.64% chance to beat

control)

3.71%

+8.64% (85%% chance to beat control)

4 36.12%

+8.69% (99.79% chance to beat

control)

3.62%

+6.04% (Not Significant chance to beat

control)

All alternate button copies drove significantly more visitors to the next step and also had a positive impact on Bookings.

3. EXAMPLE: Mixing Simple and Complex

29

Level 2

Level 1 (Control)

Level ‘Next step’ conversion rate

Lift / Chance to Beat

Control

Booking Confirmation

Rate

Lift / Chance to Beat

Control

1 33.76% --- 3.37% ---

2 35.95% 6.47%

(99.99% chance to beat control)

3.60% +6.69%

(87% chance to beat control)

Underlining City and Price and making the whole row clickable is driving significantly more visitors to the next step, and is showing promise with Bookings

3. EXAMPLE: Mixing Simple and Complex

30

The ‘Clickable Row’ had the highest impact at the

booking confirmation rate, with a +6.69% lift

+150 Bookings +6.69%

Lift

3. EXAMPLE: Mixing Simple and Complex

31

3. Planning: Mix & Match between markets/journeys

Bookings

Car Hire

Email Sign-Up

US Traffic Rest of World

English

Spanish

Mobile / Tablet Site

Loyalty Registration

32

3. Planning: Revisit priorities quarterly

© 2011 Webtrends, All Rights Reserved. | 33

34

4. EXAMPLE: EU Cookie Law Test Background

35

4. EXAMPLE: EU Cookie Law Test Background Level 4 Control

Experiment Page Run Date Click button Conversion

Lift Confirmation page

(confidence interval) Conversion

Lift

Control 12/11/2012 – 26/11/2012

- --- 2.98% (+/- 0.21%) ---

Optimal 2.43% (+/- 0.07%) - 3.09% (+/- 0.21%) +3.64%

Optimal

36

“My unborn nephew owns a smartphone,” says person on street.

4. EXAMPLE: Mobile & Tablet

37

4. EXAMPLE: Mobile & Tablet – make a case +61.32% Search “Despite the [rapid

growth of mobile] , 69 companies in the FTSE 100 had done nothing to “optimise” their websites for mobiles…”

38

4. EXAMPLE: Mobile & Tablet – fix it

39

4. Be Thoughtful: Track more, Learn more

40

When the “Filter by Location” option was amended to a dropdown… …visitors used the ‘Star Rating’, ‘Filter by Board’, and Map tools significantly more. Explanation: the dropdown takes up far less real estate and therefore places further emphasis on other filters. Additional Insight: the increased usage of star rating, by board and the map tool correlated positively with an increase in hotels booked. Recommendation: continue to test filter ordering, emphasis, and style to identify the real drivers behind hotel bookings.

4. EXAMPLE: Track more, Learn More

41

5. The Path Ahead for

•  From 0.5 FTE to 2 FTE dedicated to the conversion optimization program

•  Large and complex testing continuing with full support from experts

•  Small tests conceived and built by KLM

•  Increase communication within the organization

•  Proactive testing associated with site developments •  Responsive design •  New applications

Lessons Learned •  Takeoff requires a central owner and initial testing that

focuses on business value

•  Attain departmental buy-in through a healthy mix of sharing and gambling

•  Plan ahead, and revisit your plans regularly

•  Keep it simple and look at all your journeys, markets and goals

•  Keep your program fresh through innovation and relevance

•  Once you’ve taken off, don’t plan on ever landing!

© 2011 Webtrends, All Rights Reserved. | 45