Five Steps to Better Metrics

Five Steps to Better Metrics: How one marketer leveraged web analytics for an annual revenue increase of $500,000

#webclinic

#webclinic

Join the conversation on Twitter

#webclinic

#webclinic

Today’s team

Dr. Flint McGlaughlin Managing Director

Jon Powell Senior Manager Research and Strategy

#webclinic

4

The Challenge

Q: Are there metrics your organization does NOT monitor, only because they are not set up properly?

#webclinic

Background and Test Design

Background: REGOnline is event management software that lets users create online registration forms and event websites to manage their events. Goal: To increase number of completed leads on homepage. Primary research question: Which page will generate the greatest number of leads? Approach: A/B multifactor split test

Experiment ID: REGOnline Homepage Test Location: MarketingExperiments Research Library Test Protocol Number: TP1427

Research Notes:

#webclinic

Experiment: Control

Control - Homepage

• Our researchers hypothesized that we could increase the appeal associated with the value proposition of this offer by focusing more on the product and its specific features and benefits.

#webclinic

7

Experiment: Treatment Treatment - Homepage

• Headline was written to focus more on the product.

• Specific features and benefits are utilized to express the value.

• The page emphasizes “Free Access.”

• Also, ensured that this value was being communicated in subsequent steps.

#webclinic

8

Experiment: Side-by-side

Control Treatment

#webclinic

Experiment: Results

Versions Conversion

Rate Rel. diff

Control – Two-step homepage 2.3% -

Treatment – Three-step homepage 1.7% -24.5%

24.5% Decrease in Conversion The Treatment generated 24.5% less completed leads

What you need to understand: In spite of having a clearer value and reducing the amount of form fields in the first step, the control still outperformed the treatment.

#webclinic

10

Homepage from Previous Test

24.5% Decrease in Conversion

• Before we could get a lift, we needed to learn more about the prospects coming to this site.

• We decided to use one of their SEO pages as a research window into the cognitive psychology of the customer’s motivation.

Experiment #2: Background

#webclinic

11

Background: A technology and media company specializing in online registration and event management software. Goal: To increase the amount of leads generated online. Primary research question: Which online capture process will generate the higher addressable lead rate? Approach: A/B multifactor split test

Experiment ID: REGonline SEO landing page test Location: MarketingExperiments Research Library Test Protocol Number: TP3055

Research Notes:

Experiment #2: Background

#webclinic

12

SEO Landing Page

• This landing page was offering the same product as the home page but dealt with a smaller subset of visitors who matched the profile of those coming to the homepage.

• Our researchers could test here without the negative consequences of hurting conversion on the homepage.

Experiment #2: Control

#webclinic

Treatment SEO Landing Page

• For our first test on this page, we tested focusing on how this product made the process of creating registration forms easier and could cut the prospects’ time in half…

• …and yet it still had a robust functionality.

Experiment #2: Treatment

#webclinic

14

Experiment #2: Side-by-side

Treatment Control

Which copy language will generate the most leads?

#webclinic

15

Design Conversion Rate

(%) Relative

Difference Statistical Level of Confidence

Original Page 0.7% - -

Treatment 4.8% 548%

548% Increase in Complete Leads The new page’s conversion rate increased by 548.46%

99%

What you need to understand: By focusing on how this product made creating registration forms easier, the treatment was able to increase step-level clickthrough rate by 1,312%, and completed leads captured by 548%.

Experiment #2: Results

#webclinic

16

548% Learning Learning

SEO Page Test Original Homepage New Homepage Test

90%

• We were able to take what we learned about the motivations of their customers from testing on the SEO landing page and apply it to the homepage, which generated a 90% increase in leads captured.

Experiment #2: Final Results

#webclinic

What we discovered

1. The goal of all customer research is to enable the marketer to predict customer behavior.

2. Therefore, the primary usefulness of metrics is not in answering “how many?” but rather in answering, “why so?”

3. Ultimately, metrics enable the marketer to see the cognitive trail left by the visitor’s mind.

Key Principles F

#webclinic

18

• The problem is not typically getting sufficient data from you metrics software. Rather, the challenge is making sense of it.

How do we cut through it all?

#webclinic

19

u = 2q + t + m + 2v + i ©

Online Testing Heuristic:

u = Utility

q = Research Question

t = Treatment

m = Metric System

v = Validity Factor

i = Interpretation

Online Testing Heuristic

#webclinic

20

Today, we will walk through a simple 5-step process for translating raw testing data into predictive power

#webclinic

21

1. Establish Visibility – Ensure that your metric platforms are able to track the four primary types of analytics:

Translating Raw Data to Predictive Power

Key Steps F

#webclinic

22

STEP 1: Establish Visibility Types of Analytics – Visual

Amount

Nature

Source

Results

Page views

visitor sessions

returning visitors

impressions

referrers search terms

languages

geographic location

organizations

exit pages browsers

Screen resolution

time on page Load errors

Orders Sign-ups

Number of page views

Click trails Most requested pages

Entry pages

#webclinic

23

1. Establish Visibility – Ensure that your metric platforms are able to track the four primary types of analytics:

Amount – How many instances of a particular action are occurring?

Source – Where are prospects coming from?

Nature – What are prospects experiencing on your site?

Results – What are prospects doing on your site?


Key Steps F

#webclinic

24

1. Establish Visibility – Ensure that your metric platforms are able to track the four primary types of analytics: (1) Amount, (2) Source, (3) Nature, (4) Results.

2. Determine Objective – Determine the exact research question you are setting out to answer with your metrics.

Key Steps F


#webclinic

25

1. Whether you are running a live test or conducting a forensics metrics analysis, your research and metrics analysis must be grounded in a properly framed Research Question.

2. A properly framed Research Question is a question of “which” and sets out to identify an alternative (treatment) that performs better than the control.

Example:

Not this..

What is the best price for product X?

But this…

Which of these three price points is best for product X?

* Depending on the data available, forensics data is often grounded in a research question of “what?” rather than “which”.

STEP 2: Determine the Objective The Research Question

#webclinic

26

STEP 2: Determine the Objective Audience Exercise

How would you refine the following three research questions?

? 1. What is the best headline for my landing page?

2. Why do I have such a high bounce rate on my offer page?

3. How many objectives should I have on my homepage?

#webclinic

27

STEP 2: Determine the Objective The Research Question

4% Completion Rate

once process begins

Hotels

Autos

Activities

Travelers

Summary

Login

Contact

Payment

Flights

Unique visits

40,607,893

14,185,646

7,729,403

9,167,901

12,883,177

7,717,122

5,665,020

3,260,292

2,484,236

Not all visitors go through each of these steps

1,766,609

58%

73%

76%

71%

60%

32%

1. Often, metrics can also be utilized to determine the most effective research questions you should be asking.

2. Metrics can be a window into key gaps into your customer theory and ultimately into the highest potential revenue opportunities for marketing efforts.

#webclinic

28

STEP 2: Determine the Objective Example Case Study – Experiment Background

Background: A website that sells retail and wholesale collector items Goal: To increase conversion rate Primary research question: Which version of second step in the conversion funnel will produce the highest conversion rate? Approach: A/B variable cluster split test that focused on reducing anxiety through credibility indicators, copy, and re-organization of existing page elements

Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP1305

Research Notes:

#webclinic

29

STEP 2: Determine the Objective Example Case Study – Experiment Background

• When we analyzed the metrics, we realized there were leaks throughout the checkout process, the credit card submission page stood out as low cost opportunity for immediate return.

• When we analyzed the metrics even further, we saw that this step also had the highest lost revenue per cart (more than double of any other step).

• From this, we hypothesized that optimizing this step would have the highest potential return on our efforts.

Fallout Report: New Customers

#webclinic

30

STEP 2: Determine the Objective Example Case Study – Experiment Control

What might be causing the fallout?

• It is unclear why the credit

card is required when payment method is different.

• The complexity of the

Purchase Agreement Terms’ causes confusion and concern.

• There is no indication that my credit card information is secure.

Control

#webclinic

31

How we addressed the issues:

• Third-party security indicators have been added.

• Clearer explanation of why a credit card is required and that it will not be charged.

• “Satisfaction Guaranteed”

promise is emphasized.

Treatment

STEP 2: Determine the Objective Example Case Study – Experiment Treatment

#webclinic

32

STEP 2: Determine the Objective Example Case Study – Experiment Results

Design Conversion Rate

Control 82.33%

Treatment 86.04%

Relative Difference 4.51%

5% Increase in total conversion The new credit card page increased conversion by 4.51%

What you need to understand: While it might seem like a small increase, choosing this specific step in the sales funnel to test resulted in a projected $500,000+ increase in revenue per year. This underscores the potential impact of a properly identified research question.

#webclinic

33



3. Track and Measure – Track and measure the appropriate metrics that will provide you with the answer to your determined research question.


Key Steps F

#webclinic

34

STEP 3: Track and Measure Primary and Secondary Metrics

Secondary Metrics

Primary Metrics

1. Primary “Test” Metrics: The essential metrics that enable you to answer the research question

2. Secondary Metrics: The additional metrics you can utilize to help interpret the results of your primary metrics

#webclinic

35

STEP 3: Track and Measure Primary Metrics – Examples

Research Question: Which headline will generate the most subscriptions? Primary Metrics: Visits, subscriptions subscription rate (%)

Research Question: Which PPC ad will generate the most qualified traffic? Primary Metrics: Ad spend, conversions cost per acquisition ($)

Example #1:

Example #2:

Example #3:

Research Question: Which page will generate the most Facebook fans? Primary Metrics: Visitors, clicks on the “Like” button fans per visitor (%)

#webclinic

36

STEP 3: Track and Measure Secondary Metrics – Examples

Time on page

Click tracking

Bounce rate

Segment-level data

Form event tracking

Traffic patterns

Are visitors engaged with the content? Are they confused with the process?

What are visitors interested in? Are they confused with the process?

Is there a lack of relevance to visitors? Are there too many distractions? Is there too much (or little) information?

What motivates individual visitor types? Where are the deeper optimization opportunities?

What form fields cause anxiety or confusion? How much friction will your visitor put up with?

Who is coming and where are they coming from? Can we be more relevant to the visitor?

Secondary Metric Potential Insights

#webclinic

37

STEP 3: Track and Measure Example Case Study – Experiment Background

Experiment ID: (Protected) Location: MarketingExperiments Research Library Test Protocol Number: TP1341

Research Notes:

Background: A company offering dedicated hosting services Goal: To increase the number of leads Primary research question: Which page design will generate the greater number of leads? Approach: A/B multi-factor split test (radical redesign)

#webclinic

38

Control Treatment

Let’s consider both the primary and secondary metrics utilized for this test…

STEP 3: Track and Measure Example Case Study – Experiment Treatments

#webclinic

Control Treatment

39

Primary Metrics Visits = 31,400* leads = 628* CR = 2.0%

Primary Metrics Visits = 30,560* Leads = 1,764* CR = 5.7%

Research Question: Which page design will generate the greater number of leads?

Answer: The treatment design will generate 188% more leads.

* Numbers have been anonymized

STEP 3: Track and Measure Example Case Study – Experiment Metrics

#webclinic

40

• In addition to tracking the primary metrics, the research analysts installed some secondary event tracking metrics.

• On this page, there were six expandable sections of copy featuring different elements of the product value proposition.

• By monitoring the specific clicks of visitors on this page, we were better able to understand what aspect of this product’s value proposition was most appealing to the visitor.

STEP 3: Track and Measure Example Case Study – Experiment Metrics

#webclinic

41




4. Monitor Anomalies – Monitor the data for any anomalies that might indicate a validity threat.


Key Steps F

#webclinic

42

STEP 4: Monitor Anomalies Audience Question

3.00%

5.00%

7.00%

9.00%

11.00%

13.00%

15.00%

17.00%

19.00%

Control

Treatment 3

Co

nver

sio

n R

ate

Test Duration

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6 Day 7 Day 8 Day 9 Day 10 Day 11

What wrong with this test data set?

?

#webclinic

43

STEP 4: Monitor Anomalies

(3.000)

(2.000)

(1.000)

0.000

1.000

2.000

3.000

4.000

Satu

rday

, Oct

ob

er 1

1, 2

00

8

Sun

day

, Oct

ob

er 1

2, 2

00

8

Mo

nd

ay, O

cto

ber

13

, 20

08

Tues

day

, Oct

ob

er 1

4, 2

00

8

Wed

nes

day

, Oct

ob

er 1

5, 2

00

8

Thu

rsd

ay, O

cto

ber

16

, 20

08

Frid

ay, O

cto

ber

17

, 20

08

Satu

rday

, Oct

ob

er 1

8, 2

00

8

Sun

day

, Oct

ob

er 1

9, 2

00

8

Mo

nd

ay, O

cto

ber

20

, 20

08

Tues

day

, Oct

ob

er 2

1, 2

00

8

Wed

nes

day

, Oct

ob

er 2

2, 2

00

8

Thu

rsd

ay, O

cto

ber

23

, 20

08

Frid

ay, O

cto

ber

24

, 20

08

Satu

rday

, Oct

ob

er 2

5, 2

00

8

Sun

day

, Oct

ob

er 2

6, 2

00

8

Mo

nd

ay, O

cto

ber

27

, 20

08

Tues

day

, Oct

ob

er 2

8, 2

00

8

Wed

nes

day

, Oct

ob

er 2

9, 2

00

8

Thu

rsd

ay, O

cto

ber

30

, 20

08

Frid

ay, O

cto

ber

31

, 20

08

Satu

rday

, No

vem

ber

01

, 20

08

Sun

day

, No

vem

ber

02

, 20

08

Mo

nd

ay, N

ove

mb

er 0

3, 2

008

Tues

day

, No

vem

ber

04

, 20

08

Wed

nes

day

, No

vem

ber

05

, 20

08

Thu

rsd

ay, N

ove

mb

er 0

6, 2

00

8

Frid

ay, N

ove

mb

er 0

7, 2

00

8

Satu

rday

, No

vem

ber

08

, 20

08

Sun

day

, No

vem

ber

09

, 20

08

Mo

nd

ay, N

ove

mb

er 1

0, 2

008

Tues

day

, No

vem

ber

11

, 20

08

Wed

nes

day

, No

vem

ber

12

, 20

08

Thu

rsd

ay, N

ove

mb

er 1

3, 2

00

8

Standardized Conversion Rate

Normalized

Normalized B

Normalized Traffic

Normalized Traffic B

YES

NO

NO

Graphed results of a 4-week email test with an ecommerce retailer: • Monitor for unexplainable

temporary spikes in the amount of traffic or views to a specific online campaign

• A more subtle clue is a noticeable shift in the kind of response visitors are having to a specific online campaign (e.g., conversion rates, sales, average purchase amounts, bounce rates, etc.)

Validity Threats

#webclinic

44

STEP 4: Monitor Anomalies Validity Threats

History Effect – when a test variable is affected by an extraneous variable associated with the passage of time

Instrumentation Effect – when a test variable is affected by a change in the measurement instrument

Selection Effect – when a test variable is affected by different types of subjects not being properly distributed among experimental treatments

Anomalies in your metrics can indicate that there may be validity threats in your tests and data. Be sure to check for the following validity threats should you encounter any anomaly.

For more on validity threats, see our previous Web clinic replay: “Bad Data: The 3 validity threats that make your tests look conclusive (when they are deeply flawed).”

http://www.marketingexperiments.com/site-optimization/bad-data.html

#webclinic

45





5. Interpret Data – Interpret the data by moving from “Which?” to “Why?” to “What?” to “Where?”.

Key Steps F


#webclinic

46

STEP 5: Interpret Data From Customer Behavior to Customer Theory

Which? Why? What?

Customer Behavior Customer Theory

Which headline will generate a higher response?

What does my customer want the most?

Which testimonial will generate the most response?

What makes my customer especially anxious?

Which call to action will generate a higher response?

What is my customer’s position in the sequence of micro-yeses?

Why this headline?

Why this testimonial?

Why this call-to-action?

#webclinic

47

STEP 5: Interpret Data Example Case Study

201% 2% 29%

Test results are interpreted and second test was created based on the analyst’s observations

Again, test results are interpreted and the next round of testing is started for this page

Test is again interpreted and transferrable principles are applied to other offer pages

#webclinic

48

STEP 5: Interpret Data Where else can we apply this data?

451%

302%

603% 257% 28%

• The discoveries and insights about customer motivation from the three prior tests were applied to other landing pages and used to optimize PPC campaigns.

• The purposeful effort to identify and selectively apply these transferrable insights led to widespread optimization gains .

#webclinic

49

www.meclabs.com/BTW

Save $100 off any workshop

Promo Code: 284-WS-2022

Baltimore Training Week

July 30 - August 1

http://www.meclabs.com/BTW

#webclinic

50

Key Principles F

1. The goal of all customer research is to enable the marketer to predict customer behavior.

2. Therefore, the primary usefulness of metrics is not in answering “how many?” but rather in answering, “why so?”

3. Ultimately, metrics enable the marketer to see the cognitive trail left by the visitor’s mind.

Summary: Putting it all together

#webclinic

51

Key Steps F





5. Interpret Data – Interpret the data by moving from “Which?” to “Why?” to “What?” to “Where?”.

Summary: Putting it all together

#webclinic

How can I track and integrate social media metrics into my web analytics? -Anne

52

?

Audience Question

#webclinic

53

Is Google Analytics "good enough“ to measure everything I need? -Lou

?

Audience Question

#webclinic

54

What is the best method for calculating incremental click costs for low volume keywords? - Don

?

Audience Question

#webclinic

How should I interpret bounce rates? - Steve

55

?

Audience

#webclinic

56

MarketingExperiments.com/subscribe

http://www.marketingexperiments.com/create-your-marketingexperiments-account.html

http://www.marketingexperiments.com/create-your-marketingexperiments-account.html

Five Steps to Better Metrics

Business

Transcript of Five Steps to Better Metrics