BASIC STATISTICS

for PAID SEARCH ADVERTISING

Katharine Mission-EstenzoSGS.com Search Engine Marketing LeadResearch & Testing Specialist andQuality Management Coordinator

PPC Pinas Meetup 2013May 31, 2013Cypress Towers, Taguig

2

OBJECTIVE & SCOPE

Introduce Statistical Concepts and Tools to efficiently manage campaigns and results

Correct common misuses and misconceptions on Basic Statistical concepts

Not a Statistics crash course - Guaranteed formula-free presentation!

3

TOPICS

Statistical Sampling and Analysis

Charts and Graphs

Common Numerical Misuses

Prediction and Forecasting

Statistical Process Control

4

STATISTICS

Collection

Analysis

Interpretation

Presentation

5

STATISTICAL SAMPLING?

Population

Target Population

Samples

Selected

SAMPLING TECHNIQUES Simple Random Sampling

Systematic Sampling

Stratified Sampling

Probability-proportional-to-size sampling

Accidental / Purposive Sampling

Quota Sampling

Clustered Sampling

SAMPLING PROCESS

Define the population of concern

Specify a sampling frame

Develop a sample plan

Implementing the sample plan

Sampling and data collecting

6

WHY USE A SAMPLE?

• Lower Costs• Faster Data CollectionResearch

• Validity of Results• Robustness of

Statistical Model• Statistical Significance

Testing

SA

MP

LIN

G E

RR

OR

S

• History

• Instrumentation

• Selection

• Sampling Distortion

7

SIGNIFICANT VS STATISTICALLY SIGNIFICANT

SIGNIFICANT

Important

Essential

Meaningful

STATISTICALLYSIGNIFICANT

Pattern

Behavior

Not by ChanceBefore making conclusions, always make sure that you have sufficient sample size. All test results are invalid if:

insufficient sample sizesampling errors

8

SAMPLING ERROR = MARGIN OF ERROR

Sampling Error

• Failure to capture the profile of the true population- under representation.

Margin of Error

• The difference of the estimated value to the true population value

9

GRAPH IT!

HACCP ISO 22000 GMP FSSC 22000

BRC

0

200

400

600

800

1000

1200

1400

1600

Page VisitsJanuary 2013

Jan February March April

0

200

400

600

800

1000

1200

1400

1600

1800

Monthly Page VisitsJan - Apr 2013

HACCP GMP FSSC 22000

HACCP

33%

ISO 2200022%

GMP17%

FSSC 2200014%

BRC12%

Page VisitsJanuary 2013

Discrete/ count data – Impressions, Clicks, Conversions

Comparing data based on a single category/ criteria

Change in magnitude/ quantity

Continuous data – CTR, Conv Rate, CPC

Tracking changes over time

Trends

Correlations

Portions/ percentages of a whole – Geo performances

One variable at a time

Limit your data – use bar charts for more than six variables

Avoid using 3D rotation - deceiving

10

COMBINATION GRAPHS

Madrid Valencia Mallorca Zaragosa Tenerife

0

10

20

30

40

50

60

$0.00

$1.00

$2.00

$3.00

$4.00

$5.00

$6.00

Conversions vs CPA

Jan February March April

0.00%

10.00%

20.00%

30.00%

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

33.39% 31.90% 27.38% 27.16%

22.26% 24.07% 28.33% 28.86%

17.47% 19.06% 18.79% 16.98%

14.47% 13.14% 11.33% 11.88%

12.42% 11.84% 14.16% 15.11%

Monthly Search Traffic Share

HACCP ISO 22000 GMP FSSC 22000 BRC

Algeria Nigeria Saudi Kurdistan Kenya

0

5000

10000

15000

20000

25000

Clicks Clicks ClicksClicks

Clicks

Impressions Impressions Impressions

ImpressionsImpressions

When using combination graphs (or even simple graphs), keep in mind that your objective is to simplify data presentation. Present trends and changes in the simplest form.

Do not complicate your graphs just to give the impression of “advanced” analysis and/or analytical skills.

11

LIES, DAMNED LIES AND STATISTICS

The Danger of Averages

Bill Gates walk into a bar; on average, everybody in the bar is a millionaire.

The average human has one breast and one testicle. ~Des McHale

The interesting thing about averages is that they hide the truth very effectively. ~Avinash Kaushik

12

MEASURES OF CENTRAL TENDENCY

DayEarning(USD)

Day 1 350.00

Day 2 400.00

Day 3 400.00

Day 4 5,500.00

Day 5 150.00

Day 6 300.00

Day 7 400.00

Day 8 400.00

Day 9 400.00

Day 10 400.00

Total 8,700.00

ON IMPULSE:

My average daily earning is USD 870.00.

MEANAverage

Minimal differences

Widely dispersed data

Extremes and outliers

MEDIANMiddle value

Most resistant to outliers and extreme values

If data points are even, this is the mean of the 2 middle values

MODEMost often appears

Most likely to be sampled

Not unique – data set may be mutli-modal

13

Percentage Fallacies and Misuses

Using pure percentage values to measure effectiveness CTR

Conversion Rates

Averaging Percentages – valid or not?

Trials Successes %

10 6 60.00%

25 10 40.00%

30 10 33.33%

40 5 12.50%

Totals 105 31145.83

%

AVERAGE = 36.46 %

AVERAGE = 29.52 %

14

The Excuse of Trends and Seasonality

TREND - General tendency of a series of data points to move in a certain direction over time

Consecutive data points moving in a single direction

Majority of data points moving in a single direction

Extreme values, singular peak values and outliers (Noise) are flattened in trend analysis

SEASONALITY – Characteristic of a time series in which the data has regular and predictable changes on a specific period recurring every calendar year

Always check previous data for the same time period

Not all holidays are causal to seasonality

15

PREDICTION AND FORECASTING

TIME SERIES A sequence of data points measured successively in uniform time

intervals Use of a statistical model to predict future values based on previous

observations

! Assuming that conditions stay the same.

REGRESSION ANALYSIS A technique for estimating the relationships between variables

The value of a dependent variable is affected by the behavior of the values of the independent variables

! Check data for conformance to statistical assumptions.

16

STATISTICAL PROCESS CONTROL

FMEA – Failure Mode and Effects Analysis

Identifying potential mistakes before they happen to determine whether the effects are tolerable or not

FME(C)A – includes criticality analysis

Efficient assessment of best option

Evaluate effects of proposed changes on processes & performances

Manage risks associated with system failures and changes

Standardize procedures and practices

17

Design

Measure

Analyze

Improve

Control

DMAIC – Six Sigma Core Concept

Campaign Objectives Nature of Business Advertising Channels Type of Testing

Gap analysis/ Benchmark

Historical Data Data Collection/ Testing

Identify sources of variation Identify critical factors Validation of results

Discover process relationships

Implement optimization/ improvements

FMEA

Documentation Develop Control Plan Monitoring

18

REMINDERS

TEST! Don’t rely on assumptions.

Efficiency – cost, time, energy

Always define objectives and targets clearly

Plan carefully – ensure objectives are met

Understand your data – how, where, what and when

Statistics – Bane or Boon?

19

QUESTIONS/ CONSULTATION

katha.mission@gmail.com

nina.mission

THANK YOU!www.sgs.com