BECAUSE MATH

Ian Lurie @portentint

ian@portent.com

DIPPING YOUR TOE IN THE WATER W/ CORRELATION

#PORTENTU

portent.com

portent.co/bccorrel

HOW THIS WILL GO

Disclaimers Use cases & despair Principles 2 ways to test We all collapse

THE LINK BUNDLE

portent.co/bccorrel

DON’T BELIEVE A WORD I SAY

MY MATH QUALIFICATIONS

A degree in history A JD C- in Calculus No statistics training at all

I’VE MADE MISTAKES

LUCKY FOR YOU, I’VE HAD SOME GREAT TEACHERS

aNNIE CUSHING

kevin hillstrom

avinash kaushik

pete meyers

john caples

WHAT I WON’T DISCUSS

GOOGLE ANALYTICS R LANGUAGE TABLEAU SOFTWARE OTHER FANCY STUFF

LOVE IT ALL. BUT let's STICK TO TOOLS WE ALL KNOW AND HAVE, NO MATTER what

learn the rules before you use the tools

WE’VE ALL KNOWN DESPAIR

I WANT TO STOP THE CONTENT CAMPAIGN.

IT'S NOT EARNING ANY MONEY.

SOCIAL MEDIA DOESN'T CONVERT.

DON'T BUY PPC FOR OUR NAME. WE ALREADY RANK

#1. IT'S A WASTE OF MONEY.

THOSE ARE CALL CENTER LEADS.

INTERNET MARKETING HAD NOTHING TO DO

WITH IT.

I'M AFRAID THE WEB IS JUST

CANNIBALIZING DIRECT MAIL.

HOW DO WE KNOW IF THIS IS WORKING? WE

CAN'T TRACK IT.

WE’VE ALL KNOWN DESPAIR

gaaaah i can't take it. attribution?!!!!

hahahahahah i'll attribute my sanity to finding a job

at a bicycle shop bwahahahahahahaha

CORRELATION IS YOUR FRIEND

0 20 40 60 80

100 120 140 160 180

30 35 40 45 50 55 60 65 70

Car S

peed

Age

Car top speed vs. age of American Male

CLEAR CORRELATION

CORRELATION IS YOUR FRIEND WORST ENEMY

0 20 40 60 80

100 120 140 160 180

30 35 40 45 50 55 60 65 70

Car S

peed

Age

Car top speed vs. age of American Male

BUT… WHY?

MIDDLE-AGED AMERICAN MEN ARE COMPENSATING FOR SOMETHING

OR ALL AMERICAN MEN LIKE TOYS, AND CAN AFFORD THEM AT MIDDLE AGE

CORRELATION IS YOUR FRIEND WORST ENEMY

LATE-NIGHT SNACKING MAKES YOU FAT

WRONG

0

10

20

30

40

50

0 2 4 6 8 10

Weigh

t gain

(lbs)

Snacks/week

Late night snacks vs. weight gain

WHY IS IT WRONG?

THERE IS NO CAUSAL LINK. OR, AS STATISTICIANS LIKE TO SAY…

correlation does not equal causation

...but i wouldn't worry about that too

much.

(WE LOVE YOU, MATT)

WE CAN MAKE THIS WORK BY

USING EXCEL/GDOCS USING OUR KNOWLEDGE GETTING A BIT GEEKY TESTING ASSUMPTIONS

EXAMPLE 1

our e-mail list? make it a low priority. it doesn't sell much

anyway.

A HOLDOUT TEST!!!

TEST ONE CHANNEL’S IMPACT ON OTHERS

A HOLDOUT TEST!!!

SEGMENT YOUR AUDIENCE 5%/5%/90%

A HOLDOUT TEST!!!

5% GETS THE USUAL

A HOLDOUT TEST!!!

5% GETS NOTHING FROM THE CHANNEL BEING TESTED

A HOLDOUT TEST!!!

THE RESULT GIVES YOU INSIGHT INTO CROSS-CHANNEL IMPACT

OVERALL RESULT (all channels)

List Size Total sales Revenue/address

Total 400,000 $250,000.00 $0.63

Held out 20,000 $5,000.00 $0.25

E-mailed 20,000 $12,000.00 $0.60

Revenue from all channels for each segment

Average revenue value generated from all channels of a single address

OVERALL RESULT (all channels)

Ah HA! Held out segment generated less value from other channels, too.

E-mail caused these customers to buy more, no matter where they came from.

Organic Search PPC Social E-mail

Total $0.20 $1.25 $0.11 $0.63

Held out $0.17 $0.90 $0.02 $0.25

E-mailed $0.21 $1.24 $0.13 $0.60

OVERALL RESULT (all channels)

E-mail appears to boost PPC revenue/customer 39%

Now you estimate impact by channel.

Organic Search PPC Social E-mail

Total $0.20 $1.25 $0.11 $0.63

Held out $0.17 $0.90 $0.02 $0.25

E-mailed $0.21 $1.24 $0.13 $0.60

EXAMPLE 2

SOCIAL MEDIA SUCKS. LET'S STOP. FOREVER.

UH-OH

THIS IS ABOUT EXISTING & NEW CUSTOMER ACQUISITION.

UH-OH

HOW DO WE TRACK LIFT IN NEW CUSTOMER ACQUISITION?!

OPTION 1

TURN OFF ONE CHANNEL. MEASURE CHANGE IN THE OTHERS.

GENERALLY UNPOPULAR

SHIIIIIIIIIIIII

OPTION 2

BOOST SPEND ON ONE CHANNEL. MEASURE IMPACT ON OTHERS.

ALSO UNPOPULAR

PLUS, LOTS OF NOISE. YOU PROBABLY CAN’T BOOST SPEND 300%.

SHIIIIIIIIIIIII

OPTION 3

IAN’S SEAT-OF-THE-PANTS CORRELATION METHOD

USE AT YOUR OWN RISK. MAY CAUSE

WHAT YOU NEED

A LOT OF DATA CLEAN DATA ANALYTICS THAT WORK INTERNAL SALES DATA (NOT JUST WEB) AN OPEN MIND

WHAT YOU NEED

COMMON SENSE

STEP 1: GET YOUR DATA

YOU MUST HAVE OVERALL SALES!!!!

STEP 2: IMPORT IT

STEP 3: CREATE SCATTERPLOT

THIS TESTS WHETHER YOU HAVE A CHANCE

SELECT 2 COLUMNS CMD+SHIFT OR CTRL+SHIFT

WHICH COLUMNS?

HERE, WE’RE CHECKING FOR CONNECTIONS BETWEEN SOCIAL MEDIA ACTIVITY AND REVENUE, SO I’M STARTING WITH ONE SOCIAL MEDIA METRIC AND OVERALL REVENUE

CLICK

CHECK THIS BOX

IN GENERAL

STEEPER UP-AND-TO-THE-RIGHT = TIGHTER CORRELATION

R SQUARED CLOSER TO 1 =TIGHTER CORRELATION

BUT REMEMBER…

I WAS A HISTORY MAJOR.

HMMMM

LOW R SQUARED= LOW CORRELATION = LOW CHANCES THESE ARE CONNECTED

R² = 0.40636

-

5,000.00

10,000.00

15,000.00

20,000.00

25,000.00

30,000.00

- 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000

Reve

nue (

USD)

Visits from Social Media

Overall Rev

HMMM. MIGHT BE A CONNECTION

HMMMM

LOW R SQUARED= LOW CORRELATION = LOW CHANCES THESE ARE CONNECTED

HMMM. MIGHT BE A CONNECTION

R² = 0.81774

$- $2,000.00 $4,000.00 $6,000.00 $8,000.00

$10,000.00 $12,000.00 $14,000.00 $16,000.00 $18,000.00 $20,000.00

- 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000

Reve

nue (

USD)

Visits from Social Media

PPC Rev

R² = 0.30871

-

5,000.00

10,000.00

15,000.00

20,000.00

25,000.00

30,000.00

- 20 40 60 80 100 120 140

REVE

NUE

SHARES

Overall Rev

HMMMM

WEAKER RELATIONSHIP BETWEEN OVERALL REVENUE AND SHARES

R² = 0.04969

$- $1,000.00 $2,000.00 $3,000.00 $4,000.00 $5,000.00 $6,000.00 $7,000.00 $8,000.00 $9,000.00

$10,000.00

- 500 1,000 1,500 2,000 2,500

REEV

NUE (

USD)

SOCIAL UNIQUE VISITS

E-mail Rev vs. Social Unique Visits

HMMMM

SOCIAL MAY CANNABILIZE E-MAIL

STEP 4: USE CORRELATION

NUMBERS, TO QUANTIFY THE PLOTS

TYPE THIS FORMULA INTO A CELL: =CORREL(RANGE, RANGE2)

IN MY EXAMPLE: =CORREL(D2:D363, K2:K363)

CLOSER TO 1 = STRONG CORRELATION

MY RESULT:

MY RESULT:

DANG

MY RESULT:

HOLY @)(*!@#

correlation does not equal causation

HMMM. A GOOD POINT.

STEP 5: APPLY COMMON SENSE

REMEMBER OUR SCATTERPLOT OF SOCIAL SHARES VS. OVERALL REVENUE?

IT’S A PRETTY GOOD ‘FIT’

REMEMBER OUR SCATTERPLOT OF SOCIAL UNIQUES VS. OVERALL REVENUE?

R² = 0.30871

-

5,000.00

10,000.00

15,000.00

20,000.00

25,000.00

30,000.00

- 20 40 60 80 100 120 140

REVE

NUE

SHARES

Overall Rev

NOT THE STRONGEST, BUT SOMETHING’S GOING ON THERE.

(NO, THE NUMBERS DON’T MATCH UP. JUST SAMPLE DATA. PLUS: HISTORY MAJOR)

MORE COMMON SENSE

KNOWING WHAT I’VE DONE IN SOCIAL MEDIA RECENTLY HELPS, TOO. WE HAVEN’T CHANGED A THING. BUT STILL, THIS CORRELATION.

MORE COMMON SENSE

WE RAN A PROMO, THOUGH, VIA E-MAIL. THAT MIGHT CREATE ‘NOISE.’

MORE COMMON SENSE

SO WE DO THE MATH

MORE COMMON SENSE

THAT MAY MEAN THE E-MAIL PROMO LOOSENED THE RELATIONSHIP BETWEEN E-MAIL AND SOCIAL. SO WE RUN ANOTHER CORRELATION EXCLUDING THE DAYS OF THE PROMO.

MORE COMMON SENSE

THERE’S PROBABLY CANNIBALIZATION GOING ON.

THE RESULT

YOU CAN AT LEAST DEMONSTRATE THERE’S A STRONG CONNECTION BETWEEN REVENUE AND SOCIAL, EVEN IF SOCIAL DOESN’T DIRECTLY GENERATE THAT REVENUE.

NEXT STEP?

DO A HOLDOUT TEST (COUGH) – STOP POSTING FOR 2 WEEKS. TRY TO REALLY SPIKE SOCIAL MEDIA ACTIVITY AND SEE HOW THAT IMPACTS OVERALL REVENUE.

CAUTION

DON’T TRY TO ‘FIT’ THE DATA TO YOUR EXPECTATIONS IF COMMON SENSE <> THE DATA, GO WITH COMMON SENSE UNTIL PROVEN OTHERWISE

CAUTION

REALLY LEARN THIS STUFF

QUESTIONS?

IAN@PORTENT.COM @PORTENTINT

WWW.PORTENT.COM

portent.co/bccorrel

NEXT MONTH

MICHAEL WEIGAND NEXT-LEVEL SEGMENTATION

DIVIDE & CONQUER FEBRUARY 27TH 11 AM PACIFIC