Attribution-fu: Using Correlation Data to Track Marketing Attribution
77
BECAUSE MATH Ian Lurie @portentint [email protected] DIPPING YOUR TOE IN THE WATER W/ CORRELATION #PORTENTU
description
Marketing has become extremely data-driven. I don't love it, but we need to go with it. That means finding more and better ways to track attribution across channels. This slide deck explains some basic techniques - Pearson Correlation and holdout testing - that you can use to connect channel performance to business KPIs.
Transcript of Attribution-fu: Using Correlation Data to Track Marketing Attribution
BECAUSE MATH
Ian Lurie @portentint
DIPPING YOUR TOE IN THE WATER W/ CORRELATION
#PORTENTU
portent.com
portent.co/bccorrel
HOW THIS WILL GO
Disclaimers Use cases & despair Principles 2 ways to test We all collapse
THE LINK BUNDLE
portent.co/bccorrel
DON’T BELIEVE A WORD I SAY
MY MATH QUALIFICATIONS
A degree in history A JD C- in Calculus No statistics training at all
I’VE MADE MISTAKES
LUCKY FOR YOU, I’VE HAD SOME GREAT TEACHERS
aNNIE CUSHING
kevin hillstrom
avinash kaushik
pete meyers
john caples
WHAT I WON’T DISCUSS
GOOGLE ANALYTICS R LANGUAGE TABLEAU SOFTWARE OTHER FANCY STUFF
LOVE IT ALL. BUT let's STICK TO TOOLS WE ALL KNOW AND HAVE, NO MATTER what
learn the rules before you use the tools
WE’VE ALL KNOWN DESPAIR
I WANT TO STOP THE CONTENT CAMPAIGN.
IT'S NOT EARNING ANY MONEY.
SOCIAL MEDIA DOESN'T CONVERT.
DON'T BUY PPC FOR OUR NAME. WE ALREADY RANK
#1. IT'S A WASTE OF MONEY.
THOSE ARE CALL CENTER LEADS.
INTERNET MARKETING HAD NOTHING TO DO
WITH IT.
I'M AFRAID THE WEB IS JUST
CANNIBALIZING DIRECT MAIL.
HOW DO WE KNOW IF THIS IS WORKING? WE
CAN'T TRACK IT.
WE’VE ALL KNOWN DESPAIR
gaaaah i can't take it. attribution?!!!!
hahahahahah i'll attribute my sanity to finding a job
at a bicycle shop bwahahahahahahaha
CORRELATION IS YOUR FRIEND
0 20 40 60 80
100 120 140 160 180
30 35 40 45 50 55 60 65 70
Car S
peed
Age
Car top speed vs. age of American Male
CLEAR CORRELATION
CORRELATION IS YOUR FRIEND WORST ENEMY
0 20 40 60 80
100 120 140 160 180
30 35 40 45 50 55 60 65 70
Car S
peed
Age
Car top speed vs. age of American Male
BUT… WHY?
MIDDLE-AGED AMERICAN MEN ARE COMPENSATING FOR SOMETHING
OR ALL AMERICAN MEN LIKE TOYS, AND CAN AFFORD THEM AT MIDDLE AGE
CORRELATION IS YOUR FRIEND WORST ENEMY
LATE-NIGHT SNACKING MAKES YOU FAT
WRONG
0
10
20
30
40
50
0 2 4 6 8 10
Weigh
t gain
(lbs)
Snacks/week
Late night snacks vs. weight gain
WHY IS IT WRONG?
THERE IS NO CAUSAL LINK. OR, AS STATISTICIANS LIKE TO SAY…
correlation does not equal causation
...but i wouldn't worry about that too
much.
(WE LOVE YOU, MATT)
WE CAN MAKE THIS WORK BY
USING EXCEL/GDOCS USING OUR KNOWLEDGE GETTING A BIT GEEKY TESTING ASSUMPTIONS
EXAMPLE 1
our e-mail list? make it a low priority. it doesn't sell much
anyway.
A HOLDOUT TEST!!!
TEST ONE CHANNEL’S IMPACT ON OTHERS
A HOLDOUT TEST!!!
SEGMENT YOUR AUDIENCE 5%/5%/90%
A HOLDOUT TEST!!!
5% GETS THE USUAL
A HOLDOUT TEST!!!
5% GETS NOTHING FROM THE CHANNEL BEING TESTED
A HOLDOUT TEST!!!
THE RESULT GIVES YOU INSIGHT INTO CROSS-CHANNEL IMPACT
OVERALL RESULT (all channels)
List Size Total sales Revenue/address
Total 400,000 $250,000.00 $0.63
Held out 20,000 $5,000.00 $0.25
E-mailed 20,000 $12,000.00 $0.60
Revenue from all channels for each segment
Average revenue value generated from all channels of a single address
OVERALL RESULT (all channels)
Ah HA! Held out segment generated less value from other channels, too.
E-mail caused these customers to buy more, no matter where they came from.
Organic Search PPC Social E-mail
Total $0.20 $1.25 $0.11 $0.63
Held out $0.17 $0.90 $0.02 $0.25
E-mailed $0.21 $1.24 $0.13 $0.60
OVERALL RESULT (all channels)
E-mail appears to boost PPC revenue/customer 39%
Now you estimate impact by channel.
Organic Search PPC Social E-mail
Total $0.20 $1.25 $0.11 $0.63
Held out $0.17 $0.90 $0.02 $0.25
E-mailed $0.21 $1.24 $0.13 $0.60
EXAMPLE 2
SOCIAL MEDIA SUCKS. LET'S STOP. FOREVER.
UH-OH
THIS IS ABOUT EXISTING & NEW CUSTOMER ACQUISITION.
UH-OH
HOW DO WE TRACK LIFT IN NEW CUSTOMER ACQUISITION?!
OPTION 1
TURN OFF ONE CHANNEL. MEASURE CHANGE IN THE OTHERS.
GENERALLY UNPOPULAR
SHIIIIIIIIIIIII
OPTION 2
BOOST SPEND ON ONE CHANNEL. MEASURE IMPACT ON OTHERS.
ALSO UNPOPULAR
PLUS, LOTS OF NOISE. YOU PROBABLY CAN’T BOOST SPEND 300%.
SHIIIIIIIIIIIII
OPTION 3
IAN’S SEAT-OF-THE-PANTS CORRELATION METHOD
USE AT YOUR OWN RISK. MAY CAUSE
WHAT YOU NEED
A LOT OF DATA CLEAN DATA ANALYTICS THAT WORK INTERNAL SALES DATA (NOT JUST WEB) AN OPEN MIND
WHAT YOU NEED
COMMON SENSE
STEP 1: GET YOUR DATA
YOU MUST HAVE OVERALL SALES!!!!
STEP 2: IMPORT IT
STEP 3: CREATE SCATTERPLOT
THIS TESTS WHETHER YOU HAVE A CHANCE
SELECT 2 COLUMNS CMD+SHIFT OR CTRL+SHIFT
WHICH COLUMNS?
HERE, WE’RE CHECKING FOR CONNECTIONS BETWEEN SOCIAL MEDIA ACTIVITY AND REVENUE, SO I’M STARTING WITH ONE SOCIAL MEDIA METRIC AND OVERALL REVENUE
CLICK
CHECK THIS BOX
IN GENERAL
STEEPER UP-AND-TO-THE-RIGHT = TIGHTER CORRELATION
R SQUARED CLOSER TO 1 =TIGHTER CORRELATION
BUT REMEMBER…
I WAS A HISTORY MAJOR.
HMMMM
LOW R SQUARED= LOW CORRELATION = LOW CHANCES THESE ARE CONNECTED
R² = 0.40636
-
5,000.00
10,000.00
15,000.00
20,000.00
25,000.00
30,000.00
- 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000
Reve
nue (
USD)
Visits from Social Media
Overall Rev
HMMM. MIGHT BE A CONNECTION
HMMMM
LOW R SQUARED= LOW CORRELATION = LOW CHANCES THESE ARE CONNECTED
HMMM. MIGHT BE A CONNECTION
R² = 0.81774
$- $2,000.00 $4,000.00 $6,000.00 $8,000.00
$10,000.00 $12,000.00 $14,000.00 $16,000.00 $18,000.00 $20,000.00
- 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000
Reve
nue (
USD)
Visits from Social Media
PPC Rev
R² = 0.30871
-
5,000.00
10,000.00
15,000.00
20,000.00
25,000.00
30,000.00
- 20 40 60 80 100 120 140
REVE
NUE
SHARES
Overall Rev
HMMMM
WEAKER RELATIONSHIP BETWEEN OVERALL REVENUE AND SHARES
R² = 0.04969
$- $1,000.00 $2,000.00 $3,000.00 $4,000.00 $5,000.00 $6,000.00 $7,000.00 $8,000.00 $9,000.00
$10,000.00
- 500 1,000 1,500 2,000 2,500
REEV
NUE (
USD)
SOCIAL UNIQUE VISITS
E-mail Rev vs. Social Unique Visits
HMMMM
SOCIAL MAY CANNABILIZE E-MAIL
STEP 4: USE CORRELATION
NUMBERS, TO QUANTIFY THE PLOTS
TYPE THIS FORMULA INTO A CELL: =CORREL(RANGE, RANGE2)
IN MY EXAMPLE: =CORREL(D2:D363, K2:K363)
CLOSER TO 1 = STRONG CORRELATION
MY RESULT:
MY RESULT:
DANG
MY RESULT:
HOLY @)(*!@#
correlation does not equal causation
HMMM. A GOOD POINT.
STEP 5: APPLY COMMON SENSE
REMEMBER OUR SCATTERPLOT OF SOCIAL SHARES VS. OVERALL REVENUE?
IT’S A PRETTY GOOD ‘FIT’
REMEMBER OUR SCATTERPLOT OF SOCIAL UNIQUES VS. OVERALL REVENUE?
R² = 0.30871
-
5,000.00
10,000.00
15,000.00
20,000.00
25,000.00
30,000.00
- 20 40 60 80 100 120 140
REVE
NUE
SHARES
Overall Rev
NOT THE STRONGEST, BUT SOMETHING’S GOING ON THERE.
(NO, THE NUMBERS DON’T MATCH UP. JUST SAMPLE DATA. PLUS: HISTORY MAJOR)
MORE COMMON SENSE
KNOWING WHAT I’VE DONE IN SOCIAL MEDIA RECENTLY HELPS, TOO. WE HAVEN’T CHANGED A THING. BUT STILL, THIS CORRELATION.
MORE COMMON SENSE
WE RAN A PROMO, THOUGH, VIA E-MAIL. THAT MIGHT CREATE ‘NOISE.’
MORE COMMON SENSE
SO WE DO THE MATH
MORE COMMON SENSE
THAT MAY MEAN THE E-MAIL PROMO LOOSENED THE RELATIONSHIP BETWEEN E-MAIL AND SOCIAL. SO WE RUN ANOTHER CORRELATION EXCLUDING THE DAYS OF THE PROMO.
MORE COMMON SENSE
THERE’S PROBABLY CANNIBALIZATION GOING ON.
THE RESULT
YOU CAN AT LEAST DEMONSTRATE THERE’S A STRONG CONNECTION BETWEEN REVENUE AND SOCIAL, EVEN IF SOCIAL DOESN’T DIRECTLY GENERATE THAT REVENUE.
NEXT STEP?
DO A HOLDOUT TEST (COUGH) – STOP POSTING FOR 2 WEEKS. TRY TO REALLY SPIKE SOCIAL MEDIA ACTIVITY AND SEE HOW THAT IMPACTS OVERALL REVENUE.
CAUTION
DON’T TRY TO ‘FIT’ THE DATA TO YOUR EXPECTATIONS IF COMMON SENSE <> THE DATA, GO WITH COMMON SENSE UNTIL PROVEN OTHERWISE
CAUTION
REALLY LEARN THIS STUFF
NEXT MONTH
MICHAEL WEIGAND NEXT-LEVEL SEGMENTATION
DIVIDE & CONQUER FEBRUARY 27TH 11 AM PACIFIC
![Otus: Resource Attribution and Metrics Correlation …kair/papers/otus.pdfperformance analysis. DataGarage [11] and OpenTSDB [19] aim at warehousing time-series data for large scale](https://static.fdocuments.us/doc/165x107/5ec5d70009d47022543be3f2/otus-resource-attribution-and-metrics-correlation-kairpapersotuspdf-performance.jpg)
