Post on 18-Aug-2015
Please tweet! @jhenning @LoveStats #mrx
Survey Analysis
in a Nutshell(A Peanut Shell)
Presented by Jeffrey HenningPresident at Researchscape International
Moderated by Annie PettitChief Research Officer at Peanut Labs
Please tweet! @jhenning @LoveStats #mrx
Our Agenda
Thinking about analysis once the results have come in is too late!
Before Analysis
Start looking at each question in isolation, in detail.
It’s better together! Group some questions together, and contrast questions, too.
By Each Question InterrelationshipsCBA
Please tweet! @jhenning @LoveStats #mrx
BEFORE ANALYSIS
Please tweet! @jhenning @LoveStats #mrx
Before You Analyze
Before Completing the questionnaire design, make sure you are answering the big question. And adding the little questions you’ll need.
BC1
Please tweet! @jhenning @LoveStats #mrx
What’s the new price?
Will they buy it?
Are they happy?
THE BIG
QU
ESTI
ON
What do we do next?
Please tweet! @jhenning @LoveStats #mrx
True Stories...
“What media market do they live in?”
“Maybe we can estimate it from their area code,
assuming they haven’t moved.”
“Can you crosstab by county economic tier?”
“I guess we can lookup their county from their zip code,
and the tier from their county...”
“Let’s analyze the results by the annual revenue of the companies
the respondents work for.”
“We didn’t ask that.”
THE
litt
le Q
UES
TIO
NS
Please tweet! @jhenning @LoveStats #mrx
Before You Analyze
Before Completing the questionnaire design, make sure you are answering the big question. And adding the little questions you’ll need.
BC
Before Counting Each response, make sure you want to include each response. Clean your data!
BCE21
Please tweet! @jhenning @LoveStats #mrx
Before You Analyze
Before Completing the questionnaire design, make sure you are answering the big question. And adding the little questions you’ll need.
BC
Before Counting Each response, make sure you want to include each response. Clean your data!
After Data is in hand, think of the big picture. Plan your timeline accordingly.
BCE AD321
Please tweet! @jhenning @LoveStats #mrx
Love the Data but Leave the Data Behind
Data
Price = $2.99
Analysis An Answer
Please tweet! @jhenning @LoveStats #mrx
BY EACH QUESTION
Please tweet! @jhenning @LoveStats #mrx
Please rate your overall satisfaction with each of the
following:
Sample Size: 206 (71% of Respondents)
Please tweet! @jhenning @LoveStats #mrx
“Top 2 Box” Sums Top 2 Values
Sample Size: 206 (71% of Respondents)
Please tweet! @jhenning @LoveStats #mrx
Means are MeanMedian Corrects for Skew without Requiring Judgment Calls
$4.69
$0.99
$2.00
$99.00
Hedges and Shah BMC Bioinformatics 2003 4:31 doi:10.1186/1471-2105-4-31
Please tweet! @jhenning @LoveStats #mrx
Why do you install new apps on your mobile device?
Sample Size: 38 (4% of Respondents)
Please tweet! @jhenning @LoveStats #mrx
Why do you install new apps on your
mobile device?• "I install new apps when I need to be entertained or when I need a tool that I don't
currently have handy such as a calculator or ruler."
• "I always want to be the first to try great apps that I will want to keep. Many of the
new apps that I solve help me solve a problem and I can convey this to my friends,
which makes me look good.”
• "Most of the time it's for entertainment purposes. I like having several app options in
case I am bored. Other times, it's because I need a specific app for something (map,
flight finder, photo editor, etc.)"
• "To be able to do things more efficiently, or to try and discover new games, or to use
apps with more functions and a better interface."
• "I usually download apps that add functionality to my phone, such as a flashlight app
or a sensor reading app. I don't download many game apps, and I almost never
download an app developed for a single business, like a banking app.”
Please tweet! @jhenning @LoveStats #mrx
INTERRELATIONSHIPS
Please tweet! @jhenning @LoveStats #mrx
Cross-tabulate QuestionsTraditional Cross-tabs Draw Eye Away from Important Cells
Please tweet! @jhenning @LoveStats #mrx
Use Cell Comparisons When Possible86% of iPad Users vs. 69% of Android Phone Users
Bought App in Past Month or Less
Please tweet! @jhenning @LoveStats #mrx
Average Related Questions into IndicesCustomer Experience Index, Price Comparison Index, Customer
Satisfaction Index, and a Customer Loyalty Index
CUSTOMER EXPERIENCE
Effectiveness Enjoyability
Ease
PRICECOMPARISONS
To Expectations To Quality
To Competitors
CUSTOMERLOYALTY
RepurchaseLikelihood
Reluctance toSwitch
RecommendLikelihood
CUSTOMERSATISFACTION
Satisfaction Expectations
Comparison with Ideal
Source: Researchscape International
Please tweet! @jhenning @LoveStats #mrx
Indices Highly Correlate to Loyalty BehaviorRank CESL Metric Correlation Shared Variance
1 Customer Loyalty Index 0.762 58%
2 CSAT/Loyalty Index 0.751 56%
3 Champion Model 0.749 56%
4 Likelihood to Repurchase 0.730 53%
5 Likelihood to Recommend 0.728 53%
6 Index of Indices 0.685 47%
7 CSAT 0.639 41%
8 Apostle Model 0.604 36%
9 TNS Loyalty Model 0.590 35%
10 Reluctance to Switch 0.585 34%
18 NPS 0.502 25%
22 Ease 0.402 16%
Please tweet! @jhenning @LoveStats #mrx
Apostle ModelContrast Questions with Quadrant Analyses
• Jones and Sasser pioneered their own loyalty segmentation
• For satisfaction scale, use CSAT question or ACSI’s 3 questions
• For loyalty, use likelihood to repurchase or a loyalty index
Please tweet! @jhenning @LoveStats #mrx
Apostle ModelContrast Questions with Quadrant Analyses
• Jones and Sasser pioneered their own loyalty segmentation
• For satisfaction scale, use CSAT question or ACSI’s 3 questions
• For loyalty, use likelihood to repurchase or a loyalty index
Please tweet! @jhenning @LoveStats #mrx
Beyond Quadrant Analysis Tells a Story53eig.ht/1JkdRVi
Please tweet! @jhenning @LoveStats #mrx
“Stories are just data with a soul.”
Story
Synthesis
Analysis
Data
- Brene Brown, TEDx Houston, 2010
Please tweet! @jhenning @LoveStats #mrx
Summary
Thinking about analysis once the results have come in is too late!
Before Analysis
Start looking at each question in isolation, in detail.
It’s better together! Group some questions together, and contrast questions.
By Each Question InterrelationshipsCBA
Please tweet! @jhenning @LoveStats #mrx
Q&A: Contact InformationAnnie Pettit
Chief Research Officer
annie@peanutlabs.com
ca.linkedin.com/in/AnniePettit/
twitter.com/LoveStats
facebook.com/AnniePettit
Jonathan Cheriff
Director of Sales & Marketing
jonathan.cheriff@peanutlabs.com
Find PeanutLabs on
LinkedIn Facebook Twitter YouTube
DIY POLLINGDIY SUPPORT
Jeffrey Henning
President
jhenning@researchscape.com
www.researchscape.com
twitter.com/jhenning
DIY SAMPLE