THE BLACK BOOK OF OUTSOURCING 2009 SURVEY Top Mortgage Banking
Pinnion survey book
-
Upload
pinnion -
Category
Self Improvement
-
view
840 -
download
0
description
Transcript of Pinnion survey book
Customer Opinion and Audience Response
Surveys and trivia games that are Easy to Make, Fun to Take
Bill Leath and Barry Fuchs
© 2012 Pinnion
www.pinnion.com
Customer Opinion and Audience Response
2
Table of Contents Introduction .................................................................................................................................................. 3
Why? ............................................................................................................................................................. 5
Survey justification statement .............................................................................................................. 7
Who? ............................................................................................................................................................. 9
Your Customers ......................................................................................................................................... 9
Your Colleagues ....................................................................................................................................... 11
What? .......................................................................................................................................................... 14
Multiple choice and structured communication .................................................................................... 14
Content ................................................................................................................................................... 16
Where? ........................................................................................................................................................ 18
When? ......................................................................................................................................................... 20
How? ........................................................................................................................................................... 22
Answer options ....................................................................................................................................... 23
Dichotomous ....................................................................................................................................... 23
Multiple choice ................................................................................................................................... 23
Rating scales – Likert and Semantic Differential ................................................................................. 24
Image select ........................................................................................................................................ 25
Demographic ....................................................................................................................................... 25
Skip logic and branching ..................................................................................................................... 26
Text piping ........................................................................................................................................... 27
Distribution options ................................................................................................................................ 28
Conclusion ................................................................................................................................................... 31
About Us ..................................................................................................................................................... 32
Bibliography ................................................................................................................................................ 33
www.pinnion.com
3
Introduction
“Let’s do a survey!” You’ve probably heard that phrase countless times around the office, at a meeting
of your favorite club or association, or while planning events at your child’s school. After all, what better
way is there to show that you really want to know what’s on the minds of your customers, members, co-
parishioners, fellow parents, etc? A survey says “I’m listening,” provides you with information, and can
take the pressure off of making big decisions. That’s the good side.
There is a bad side to surveys, though. Or, should we say, a dark side. After all, dark is the bottom of the
trash can where countless ignored surveys have ended up. And, dark are the desk drawers where many
survey results get stuffed away – quickly forgotten by managers who are too busy to remember why
they even created a survey in the first place. When survey requests appear on every sales receipt and
arrive in your inbox after each hotel stay or car repair, it’s easy to see why “survey fatigue” is causing
many of us to cringe each time we’re asked to give feedback (Peltz, 2012).
That’s why it’s time for a new type of survey guide. Sure, it helps know your dichotomous question from
your likert scale if you want to write a good survey, but we believe it’s more important than ever to
think of surveys as snapshots of customer opinion. Like a photo snapshot, they should be easy, quick,
and fun for everyone involved. Like a traditional survey, they should provide you, the creator, with
meaningful information that’s easy to process, understand, and share with others. Bottom line: surveys
should be easy to make and fun to take.
We’ve tried to organize this book so that it’s easy and fun for you, too. The easy part comes from our
“just the facts” approach, where you’ll learn about surveys through these familiar chapters:
Why
Who
What
Where
When
How
The fun should come naturally, because we actually want you to read this book and learn how to create
fun surveys. Just in case we’re not fun, or funny, enough on our own, we’re also going to explain how
you can include trivia, quizzes, and other fun stuff in your surveys so that your customers feel less like
they’re doing their chores and more like they’re actually have a good time interacting with your survey.
Customer Opinion and Audience Response
4
Note: We will use “customer” throughout this book. Please feel free to substitute “readers,”
“members,” “co-workers,” “classmates,” or whatever term best describes your target audience.
www.pinnion.com
5
Why?
One of our main goals is to help you create surveys that are
actionable – that allow decisions to be made based on the
responses that you receive. It’s for this reason that “Why?” should
be the first question that you ask, and answer, when you’re
considering whether to conduct a survey.
As you come up with answers to that question, you’ll probably be
surprised at how many different reasons there can be to create a
survey. Here are just a few examples:
Keep in touch with your customers between purchases
Remind customers of your brand at times when you would
like for them to make a purchase
Solicit opinions on a new product line before making a
major investment
Generate fresh content for your Facebook page or website
Get feedback on the quality of your company’s customer
service
We’ll cover more examples throughout this book, but the
important thing to notice is how the strategy behind surveys has
changed. Three of the five examples listed above are designed to
do something more than just collect customer opinions. They are
maintaining relationships, drawing customers in to your website or
brand, and perhaps even driving additional purchases. The nature
of online surveys allows you to do this in a way that doesn’t detract
from the primary reason that most surveys are created: to collect
customer opinions.
Until now, surveys have often felt like transactions, because that’s
how they’ve been designed. And, frankly, how many of us want to
go through another transaction – in the form of the survey – to ask
about an earlier transaction – such as going through the grocery
checkout – that we already completed? To add insult to injury,
If we were going to write
a news article about your
survey, then “Why” might
be the last thing that we
mention. For example, we
could write “Our most
awesome reader surveyed
his audience of 6,000
people in the Chicagoland
area through his blog and
a mobile survey app
during the second week of
March because he needed
their input to create the
biggest impact with his
upcoming project.” In this
case, you learned about
the who, what, when,
how, and where before
we got to the why:
“because he needed their
input to create the
biggest impact with his
upcoming project.”
Surveys are different,
though. It’s important to
always ask Why as the
first step of your research.
Customer Opinion and Audience Response
6
these transactional surveys often arrived at inconvenient times, which is exactly why so many of us got
in the habit of simply deleting the emails and hanging up the phone whenever we were asked to
participate.
Wegman’s, a popular East Coast grocery chain based in Rochester, NY, has moved away from this
transactional model through the creation of the Wegman’s Opinion Panel. Customers on the panel
“have agreed to participate in ongoing Wegmans surveys as part of their membership” and “directly
influence the direction of future products and services that benefit (the members of the panel).” Wow!
The word “panel” alone makes this online group of customers sound prestigious and Wegmans talks a
good game about how much influence the panel can have over product decisions. But, is it real or just
marketing-speak? Allow this Wegman’s customer to answer that question with her enthusiastic Tumblr
post:
Double wow! Imagine having customers who not only read your emails, but actually look forward to
having them arrive. The right mix of feedback opportunity and valuable content can make this possible.
This is not to say that you’ll only hear from happy customers who want to shower you with praise. That
shouldn’t be the goal, either, because hearing about the negative situations that some of your
customers have experienced provides important opportunities to improve your products and services.
Remember, this is the age of oversharing and constant status updates. If your unhappy customer can
quickly and easily provide direct feedback, knowing that someone on the other end is listening, you may
avoid an off-the-cuff, upset post to Facebook and get the chance to turn his experience around. This can
Figure 1 – Source: http://www.tumblr.com/tagged/wegmans?before=1314840187
Key Point – The Why of Surveys: Surveys are no longer a one-way proposition, e.g. “give us information that’s valuable to us.” Now, surveys and survey results can be offered as content that provides value back to your customers.
www.pinnion.com
7
be especially valuable if you or your staff are able to correct a simple misperception that the customer
had about your products or services.
Keeping all of this in mind, let’s get down to the reason(s) why you want to create a survey. You may
want to start by filling in the blanks of this statement:
Survey justification statement
This survey will provide (name(s) of decision makers or committee) with the necessary data to (decision that
will be made) by asking (customers, members, etc.) to provide opinions regarding (a past experience or upcoming
situation).
We recommend a survey for this project instead of other feedback mechanisms because (list of
reasons that other options were rejected). We have already checked for this data at (potential existing sources of
information that have already been explored) and have not found the information that we need.
That last sentence is very important, because surveys are just one way of getting the information that
you need. And, for all of the advantages of modern surveys, it is still possible to annoy your customers
by over-surveying them. This is especially true if you ask them a question that has already been asked
before, which gives the impression that the information they provided to you earlier was simply ignored.
So, think of all of the places where the information that you need could already be lurking: sales reports,
surveys conducted by other departments, trade publications, Census data, etc. You might find enough
existing data to realize that you don’t need to create a survey at all. That’s great! Seriously. You’ve saved
yourself, your customers, and your organization valuable time and money. You’ll find another reason to
do a survey soon enough, so don’t worry if you end up not needing to do one right at this moment. The
rest of this book will still be here when you need it.
Even if you’ve determined that the necessary data isn’t available, there’s still the question of whether a
survey is the most appropriate method of getting the information that you want or need. Let’s say that
one of your product lines has experienced a substantial drop in sales during the past year. You could
survey your customers to find out what they like and don’t like about that product, but the first step
may be to do some industry research to determine if your competitors are seeing a similar drop in sales.
The clues provided by other types of research may persuade you that a survey is not necessary. Or, you
may still want to do a survey, but with a different set of questions than you had originally planned to
use.
Customer Opinion and Audience Response
8
Other types of research include:
Primary sources
Focus groups
Secret shoppers
Committees and advisory boards
Individual interviews (aka depth interviews)
Mall Intercepts – quick, face-to-face feedback solicited from consumers at shopping malls
Secondary sources
Internet searches
Industry reports, such as Hoover’s and IBISWorld
Books, magazines, and trade publications
Census data
Survey data from other organizations
If you’ve made it this far, then you must have decided that a survey is right tool for getting the
information that you need. And, by taking a bit of time to get to the Why of your survey, you also figured
out a lot about the Who, What, and How. Let’s take a look at the first of these in the next section.
www.pinnion.com
9
Figure 2: The Two Whos - Your Audience and Your Colleagues
Your Customers
• Needs clear, concise survey questions
• To provide honest, thoughtful answers
Your Colleagues
• Need a clear, concise summary of answers
• To make good decisions based on audience opinion
Who? When you start to answer this question, it probably won’t take you long to figure out that there are
really two sets of Whos that you need to think about. In fact, it definitely won’t take you very long
because we are about to tell you:
One set of Whos are the
people who will be
responding to your survey,
your customers. The other
set of Whos are the people
who will make decisions
based on the survey results,
your colleagues. Your
ultimate goal is to provide
your colleagues with the
data that they need in the
clearest, most concise
format possible. This means
that you need to create a survey that’s equally clear and concise, so that your customers will provide
honest, thoughtful answers to each of the questions that you present.
Your Customers
We’ve written this book as a resource for people who want to gather customer opinions for use at a
business or non-profit organization. We are not scientists and assume that you are not, either, so you
are likely interested in gathering a “sufficiently reliable” (Shapiro, 2008) estimate of customer opinion
through your surveys. In other words, you’re interested in creating a simple tool that will give you a
Key Point – The Who of Surveys: Your colleagues and your customers both want to contribute and provide good information. Both groups are also made up of very busy individuals, so it’s critical to respect everyone’s time and good nature by ensuring that each survey question:
• Is presented only to the proper individuals • Will provide valuable data for your colleagues or yourself
Customer Opinion and Audience Response
10
good idea of customer opinions, not in developing a complex study that strives for a perfect set of
results.
One way to do get good results is to ensure that your sample, or the number of people who respond to
your survey, is large enough to provide you with good “ballpark” data that can be assumed to represent
the opinions of your customer base as a whole. One number that’s batted around quite frequently is 30.
As explained wonderfully in these charts
from Jed Campbell, 30 is the magic number
where things become a lot more stable,
even if you would need a much larger
sample in order to conduct a formal study.
If you’d like a better idea of the number of
responses you’ll need for a meaningful
sample, check out the Sample Size
Calculator available online from Creative
Research Systems.
These estimates assume that you will be
using a random sample of respondents
who, in turn, will be representative of a
larger group of random potential
customers. Of course, if you are asking
current customers to participate in your
survey, then your sample will not be
random because your respondents will all
have at least one trait in common: they are
all customers of your business. The sample
will be even less random if you undertake some of the relationship-building strategies that we
mentioned in the Why section, because now your respondents all have two things in common: they are
customers and they have agreed to participate in a series of surveys that you send to them over time.
This is not necessarily a problem in itself and does not suggest that the information that you receive isn’t
valuable. It just may not be scientifically-valid. You’ll need to assess whether the results you receive are
Figure 3 – Source: http://www.jedcampbell.com/?p=262
www.pinnion.com
11
indicative of your base of customers or potential customers as a
whole. Perhaps you’ll learn that your respondents are pickier or
more demanding, but also more loyal, and keep that in mind as you
review and evaluate the feedback that they provide.
You also want to be sure that you’re approaching a relevant group,
or segment, of customers with the invitation to participate in your
survey. By determining who you wish to invite, and why, you’ll help
to ensure the validity of the opinions you receive. You’ll also avoid
bothering a group of individuals who likely will not have any
interest in a particular survey, thereby increasing the chance that
they’ll read the next communication that you send their way.
Determining the right people doesn’t stop at the survey level. It
goes right down to the individual questions. You probably
remember old paper surveys that asked you to, for example, Skip
to Question 9 depending on your answer to Question 6. With
online and mobile surveys, there’s no need to ask individuals to
skip ahead, because you can automate your survey to do the
skipping for them. This is called Question Branching, or Skip Logic,
and we’ll talk about it more in the How section. For now, just
remember that each survey and each question should be
presented only to those who can help you to get the information
that you need.
Your Colleagues
There are a few phrases that you don’t want to hear from your
colleagues as you walk into the conference room, giddy with
excitement from all of the great data you were able to collect from
your survey respondents:
“So what?”
“I’ll take a look at this next week.”
Imagine that you run a
fashion brand for men
and women. If you’re
planning to ask for
opinions about ladies
handbags, then you’ll
probably want to invite
only women to answer
your survey.
Probably, but not
definitely.
If you’re nearing the
holiday season and
believe that men may be
interested in purchasing
handbags as gifts, then it
may be worth sending the
survey to them as well.
And, if you’re in touch
with a group of men and
women who blog about
fashion, then you may
want to include all of
them, or none of them, in
your invitation.
Customer Opinion and Audience Response
12
“Wait, we did a survey?”
“Just give a copy to my assistant.”
It’s not hard to imagine how deflating these comments would be. After all, you’ve worked to collect this
data and now it’s clear that the results won’t see the light of day. What’s more, your customers took the
time to share their opinions – how would they feel to know that the information was just filed away,
never to be seen again?
The key to avoiding this ugliness and despair is to remember the ultimate goal that we mentioned
earlier in this chapter: to provide clear, concise data. This starts by thinking about your colleagues as you
create your survey questions. When you come up with a question that you would like to ask, consider
how the data that comes
from the responses to that
question will be used.
Questions without a clear
purpose should be
eliminated to shorten the
survey and save time for
everyone involved
(Walonick, 2004). This way,
you’ll end up with survey
results full of must-have
information that your
colleagues will be
clamoring for, instead of a
bland report with nice-to-
know facts that are easily ignored.
Your colleagues also have a role to play during survey preparation: helping to test it out before
distribution to your customers. If your survey is longer than two or three questions, then there’s a good
chance it has some complexity to it. Testing will not only find any mistakes, it will also alert you to any
questions or multiple-choice answers that are confusing, ambiguous, or out of order. Think about all of
the ways that you plan to distribute your survey – online, mobile app, website embed, mobile website,
etc. – and make sure that at least one of your testers attempts to take the survey via each of these
Create
•Each question should provide data that your colleagues need
•Eliminate any question that only gives "nice to know" information
Test
•Ask colleagues to look for confusing or out-of-order questions
•Try taking the survey in multiple ways, including on a smartphone
Share •Explain the Why of the survey and give an overview of results
•Different audiences will need the results in different formats
Figure 4 - Basic steps for a clear, concise survey
www.pinnion.com
13
platforms (Flagg, 2011). Remember to also test the various types of online browsers that are available,
including Google Chrome, Mozilla Firefox, Microsoft Internet Explorer, and Apple Safari.
When it comes time to present the results, remember the importance of selling the recipients on the
importance of the data that you have collected. Give them a brief overview of the Why for your survey,
so that they can see why the survey was the right application for this research challenge, along with a
summary of the key learnings that you have identified from the results. Consider the ways in which your
results should be presented and keep in mind that you may need to make multiple versions available: a
PDF of simple charts and graphs may be appropriate for senior managers and committee members who
need a quick view of the key information, while department managers may appreciate having access to
the complete results set in spreadsheet format so that they can sort and calculate the data however
they wish.
Customer Opinion and Audience Response
14
What?
We mentioned in the last section that there are at least four different ways to distribute an online
survey via the Internet and mobile devices, plus at least four different web browsers. (We won’t even
get into all of the different versions available for each browser.) This is remarkable considering that
household telephone ownership did not become prevalent enough to make telephone surveys possible
until the 1960s and 70s (Kohut, 2009). In other words, it only took about three decades to go from an
age when most polling was done through personal interviews to today’s world of instantaneous
response by individuals who can share their opinions from nearly any location in the world.
The fact that it’s so easy to create surveys today means that there has been an immeasurable increase in
the number of surveys, not to mention in the number of people who create surveys. Many survey
creators today do not work with surveys on a full-time basis and have no formal training in survey
methodology or market research. That’s why we wrote this book and it’s probably why you’re reading it.
So, this section is devoted to exploring a topic that may seem obvious: What is a survey? In short, it’s a
questionnaire or poll that is distributed to a group of constituents – random or not – for the purpose of
collecting information that can aid the decision-making process. Now, let’s break it down and find out
about the elements that can ensure your online survey gets a lot of love, not hate (Or, as they say on
Twitter, an #FTW, not a #Fail.)
Multiple choice and structured communication
The fact that you have chosen to use a survey, instead of another type of primary or secondary research,
suggests that many of your questions will be structured with multiple choice options for the answer.
After all, if you were interested in detailed or open-ended responses, then focus groups or individual
interviews would be a better option for collecting information. These qualitative forms of research have
their advantages:
Complete, unfiltered opinions directly from customers
Ability to ask detailed follow-up questions
Face-to-face interaction can develop brand loyalty that is not possible with other research
methods.
www.pinnion.com
15
On the other hand, focus groups and individual interviews are extremely high-cost in both time and
money. Most organizations simply cannot afford to do them very often, if at all. The information
gathered through these methods is also unstructured, meaning that no two answers will be exactly the
same. It takes a lot of time and effort to attempt to summarize open-ended responses so that they can
be compared to one another. And, the original words spoken by respondents are often lost once the
summary is created anyway.
We believe that the quantitative nature of structured communication provides a framework to simplify
the research process in many ways:
Respondents can move quickly through each question by choosing from a list of pre-determined
answers
Creators get a data that’s easy to analyze, so that trends and preferences can be easily spotted
The data can be processed and calculated in a variety of ways.
Answer choices can be re-used at the same time or in future surveys to allow for comparisons
between different groups of respondents.
Sheetz, a regional convenience store chain, uses structured communication to take customer orders via
in-store kiosks. This speeds the ordering process by showing customers exactly what is available for each
sandwich. As the company states,
the method is “quick, easy, helps to
ensure the accuracy of (customer)
orders and prevents others from
knowing” about whatever unusual
food craving you may be having at
that moment (Sheetz, n.d.).
The speed and ease of structured
communication applies whether
you’re a sandwich maker juggling
several orders at once or a survey
maker who needs to collect and
Figure 5 - Your lunch order can be a form of structured communication
Key Point – The What of Surveys: Structured communication creates usable data and simplifies the research process by providing a list of potential responses for each question.
Customer Opinion and Audience Response
16
process responses from many individuals. Therefore, most of your survey questions will probably be
created in a multiple-choice format. You may need to include some options for short, open responses,
but these should be used sparingly. We’ll cover question types in detail in the How section.
Content
In the 1930s, IBM hired a teacher named R.B. Johnson to create a mass-produced version of his
invention: a test-scoring machine that could sense pencil markings on a sheet of paper (IBM, n.d.).
Sound familiar? Nearly 100 years later, we’re all too familiar with the ubiquitous “#2 pencil” form that
persists to this day in classrooms and polling stations. These forms may fit the bill for the quickness and
accuracy that we look for in a good survey, but they’re pretty lousy when it comes to customer
engagement. They may have been unusual eight decades ago, but any novelty value that these forms
may have had when they were invented has long since worn away.
With online surveys, we’re no longer bound by the size of paper, the cost of postage, or the complexities
of printing. Some may argue that this allowed survey makers to swing too far in the other direction –
using the free, unlimited nature of the web to create overly-complex surveys that seemed to go on
forever. Luckily, the advent of surveys that can be distributed through mobile devices has brought us an
ideal compromise. Mobile surveys still allow for the type of multimedia content that was never possible
with paper surveys. Yet, the small screens of mobile devices – and hurried nature of mobile users –
require survey designers to bring some discipline and restraint back into the design process.
Which sounds wonderful except for one problem: it’s still a survey. And, consumers are tired of filling
out surveys. Those decades of filling in little circles with #2 pencils, followed by another decade of
constant requests for interminable online surveys, have taken their toll. Think about your own
experience – how many times have you come up with an excuse to get off the phone with a pollster?
How quickly do you delete the survey request emails that arrive in your inbox? But, even if you can’t
stand the thought of another survey, does that mean that you’ve stopped sharing your opinions
altogether? Of course not!
People still love to tell you what’s on their mind. So, why not ask them that? Instead of “Please fill out
this customer survey,” what about “Hi Jodi, we’d like to know what’s on your mind today.” That is the
actual reason you are contacting Jodi, after all, and it avoids the negative connotations that come with
“survey,” “poll,” or “questionnaire.” We recommend banishing those words from your customer
communication because they don’t refer at all to the relationship that you are trying to build. There’s a
www.pinnion.com
17
reason that the audience at a movie theater is invited to “enjoy the show” and not “view this
projection,” after all. Talk to your audience about the experience, not the tool.
This means you’ll have to live up to your promise of meaningful interaction with a survey that’s
interesting and engaging to your respondents. Consider the options that your survey provider offers in
this area. Will you have the ability to:
Include pictures in various locations
o The start or end of the survey
o Each question
o Each answer option
Display results for each question
o Online
o On iPhones or Android devices
o In the same screen as the survey (instead of a link
to another page)
Use different types of input options for each question
o Touchscreen buttons
o Checkboxes
o Picker wheels
o Sliders
o Open response/free text
Include trivia questions or other diversions
Offer points or rewards for responses
Create surveys with unlimited questions and then collect
unlimited responses
You can check the About Us page if you’d like to learn about a survey provider that offers each of these
options, but it really is important to consider these types of features no matter which provider that you
use. The more that you can do to create surveys that feel new and different from what’s been offered
before, the more innovative your organization looks and the better responses you’ll see. What you offer
is only part of the calculation, though. You’ll also need to consider Where your surveys can be
completed. Let’s take a look at that on the next page.
Figure 6 - Touchscreens allow for new types of input options, such as
sliders
Customer Opinion and Audience Response
18
Where?
In the world of surveys, this used to be a fairly straightforward question. Your respondents might have
taken your survey while they were at your business or event location. Otherwise, consumers probably
completed your survey at home and business customers provided feedback when they had a few spare
moments at the office. How quickly things have changed.
Mobile devices, aka smartphones and tablets, are the most quickly-adopted consumer technology in
history (Kang, 2011), with nearly 100 million people owning one in the US as of December 2011
(comScore, Inc., 2011). About half of these smartphone owners are checking email on almost a daily
basis and this figure rose 33 percent during 2011 alone (comScore, Inc., 2012).
The number of locations where customers could be responding to your survey has increased as a result
of the explosive increase in iPhones and Android phones. No longer confined to home or the office, your
customers are very likely to be responding wherever they may be, simply because they’re bored. In
waiting rooms, on buses and trains, and at the cafeteria, mobile devices are the new way to connect
with friends, families, and brands whenever there’s a dull moment…no matter how short that moment
may be.
If you plan events, such as conferences, trade shows, festivals, or even a company picnic, then the
proliferation of mobile devices gives you a great opportunity to request feedback from your participants
while they are on-site and ready to provide their opinions about the event. You probably know how hard
it is to get people to provide feedback after they’ve left the event, and paper forms handed out at the
event are equally problematic, so mobile really has the potential to change the dynamic in this industry.
The only catch is that you’ll still need to capture feedback from your participants at other times of the
year, as we’ll explain in the When section.
All of this talk about mobile devices raises another point – Where is no longer just a physical or
geographic concept, but also digital and virtual one. When customers take your survey on a mobile
device, is it because they received the invitation:
In an email?
Via their Facebook or Twitter app?
From a survey app notification?
For those who respond via their computers, the ways in which they could have discovered your survey
are almost too numerous to mention:
www.pinnion.com
19
Company website
o Embedded survey
o Link to survey
o Pop-up survey
e-Commerce or sales site
o Embedded survey on confirmation page
o Link to survey on confirmation page
o Pop-up survey after sale is complete
Blog
Twitter link
o Link to survey
o Embedded into a Facebook app
Email link
Just as we encouraged you to consider the abilities of your survey provider to offer customer
engagement capabilities, we suggest you take a close look at the ways in which surveys can be
distributed through your provider or platform. If you are using a web link to distribute your surveys via
email, Twitter, Facebook, or anywhere else, be sure to test the link on your own smartphone to ensure
that it’s easy to view and complete even on a small screen. After all, if your customer attempts to take
the survey on their smartphone and is frustrated by the experience, what is the chance that he or she
will remember to come back and try again from the home computer? Chances are, the opinions from
that customer are gone for good.
Figure 7 - Pinnion has been designed to help you reach your customers wherever they are
Key Point – The Where of Surveys: Even if you are communicating with customers via email, those emails are being read more-and-more on mobile devices. iPhone surveys and Android surveys will become commonplace as smartphones and tablets begin to replace the PC in many households.
Customer Opinion and Audience Response
20
When?
It probably won’t surprise you to find out that two guys who work at
a survey company think that you should do surveys all the time. Not
literally, of course, as no customer wants to provide you with
feedback every day or even every week. The trick is to let them
know that you are always willing to listen, then give them the
opportunity to share their opinions at a place and time that is
convenient for them. Mobile devices and website embeds make it
possible to offer your customers a feedback mechanism that is quite
visible when it’s needed, yet subtle enough to avoid the annoyance
and spam concerns that come with email surveys.
If you were hoping for a more specific answer than “always,” don’t
worry. A 2004 study actually found that the best time to send an
online survey is Wednesday morning, when it was observed that
half again as many individuals responded than at any other time of
the week (Faught, Green, & Whitten, 2004). On the other hand, a
2001 study found that the specific time of the week may not matter
as much as the amount of time that has passed since the customer’s
last transaction. That’s because customer perceptions vary
throughout the year and are generally higher right after a service
has been used than they are a month later (O'Neill & Palmer, 2001).
Here are some ideas for
scheduling your surveys
over a period of time:
Could you do a monthly
survey series, in which
each survey is tied to an
upcoming holiday or sales
event?
If you are able to
incorporate trivia or other
games into your survey,
then how about a daily
survey series that your
customers can play and
respond to during their
commute?
Or, you might decide to
run one survey for mobile
customers in long
months, another for
website visitors during
short months, and a
Facebook survey that is
segmented by age group
in advance of a major
product launch.
Key Point – The When of Surveys: O’Neill & Palmer found that year-around surveying increases reliability when measuring customer opinion, because doing so will smooth out variable perceptions over time.
www.pinnion.com
21
You have almost limitless opportunities to make your surveys engaging and interactive throughout the
year. Keep that in mind, review your marketing and customer engagement goals, and then consider how
surveys, polls, quizzes, and games can be used on a constant basis to stay connected with your
customers.
Customer Opinion and Audience Response
22
How?
Can you believe it? You’re almost there! Now that we’ve gone over all of the “Ws” for your survey, we
just have the big H left: How? This is where we tie everything together and get your first survey
prepared. Don’t worry: it won’t be your last!
Way back in the Why section, when we considered the reasons for doing a survey in the first place, you
identified a “past experience or upcoming situation” that you wanted to ask your customers about. You
probably have a hunch about the type of feedback that you will receive, so you’re doing the survey in
order to investigate whether your hunch is accurate. This is called “hunch verification,” which
occasionally gets a bad rap. But, when you think about it, this is a pretty close parallel to the scientific
process. Replace “hunch” with “hypothesis” and there you go.
You’ll need specific questions in order to test the hypothesis. How many questions? There is no hard-
and-fast rule, so the best advice is simply to make sure that every question is serving a purpose by
providing actionable data or by increasing engagement. Come up with as many questions as you can
think of or as many as you think you will need, then begin editing them down until you’ve created a
survey full of questions that you know you will need. For example, you may think it would be interesting
to ask each respondent for his or her age range. But, if the decision makers aren’t going to segment the
results by age when they consider the data, then there’s no need to collect that information this time
around.
That said, if you’re having trouble coming up with a list of potential questions, here are some ideas that
may help with your brainstorming:
Ask some questions of your employees, such as:
o “What have you been hearing from our customers lately?”
o “Have you heard any comments from the new customers?”
Find a situation where you can physically observe your customers
Review the company mission statement
o Or, for your own company, write down the biggest, bedrock assumptions on which you
built the business
Are there any financial trends you’d like to understand better?
In a service business, how has the tip money been? Are tips up or down?
If all else fails, start with broad questions
o The question “would you recommend us to your friends?” by itself is extremely
important.
www.pinnion.com
23
You’ll probably come up with plenty of questions once you’ve taken a few of the steps listed above.
Answer options Once you’ve got your questions narrowed down, you’ll want to create answer options for the multiple
choice questions and determine the order in which the questions will be asked. Let’s review some
examples of the various structures that are available for survey questions:
Dichotomous
If this were a quiz, then the answer would be Yes.
“Dichotomous” simply refers to a choice between two
options, such as Yes/No or True/False. These questions are
quick and easy to answer, but their black-or-white nature
doesn’t always provide the level of detail that you may
want or need.
Multiple choice
A multiple choice question presents the respondent with
several answer options. There will be some times when you
want the respondent to choose only one answer option and
others when you will want to allow them to choose multiple
answers together. Your survey software should allow this,
but you may need to include a note such as “Please select
all that apply” or “Please select two options.”
Customer Opinion and Audience Response
24
Rating scales – Likert and Semantic Differential
The Likert scale asks the respondent to indicate the degree
to which they agree or disagree with a statement. You can
have five or seven options and the scale can be “fully
anchored” or “end anchored.” The example here is fully
anchored, in that each number on the scale has a text
description to go along with it (Options 2 and 4 are hidden).
An end anchored scale would only have text for the first
and last options, leaving the weight of the middle numbers
a little more open to interpretation by the respondent.
The same rules for Likert scales apply to Semantic
Differential scales. The difference is that semantic
differential scales don’t ask the respondent to agree or
disagree, they ask for a choice to be made amongst
contrasting options (Options 2 and 3 are hidden).
www.pinnion.com
25
Image select
A multiple choice or semantic differential question can be
set-up as an image select question, where the respondent is
asked to choose from the pictures presented. This can be
more fun than reading and choosing words, while also
conveying information more clearly in some situations.
Keep in mind that images used in a semantic differential
question need to show a clear difference between the
positive and negative options. Be sure to pre-test your
survey so that others can tell you if the pictures are as clear
as you think they are.
Demographic
A demographic question is any question in which you
request personal information about the respondent, such
as age, gender, home address, zip code. Demographic
questions should normally be asked towards the end of
your survey so that your respondent has had time to get
comfortable with the questions that you are asking before
having to answer anything of a personal nature (McDaniel
& Gates, 2008, p. 307). An exception is when demographics
are used to eliminate ineligible respondents. For instance,
if the survey is intended only for adults 25 or older, an age
range question may need to be presented at the very
beginning of the survey.
Customer Opinion and Audience Response
26
Skip logic and branching
As we mentioned in the Who
section, it’s important to
automate your survey so that
each individual only sees
questions that are relevant to
him or her. This can be achieved
by using skip logic to determine
which questions should be shown
to which respondents.
In this example, which is based
on the Multiple Choice question
we showed you on Page 23, it only makes sense to ask
about specific communities on the island of Maui if the
respondent has indicated a desire to travel to that location.
You can see the logic in the screen shot above. Those
individuals whose answers match that logic will see the
question shown to the left.
www.pinnion.com
27
Text piping
The last question that we looked
at included four possible
locations on the island of Maui,
plus one option for those who
aren’t sure where they would
want to stay and one option for
those who plan to stay in another
town. For the next question, we
are going to use skip logic again
and introduce a new concept:
text piping.
Text piping takes an earlier answer and makes it part of the
question. In this screenshot, we use the placeholder
[PIPETEXT] to hold a spot for the answer text. You can see
the result below: the words “I don’t know” have been
carried over from the previous question. “Other” would
appear if that answer had been selected instead. And, the
question would be skipped entirely if neither of these
options were chosen.
You can combine Skip Logic and Text Piping to make surveys
that are very efficient and highly personalized. This helps to
ensure that each question is relevant to each respondent. It
also makes it easier for your customers to provide accurate
data, since there is less room for errors and
misunderstandings.
Note: Please see the Bibliography entries for Flagg, Shapiro, and Walonick to learn more about survey
question formats. Their information was helpful as we developed this section of the book.
Customer Opinion and Audience Response
28
Distribution options Your survey has been written, tested, and refined. You’re ready to present it to your customers and start
collecting their opinions. Uh-oh. How are you going to reach your customers so that they know you are
seeking their input?
If you already communicate with your members or customers online, such as via email, then that same
method may be a good option for sending your survey. However, many brick-and-mortar businesses
may not have any contact information for their customers. Even if you have some customer data, such
as addresses and phone numbers, it’s possible that you won’t have the information you need to
distribute an online survey as quickly as you had hoped. When Barry worked in the electric utility
industry, he heard from utilities who struggled to collect customer opinions online because they had no
way to reach their customers aside from snail mail or the phone.
The good news is that social media is helping to overcome this issue. Even those customers who refused
to share their email address may be following your organization on Facebook or Twitter. And,
announcing your survey through these channels may help you to find new customers or to gain insights
from customers who have not previously shared their opinions with you. Your website can be another
source of opinions, so your survey should be prominently embedded on or linked from your web site.
What’s that? Your organization doesn’t have a website? Or, it’s not on Facebook or Twitter? Then put
this book down and go fix those problems immediately. Customers today expect to find you in online
and in social media as much as a customer two decades ago would have expected to find you in the
Yellow Pages. These outlets are not optional anymore.
You’ll want to consider how your survey provider works on mobile devices
and whether it’s possible to take advantage of the notification system that is
built into many types of smartphones. These notifications can alert the
owner of the phone to all sorts of new items, such as text messages, emails,
sports scores, and weather forecasts. Why not alert them to the fact that
you need their opinion, too? Alerts come in many forms, from silent to loud
and from subtle to intrusive. Avoid annoying your customers by ensuring
that your survey app allows for each individual to adjust notifications to her
or his liking.
Figure 8 - This iPhone notification badge shows that two new Pinnion surveys are available
www.pinnion.com
29
The most common form of distributing an online survey is by providing a simple Web Link. This is an
easy option because the link can easily be pasted into all sorts of places: email, Facebook, Twitter,
newsletters, your web site, blog, press releases, etc. The list goes on and on. The good news is that a
Web Link is a simple way to quickly distribute your survey in as many ways as possible for maximum
distribution. On the downside, it may be difficult or impossible to know much about the individuals who
responded to your survey. And, a web link does very little to build a relationship with respondents –
click, answer, done, forgotten.
That’s why most survey providers offer you at least one other way of getting your survey out into the
world. SurveyMonkey, for example, provides an internal email service. You upload your contact list into
their system and it automatically sends a personalized email to each individual on the list. Instead of a
generic web link, each person’s email contains a unique, trackable link so that you will know who
responded, who didn’t, and when each response arrived.
At Pinnion, we have a concept called Channels for distributing surveys. Your customers can subscribe to
your channel(s) – you might have a generic channel for your organization or several channels designed
to reach different segments – and then be notified whenever a new survey is available in that channel. A
channel can also be
discovered by new individuals
with an interest in the
product or services offered
by your business.
Another downside of web
links is that they take your
respondent away from
whatever it was she was
doing before she decided to
take your survey. If Christine
the consumer is working her
way through a purchase on
your site, the last thing that
you want to do is have her
Figure 9 – Web page embeds are eye-catching and allow the survey to be answered directly, without having to navigate to another website
Customer Opinion and Audience Response
30
click out of that process for any reason. Research has shown that an individual who is interrupted while
proceeding through the flow of a webpage is less likely to complete the purchase or even to return to
the site (Hausman & Siekpe, 2009). You’ve probably experienced this firsthand while surfing the web.
Therefore, try to display your survey in an embed, widget, popup (the friendly kind, not the annoying
kind), or app that will keep your respondents on the same page while they provide their opinions.
Christine’s focus may shift to the survey while she completes it – in fact, we hope that she gives the
survey her complete attention – but she’ll be right back where she started as soon as the process is
complete. What’s more, the user can actually see the introduction to the survey and/or the first
question, which is more likely than a link to draw the user’s eye to your content.
Here are some distribution options to consider when researching survey providers:
Generic web link
o Non-personalized
o Can be pasted into any email or program
o Can be forwarded to other individuals.
Personalized web link
o Includes a tracking code that links the survey response back to the recipient.
o Cannot be forwarded to other individuals.
Mobile web link
o The link itself may not be any different from the ones above, but it should lead to a
“mobile-friendly” interface on small screens.
o Make sure that your survey provider automatically adjusts for proper screen size.
Embed/popup code or widget
o Uses iFrames or Javascript to actually place your survey content into a webpage
Facebook app
o Allows your survey content to appear on your organization’s Facebook page and in your
followers’ News Feeds.
Mobile app
o Displays your surveys within an iPhone or Android app (you may also consider versions
for Windows Phone and/or BlackBerry devices).
o Ideal choice when your organization has multiple surveys open at the same time.
o Apps can notify the individual whenever new surveys are available (although the
individual may choose to turn the notifications off).
Survey site
o Pinnion Channel Pages are dedicated to gathering user opinions.
o These sites pull new individuals to your survey (and your organization).
Reverses the traditional model of pushing surveys only to those individuals who
are already familiar with your organization
www.pinnion.com
31
Conclusion
Let’s take another look at the key points we tried to convey in each section of this book:
Why: Surveys are now a form of content that can provide value back to your customers
Who: Customers and colleagues need well-written surveys for quality feedback and decisions
What: Structured communication creates usable data
Where: All online surveys are now mobile surveys, too.
When: Collecting feedback throughout the year will increase the reliability of your data
How: Combine the right answer options with proper distribution to make a great survey
Most of all, keep it simple. We said at the beginning of the book that surveys should be a snapshot –
they don’t need to cost thousands of dollars and they don’t need to take hours to create or complete.
Keep your goals in mind, remember to always ask “Why” first, and then have fun creating surveys that
inform you while entertaining your customers.
We look forward to hearing about your experiences. Please share your questions and feedback with us:
Pinnion page at Facebook
Bill’s Twitter account
Contact Us page at pinnion.com
Customer Opinion and Audience Response
32
About Us
Bill Leath is the president of Pinnion, a Seattle-based startup with a platform that anyone can use in
order to create multiple-choice questionnaires such as surveys, trivia games, polls, and quizzes. A
lifelong entrepreneur, Bill holds a M.S. in Mechanical Engineering from the University of Washington
and currently resides in Seattle.
Learn more about Bill
Barry Fuchs is a marketing contractor with Pinnion and has authored countless membership surveys
during his career in the association industry. Barry holds a M.S. in Integrated Marketing Communications
from West Virginia University and currently resides in Seattle.
Learn more about Barry
www.pinnion.com
33
Bibliography
comScore, Inc. (2011, December 29). comScore Reports November 2011 U.S. Mobile Subscriber Market
Share. Retrieved January 20, 2012, from Press Release:
http://www.comscore.com/Press_Events/Press_Releases/2011/12/comScore_Reports_Novemb
er_2011_U.S._Mobile_Subscriber_Market_Share
comScore, Inc. (2012, January 9). U.S. Mobile Email Audience Grows by Nearly 20 Million Users in the
Past Year. Retrieved January 20, 2011, from comScore Data Mine:
http://www.comscoredatamine.com/2012/01/u-s-mobile-email-audience-grows-by-nearly-20-
million-users-in-the-past-year/
Faught, K. S., Green, K. W., & Whitten, D. (2004, Spring). DOING SURVEY RESEARCH ON THE INTERNET:
YES, TIMING DOES MATTER. Retrieved January 20, 2012, from The Journal of Computer
Information Systems: ABI/INFORM Global
Flagg, R. (2011, September 28). Online Surveys. Retrieved January 18, 2012, from HowTo.Gov:
http://www.howto.gov/customer-service/collecting-feedback/online-surveys-fact-sheet#test-
survey
Hausman, A. V., & Siekpe, J. S. (2009). The effect of web interface features on consumer online purchase
intentions. Retrieved January 30, 2012, from Journal of Business Research, 62: ScienceDirect
database
IBM. (n.d.). IBM 805 Test Scoring Machine. Retrieved 20 2012, January, from IBM Archives: http://www-
03.ibm.com/ibm/history/exhibits/specialprod1/specialprod1_9.html
Kang, C. (2011, July 11). As smartphones proliferate, some users are cutting the computer cord.
Retrieved September 19, 2011, from The Washington Post:
http://www.washingtonpost.com/business/economy/a-smartphones-proliferate-some-users-
are-cutting-the-computer-cord/2011/07/11/gIQA6ASi9H_story.html?hpid=z3
Kohut, A. (2009, October 14). But What Do the Polls Show? Retrieved January 18, 2012, from
PewResearchCenter Publications: http://pewresearch.org/pubs/1379/polling-history-influence-
policymaking-politics
O'Neill, M., & Palmer, A. (2001). Survey timing and consumer perceptions of service quality: An overview
of empirical evidence. Retrieved January 20, 2012, from Managing Service Quality; 11,3:
ABI/INFORM Global
Peltz, J. (2012, January 8). For some consumers, surveys breed feedback fatigue. Retrieved January 16,
2012, from AP News: The Orange County Register:
http://hosted.ap.org/dynamic/stories/U/US_FEEDBACK_FATIGUE?SITE=CAANR&SECTION=HOM
E&TEMPLATE=DEFAULT
Customer Opinion and Audience Response
34
Shapiro, G. M. (2008). Sample Size. Retrieved January 17, 2012, from Encyclopedia of Survey Research
Methods: SAGE Reference Online
Sheetz. (n.d.). Our Menu. Retrieved January 19, 2011, from Sheetz.com:
http://www.sheetz.com/main/food/menu.cfm
Walonick, D. S. (2004). Survival Statistics. Retrieved May 2, 2010, from StatPac, Inc.:
http://www.statpac.com/surveys/surveys.pdf