Would you answer these questions?

Would you answer these questions?

Susan FredeVP, Research Methods & Best PracticesMarch 2016

Copyright © 2016 Lightspeed, LLC. Proprietary and Confidential. All rights reserved.

Poor quality survey design contributes to…• Low completion rates• High dropout rates • Speeding • Straight-lining and other suspicious behavior• Panel attrition • Higher sample costs

Potentially leading to bad data, unrepresentive samples, and ultimately incorrect business decisions


Mobile may finally force better survey design


Let’s look at blinded screen shots from real-world surveys

Think like a respondent…• Would you answer/be able to answer each one?• What would be your reaction to each one?• What is bad about each one?• How would you improve each one?


• Unnecessary scene setting/introduction

• Most respondents are not going to read all of this


Company XYZ


• Need clear instructions – format for date

• Some respondents are hesitant to give exact birth date for privacy/security reasons

• Better to ask age or age range question


Company XYZ


• Use appropriate terminology for specific country

• Current account is a common UK term for checking account

• It is important on global projects to get regional input


• Need to clearly define terminology

• What is meant by charitable donations? Money? Used clothing/household items? Time?

• Five years is a long time period for respondents to recall


• Poor layout

• Check boxes need to be much closer to the answer text

• Could also shade every other row

• Or when a row is selected it is shaded


Company XYZ


• Missing a ‘none’ option

• Could not proceed without choosing one of the four answers


Company XYZ


• Extremely long list of almost 100 TV programs

• Best bet is to cut the list down or randomly select a subset

• Show in multiple columns; however, old research has shown mixed results with multiple columns


Company XYZ


• Another long list with nearly 100 items

• Have to respond with something for every item so this is extremely time consuming for the respondent

• Some would split this into multiple questions; however, this just causes respondents to complain about repetition and increases dropout rates with each additional grid

• Again, best bet is to cut the list down or randomly select a subset of movies

• Might also try a more interactive question style

• Watch wasted space…large gray box and company logo


Company XYZ


• Factual questions should be in alphabetical order so respondents can easily find their response


Company XYZ


• Horizontal scrolling is a huge no-no…there are five more columns when scrolling right

• Unequal column widths can cause bias in the answers

• Would be better if first ask what social networks respondent has used in last seven days

• Then specifically ask what activities the respondent has done on each site

• This will likely allow social networks to run across the top with no scrolling and activities down the side

• Could also use a more interactive question style


• Column headings should always be visible

• Scrolling up and down leads to errors


Company XYZ


• This is a large grid…could never read this on a mobile phone

• Two key improvements: 1. change to five point scale 2. use a more interactive question style such as

dynamic grids

• Other issue is topic relevancy…can someone who has only heard of the brand really answer these questions?


www.lightspeedgmi.com [email protected]

CONTACT US

http://www.lightspeedgmi.com/

mailto:[email protected]

https://twitter.com/LightspeedGMI

https://www.facebook.com/LightspeedGMI/?ref=aymt_homepage_panel

https://www.linkedin.com/company/lightspeed-gmi?trk=company_logo

Would you answer these questions?

Mobile

Transcript of Would you answer these questions?