Post on 19-Jul-2020
. 10/29/14 1Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Smartphones and Smarter Surveys:
Trent D. Buskirk, Ph.D.
PANJAAPOR October 29, 2014
Practical Considerations
for Optimizing Mobile
Surveys
. 10/29/14 2Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Instructor: Trent D. Buskirk, Ph.D.
Current Position: Vice President of Statistics
and Methodology @ Marketing Systems Group
Lives in: Saint Louis, MO
Focus on:
Use of new technologies for data collection
Mobile Phone Sampling
Dual Frame Designs and Weighting
Mobile Analytics and Paradata Collection and Analysis
Addicted to:
Playing Prince for my little Princesses
APPS and iPhones
SAS, SUDAAN and R
Tennis
Boost 101.9 (www.boost1019.com)
http://www.m-s-g.com/Web/Index.aspx
. 10/29/14 3Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
PANJAAPOR Committee for their invitation!
Colleagues at MSG for their assistance and support
Colleagues at Maritz Research for collaborations on an
advanced mobile survey mode comparison study
3
Acknowledgements and Thanks!
. 10/29/14 4Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
A market research consultant goes to lunch with his
clients –
one is in the Marketing department,
the other is in Operations.
The three of them get in the car together.
The Marketing manager immediately has his foot on the gas,
the Ops guy has his foot on the brake, and the Survey guy is
looking out the back window telling them where to go.
Warm-Up
. 10/29/14 5Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Agenda & Goals
Framework for Data Collection on Mobile DevicesAn overview of the new mobile survey landscape
Developing Emerging Best Practices: A blend of old and new
Types of Mobile Surveys
Recommendations for Internet/App based survey solutions
Evaluating Mobile Surveys – User Perspective
Goal: Leave with a solid appreciation of:Some practical “best practices” for deploying mobile surveys
Issues to consider for optimizing data collection efforts via mobile devices
. 10/29/14 6Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Focus on Smartphones
The Picture
Summary!
. 10/29/14 7Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Remembering a simpler time…And a simpler phone…
Cartoon Sourcehttp://voices.washingtonpost.com/comic-riffs/2009/06/top_picks_what_this_riffster_i.html
. 10/29/14 8Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
And now introducing FUTURE PHONES!
Cartoon Sourcehttp://voices.washingtonpost.com/comic-riffs/2009/06/top_picks_what_this_riffster_i.html
. 10/29/14 9Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Section 1:
Developing Best
Practices in the
New Landscape…
6:59
. 10/29/14 10Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Mobile Devices Covering the World
Sources: http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536
http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15
4.55 Billion People Worldwide Using
Mobile Phones
1.75 Billion People Worldwide Using
SmartPhones
400 Million People Worldwide Own
Tablet Devices
788 million mobile-only internet users
by 2015 worldwide (Cisco, 2014)
. 10/29/14 11Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Mobile Devices Covering the USA
240 Million Mobile Users (age 13+) in the
U.S.
91% of All U.S. Adults have a Cell Phone
156 Million Smartphone Owners in the
U.S. (age 13+)
52% of U.S. Adults have a Smartphone
82 Million Tablet Owners in the U.S.
42% of U.S. Adults have a Tablet
Sources: comScore (2014) and Pew Research (2014) http://www.comscore.com/Insights/Presentations_and_Whitepapers/2014/2014_US_Digital_Future_in_Focus?elqCampaignId=252/
http://www.pewinternet.org/fact-sheets/mobile-technology-fact-sheet/
. 10/29/14 12Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Developing Mobile Surveys: Interface & Flow
User interface: usually an important consideration in data collection,
with new technologies it is essential
New & extended human-computer interaction concepts:
Visual appeal
Intuitive flow (more than simply “logical”)
Ease of use /functions are readily apparent
Few, if any, text instructions
Optimize mode capabilities and reduce user anxiety and frustration
Study of why people participate in mobile surveys and mode choice:
Perceived enjoyment; attitudes toward participation; self-expressiveness, & trust
factors most important (Bosnjak et al 2010)
Previous experience with alternate survey modes (other than computer) increases
preferences for mobile surveys (Buskirk, et al., 2013)
Blend current understanding of CAI systems with expanding literature
on human-computer interactions (use & interaction)
. 10/29/14 13Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
The new paradigm: MOBILE Surveys
Smartphone Survey
Design Research
Social Science
ResearchBuskirk and Andrus (2012)
Link and Buskirk (2012)
Kinesis White Paper (2010)
Buskirk et al. (2011)
Callegaro (2010)
Callegaro and Macer (2011)
Peytchev and Hill (2010)
Zahariev et al. (2009)
Smith (2011)
Bosnjak, Metzger, Graf (2010)
Lai et al. (2010)
Couper (2010)
Millar and Dillman (2012)
Human Computer
Interaction
Parush and Yuviler-Gavish
(2004)
Jones et al. (1999)
Sweeney and Crestani
(2006)
Fitts (1954)
Goldberg, Fariden and
Alterovitz (2007)
Zichermann and
Cunningham (2011)
Computer Science
and Web
Development
Koch, P-P. (2010)
Campaign Monitor (2012)
Kriss (2013)
Medallia (2013)
EmailVision (2013)
Html5rocks.com (2013)
Pelletier (2013)
Return Path (2013)
. 10/29/14 14Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Respondent Burden in the Mobile World
Respondent burden in mobile world is similar to respondent
burden in the regular world with a few differences
Two Groups: Personal and Technological
Personal: Safety
Respondents aren’t just responding at home anymore!
Steeh, Buskirk and Callegaro (2007) report respondents completing cell
phone surveys while boarding a helicopter.
Personal: Privacy
New data types like photo/voice may mean information is collected from
more than the consenting respondent
Age of assent on internet is younger than consent via phone (13 vs 18/19)
Personal: Time
To complete survey tasks [ACTIVE]
To load survey pages [PASSIVE]
. 10/29/14 15Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Respondent Burden in the New Mobile World, Cont.
Technological
Data consumption
Both receiving and sending information (survey questions/content and
submitting responses)
Nielsen (2012, Q3/Q4) Estimates that 96% of mobile phone users have
a data plan and that the average monthly bill is $66 (all mobile) and $93
for smartphone users.
Bandwidth
Slower bandwidth increases time required to send and receive data
Higher data consumption may lead to throttling by the provider –
meaning reduced bandwidth in future
Battery Drain
Persistent GPS capture drains mobile device batteries
Extra HTTP requsts can drain mobile device power
. 10/29/14 16Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Respondent Expectations
Being involved in a scientific survey is a relatively rare occurrence
Little, if any, expectations by respondents
Respondent experience with new technologies is much different
Respondents are more savvy
Technology changes rapidly – some respondents will find some tasks difficult
or not easy to use.
[Example: Michaud, Buskirk and Saunders, 2014 Voice Data Entry]
Developing expectations from these experiences
Ease of use/intuitive
Speed
Usefulness/utility or fun/entertainment or both
Ability to share experiences w/ others
Location awareness
Auto detection/passive collection
. 10/29/14 17Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Even my little one knows that a phone isn’t just for
talking!
“Can I WATCH
your Phone?”
. 10/29/14 18Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Section 2: Types of Mobile
Browser Surveys
16:59
. 10/29/14 19Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Various Approaches for Survey Data Collection
via a Smartphone
Use the regular phone mode
Use Short Text Messages (SMS)
Flash polls, Invitations, Synchronous Survey Interviews
App-based/administered surveys
Online surveys accessed via Smartphone Browser
Passive and Active versions
Hybrid Approach: App-Like Smartphone Browser
Surveys
Source: Buskirk and Andrus (2012) www.surveypractice.org.
. 10/29/14 20Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Online Surveys via Mobile Web Browser
Passive: Surveys intended to be completed online are
sometimes completed via mobile phone web browser (passive
mobile web browser approach).
Active: The online web-based survey is optimized for the
mobile environment. (think mobile versions of survey
websites)
Many survey software programs (e.g. SNAP, Qualtrics, etc.)
have mobile publishing options that optimize an otherwise
online survey for the mobile browser.
If you know the type of devices on which those sampled will
complete the survey, you can optimize the survey design for
the particular browser/screen resolution (e.g. Safari / iPhone).
. 10/29/14 21Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Passive versus Active Smartphone Surveys
Passive Mobile
Survey from Hotwire
Survey Appearance
after a “double tap”Active Mobile Survey
Both viewed on an iPhone 5
. 10/29/14 22Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
APP-LIKE Mobile Browser Surveys
APP-LIKE Mobile Browser Surveys (A-LMBS) represent a hybrid of
the APP and the Active Mobile Browser Approaches.
A-LMBS are completed via the mobile web, but they rely on active
browser refreshment using a combination of PHP Programming and
Javascript to create “Active” Mobile web pages.
A-LMBS can be developed to run without a persistent internet
connection on multiple mobile browsers (HTML5)
A-LMBS offer native, APP like functionality under the mobile browser
context.
In today’s browser vernacular, app-like surveys might fall under the
class of Web apps which are defined as web pages that function
and appear almost app like but are accessed via browsers
Source: Buskirk and Andrus (2012) www.surveypractice.org.
. 10/29/14 23Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
A-LMBS Compared to Active Mobile Web Browser
A-LMBS
Approach
(iPhone)
Active
Mobile
Browser
Approach (iPhone)
Active
Mobile
Browser
Approach (Android)
A-LMBS
Approach
(Android)
. 10/29/14 24Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
App-based Smartphone surveys administer and collect data via
an (native) app that is installed on the sampled user’s
Smartphone.
Surveys can be “pushed” to the app and then executed by the
end user without the need for perpetual internet connection
and can be designed to take full advantage of the phone’s
capabilities like camera, voice, video, image capture, etc.
The App can also serve as a survey manager and push people
to surveys that are actually optimized and executed via the web
(natively, App-like)
App-based/administered surveys
Data transfer for completed surveys can occur once internet
connection is established.
One popular example of this approach comes from
Confirmit/Techneos’ Survey On-Demand Application (SODA)http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx
. 10/29/14 25Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Section 3: Smarter
Smartphone
Survey Data
Collection and
Design Recommendations
S3DR
16:59
. 10/29/14 26Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DR #1: Mobile Survey Approach
Select an implementation
approach that:
(a) optimizes presentation of
survey content
(b) is consistent with the type of
survey data desired
(c) is appropriate for the context.
S3DR #1
. 10/29/14 27Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
How to decide the approach used here?
. 10/29/14 28Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DS #1
Email Invitations and Web Addresses for
Smartphone Surveys should be as simple and
straightforward as possible.
Wherever possible, email invitations should be
designed responsively for easy reading and
navigation on smartphone and tablets.
S3DR Tip
S3DR #2: Recruitment for Mobile Surveys via Email
S3DR = Smarter Smartphone (and Tablet) Survey Design Recommendation
Place the key information (e.g. survey sponsor, incentive…) at the
beginning of the email subject.
Place the survey link as soon as you can in the email invitation, to avoid
scrolling.
Minimize the number of “special characters” in the survey web address.
If at all possible try to avoid “aliased” web addresses (especially for
mobile web version).
. 10/29/14 29Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Email on Smartphones and Tablets
Email and the Mobile Device Reality:
52% of all cell phone owners use email on their phone (Pew, 2013)
78% of smartphone owners check email on their phone. IDC and
Facebook – “Always Connected” (2013)
10% Of iPad users say Mobile is their preferred device to read
email and 9% for sending it. –Perion “iPad Owners Survey” (2013)
Among mobile email users, If an email does not display
correctly, BlueHornet “Consumer Views of Email Marketing” (2012)
69.7% will delete it immediately;
17.7% will view it on a computer
7.6% read it anyway on their device .
Men open 20% more emails on mobile, but WOMEN click 10%
more often on mobile e-mail TailoredMail – “It’s time to wake up and mobilize!”
(2012)
. 10/29/14 30Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
With the surge in Mobile Internet Use comes
the rise of unintentional mobile respondents
http://www.maritzresearch.com/ Source: Maritz Research
. 10/29/14 31Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Email Formatting Recommendations
EmailVision provides additional recommendations
pertaining to optimizing email deliverability
For Subject lines and sender options, EmailVision
recommends:
Rather than generic emails such as info@company.com use
relevant email addresses such as:
givefeedback@company.com,
your2cents@company.edu
takesurvey@company.gov
enable the recipient to understand the intent of the email and
potentially reduces auto spam classifications.
http://www.emailvision.com/sites/default/files/email
vision-deliverability-bestpractices-2011-03.pdf
. 10/29/14 32Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Responsive Email Designs for Survey Invitations
Campaign Monitor’s E-Publication “Responsive Email
Designs” offers several considerations for developing mobile
responsive emails including:
Single column layouts (500 – 600 pixels)
Links/Buttons minimum target area of 44 X 44 pixels
Minimum font displayed on iPhones is 13 pixels – smaller font than
this will be upscaled and formatting may change
Call to action (i.e. “start survey button” or survey link) should appear
toward the upper portion of the email to avoid scrolling
For CSS programming use display:none option to limit social sharing
buttons and other extraneous details in the mobile environment.
SOURCE: Campaign Monitor Responsive Email E-Book (2013):
http://www.campaignmonitor.com/guides/mobile/
. 10/29/14 33Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Why focus on Responsive Mobile Emails?
Email Invitation on
iPhone
Survey that appears
after tapping “Yes”
. 10/29/14 34Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Mobile Optimized and Non-Optimized Emails
SOURCE: Campaign Monitor Responsive Email E-Book (2013):
http://www.campaignmonitor.com/guides/mobile/design/
Mobile Optimized Not Mobile Optimized
. 10/29/14 35Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Example of Responsive Email from Amazon.Com
Email sent to Amazon
Customers Viewed on
Desktop/Laptop (responsive)
Call to
Action
Here
Same email
viewed on
Smartphone
Source: http://marketingland.com/four-responsive-email-layouts-15858
Column Dropping
. 10/29/14 36Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
From Responsive Email to App-Like Mobile Survey…
. 10/29/14 37Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
From Email to Mobile Web…
. 10/29/14 38Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Internet Use, Time of Day and Device
Source: comScore, 2013 http://bit.ly/1c9vneo :
. 10/29/14 39Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DR #3: Device Paradata
Successful implementation of multimode surveys requires
that the survey software include comprehensive device detection
functionality. Device detection is needed to automatically identify
the type and brand of each respondent’s device, and then display
the survey according to that device’s specifications.
-Cazes, et al. Kinesis Mobile Landscape Report, 2010http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf
Plan to collect paradata from mobile sample
including device type, size and browser size.
Browsers and devices matter and can be key to
an overall survey’s functionality on a
respondent’s device.
S3DR #3
. 10/29/14 40Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Here’s a look at why browsers and device types matter
Users from Android devices have some systematic differences
from those of iOS devices – can be used as a covariate in
analyses.
Ultimately, device detection via user agent strings helps to
deliver overall content to the correct type of device (phone
versus tablet versus computer) and perhaps the operating
system.
However, the way the device translates the survey
programming (HTML markup etc) is determined by:
The browser in use
The features of the browser
The features of the device (e.g. hard keyboard versus virtual, front facing camera or not)
. 10/29/14 41Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Android Owners vs iPhone Owners
Source: Estimates Recomputed from Tables Included in Pew’s
Smartphone Ownership – 2013 Update (Smith, 2013): http://bit.ly/Qnt4Rr
Demographic VariableDistribution among
iPhone Users(U.S. Adults)
Distribution amongAndroid Users (U.S. Adults)
Men 45% 51%
18-34 36% 43%
55+ 27% 20%
High School or Less 20% 32%
College Degree (or More) 54% 39%
Non-Hispanic Black 8% 19%
Non-Hispanic White 80% 70%
<30K annual HH income 15% 28%
>75K annual HH income 51% 35%
. 10/29/14 42Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Android and iDevices and Geography!
Geographic differences may also exist among the iPhone
and Android User base within the U.S.
Study conducted by Mobify in 2013
Using 20,000 partner websites with over 200 million visits through 2012
93% of mobile traffic came from either an iOS or Android mobile devices
(tablets and smartphones)
Source Mobify (2013): http://www.mobify.com/blog/ios-vs-android-in-2012/
More blue=higher percentage
of views from iOS
More Red = Higher Percentage
of views from Android
. 10/29/14 43Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Specific Example: Same Phone…Different Browser
http://user-agent-string.info/
. 10/29/14 44Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Why browsers matter for Mobile Surveys!
Android Browser
(default/native)
Galaxy Note 3
Android (KitKat
4.4.2) SmartphoneChrome Mobile
Browser (activated)
Only first three choices
visible without scrolling
(2 taps)
ALL Choices
Visible
(One Tap)
. 10/29/14 45Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Smartphone surveys should be designed to
be as short as possible, but not necessarily
abbreviated versions of longer online
surveys.S3DR #4
S3DR #4: Survey Length
Dead spots still exist for Smartphones
In both voice and data (i.e. edge, roaming)
Internet speeds vary also by area
Sprint has both 4G and 3G speeds in St Louis, for example.
Smartphone users multitask
Three good reasons to allow for users to return to the survey site
Start over?
Begin where they left off (approximately)
. 10/29/14 46Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DS #1
The overall survey layout should minimize the need
for scrolling (either horizontally or vertically) to the
extent possible.
Survey layout should also minimize the need for
pinching/zooming.
The number of questions per screen should
generally be determined in order to minimize
scrolling (implied scrolling)
Be mindful for the device and place key actions in
“good” zones
S3DR #5
S3DR #5: Survey Layout
Some devices don’t support zooming
Answer choice layout may create need for scrolling on some devices.
. 10/29/14 47Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Putting Responses within Reach, for mobile devices!
Source: http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/
Smartphone
in Portrait
Tablet in Portrait
Tablet in Landscape
. 10/29/14 48Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Horizontal Scrolling Experiment (Peytchev & Hill, 2010)
No Significant Differences Noted
More
Responses
M
o
r
e
Q
u
e
s
t
i
o
n
s
. 10/29/14 49Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Example of Survey from Frontier Airlines
with Horizontal Scrolling Prompts
. 10/29/14 50Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Consider reducing the number of response options
for grid questions or reorganizing grid questions as
single questions per screen.
Consider organizing response options vertically or
horizontally with scale labels provided per
occurrence (see Peterson, 2013).
If Multiple choice/select all that apply require a long
list of alternatives, consider converting question
into free response with as many single-text fields
as answers expected (i.e. what are three of your
current research interests – list up to three).
S3DR #6
S3DR #6: Question Formats
. 10/29/14 51Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Some Grid Examples on Mobile Devices
Example of Grids – reworked for mobile surveys
Source: Peterson et al. (2013) http://bit.ly/1i5WYo0
HARD!! Easier! Easier!
. 10/29/14 52Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Got Health Apps Study Screenshots Illustrating
Question Layout and Response Choice Feedback
Source: Buskirk and Andrus (2014)
http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref
. 10/29/14 53Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Paging Versus Scrolling Question Presentation
Scrolling question presentation in mobile surveys
presents a series of survey items on a single page
Requires respondents to scroll (vertically) in order to see (and
answer) all survey questions on the page
Paging question presentation in mobile surveys
limits the number of questions per page
Generally one question per page and the respondent
navigates multiple pages in order to complete the survey
Buskirk and Andrus (2014) present a hybrid with two
questions per page to limit (a) the vertical scrolling required
on mobile; (b) the overall number of pages needed for survey
completion and (c) increase comparability for desktop version
. 10/29/14 54Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DR #7: Scrolling Versus Paging
(a) Consider an optimal combination of these
methods for presenting questions.
(b) If you have a series of questions with similar
response options and type then scrolling may
help move a respondent along more quickly
and simply.
(c) If you have multiple skip pattern logic in your
survey, paging may be preferred (at least for
that section).
(d) Be careful that scrolling is suggested or implied
so respondents won’t skip (especially if
Next/Back are at the top)
S3DR #7
. 10/29/14 55Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
An example of Scrolling in practice
Requires
tapping here;
then entering
value; then
tapping out
then scrolling
and repeating.
. 10/29/14 56Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Example of a Survey w/ a Heavy Scrolling Section
IMPLIED Scrolling
here
Questions have
similar answer
choices and
method of entry
. 10/29/14 57Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Sliderbars, sum tallies and drag and drop sorters
may not work on all devices.
These question types should be tested extensively
using emulators prior to their adoption in your
mobile survey.
Text entry fields should be wide enough so that
users can see their entire entries without scrolling
horizontally. Use this type of entry judiciously.
Decisions about types of input/forms or input should
include consideration of number of taps or clicks
required (NTR or NCR) to enter (and register) an
answer.
S3DR #8
S3DR #8: Question Types
. 10/29/14 58Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
A quick note about sliders
Be wary of the default starting positions – could
grossly impact final results
Michaud, Buskirk and Saunders, 2014
Peterson et al., 2013
Can’t distinguish a “3” from missing if default start is midpoint
Source: Peterson et al. (2013) http://bit.ly/1i5WYo0
. 10/29/14 59Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Using the NTR metrics to help weigh design
choices, in practice.
Michaud, Buskirk and Saunders (2014) included an
experiment to compare radio button and list inputs for
various substantive questions.
Initially respondents that were randomized to lists received
this format for all survey questions including demographic
questions about gender, education, race, etc..
Pilot testing and the NTR metrics changed our plans for the
demographic questions…
. 10/29/14 60Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Using the NTR metrics to help weigh design
choices, in practice.
Radio Button
Version (all devices)
List/Dropdown iOS
and Stock Android
Selector Android
(some browsers)
NTR=1
NTR=3/4
NTR=2
. 10/29/14 61Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Beware of the Pagination/Scrolling Mashup!
Just because it’s optimized for smaller screens doesn’t mean it’s
optimized for respondents!
Presentation of questions only without their response options will save space
on a screen, but makes data collection/completion a mine field!
Example from Major League Baseball!
https://www.fansatbat.mlb.com/PORTAL/default.aspx
The survey question (depicted on an
iPhone 5):
“Are any of these networks included
in your current cable lineup?”
Tap Here to make selection
NTR=2 per item
. 10/29/14 62Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
App-Like Mobile Browser Surveys are easier to
deploy using HTML 5 functionality
An App-based approach may not be the only way to
collect modern types of mobile data – the mobile
browsers may now access many “on-board”
components of the mobile phone via HTML 5.
Familiarize yourself with the functions that are
available for the majority of operating
systems/devices for your users
Make sure you have a “back-up” plan for when the
HTML 5 functionality is not available
S3DR #9
S3DR #9: Get Familiar with HTML 5 and Use its
functionality when possible:
. 10/29/14 63Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
HTML5 and the rise of app-like functionality
The advent of HTML5 has given programmers and
researchers a lot of flexibility and versatility in creating
mobile versions of websites with many supported input
types for mobile forms and capabilities:
Geolocation collection
Media capture (voice, photo and video)
Improved and expanded input types
Web storage
For survey researchers, this new development means that
our active mobile browser surveys can be made more and
more app like with standard, predefined enhancements that
have been include in HTML5.
But this is not yet a silver bullet!! Care must be taken!
http://html5test.com/index.html
. 10/29/14 64Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Case Study Example of HTML 5 in Action
Walmart in partnership with Mattel Hot Wheels launched a loyalty
program for shoppers.
Shoppers who purchased Hot Wheels products could receive
points redeemable for various rewards by scanning their receipts
showing the Hot Wheels item purchased.
The loyalty program required NO app, yet enabled mobile photo
capture!
Scanning/photo functionality and uploads to the Walmart program were
managed via an online website optimized for mobile visitors with (modern)
browsers
. 10/29/14 65Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Hot Wheels Redemption Program on iPhone
SOURCE: Hot Wheels (2013) https://hotwheelsrewards.com/
. 10/29/14 66Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Practical Example from New Study!
Michaud, Buskirk and Saunders
(2014) designed a study in
partnership with Decipher and
Research Now to compare various
touch and data collection
possibilities across three modes –
computers, tablets and smartphones.
A photo capture question was
deployed to all devices detected as
either “Tablet” or “Smartphone”
using user agent string classification.
Kindle Fire Tablets Respondents
running the native Silk browser had
NO photo uploads as expected from
browser limitations
Those with devices without this
functionality had to press continue
twice to continue past this screen.
. 10/29/14 67Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Taking Advantage of New Input Types from HTML5
With the release of HTML 5 there are several new input
types available that might be of interest to survey
researchers and questionnaire designers including:
Number
Tel
Range (a.k.a. sliders)
Time
These new input types when used wisely will likely reduce:
The overall amount of additional scripting that is required to format
the survey questions and response options
Reduce survey page load times
The NTR metric for a given question
Source: MobiForge, 2013 http://bit.ly/1aH3vnM
. 10/29/14 68Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Numeric, open ended questions using new HTML
input options Number and Tel
How many miles per year do you drive the car that you personally drive
most often. (Enter a number between 0 and 50000)
Question PageDefault Input Using HTML5
“number”
input type
By using a text default,
the NTR increases by one
because respondents
have to select the number
window
. 10/29/14 69Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Making Numeric Input Similar across Devices
By using the HTML5 “tel” input type, a numeric
keypad similar to what is displayed when making
calls is displayed on all devices.
Formats consistent
across devices in
terms of number entry.
Note however, that if
decimals are required,
then iOS keypad does
not offer this capability
directly.
. 10/29/14 70Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DS #1
Limited screen landscape requires compromise
and judgment when using logos, progress bars,
disclaimers and help links in mobile surveys.
Consider using abbreviated progress bars
Placement of next and back buttons should be
oriented toward the top of the screen or at the
bottom of the screen as long as they are
persistent and visible without scrolling
Disclaimers should be placed on the welcome
page.
S3DR #10
S3DR #10: Navigation/Progress
. 10/29/14 71Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Illustrating Back Button and Progress Bar Placement
Next/Back Buttons
Progress Bars
. 10/29/14 72Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Another Take on Next/Back and Progress Bars
Progress Bar
(Persistent)
Next/Back
Buttons
Persistent
. 10/29/14 73Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DS #1
Response selections, action buttons and navigation
tabs in surveys should be made as large as possible
on mobile surveys.
Touch input differs from Mouse input.
Fattest Fingers – Average width of index finger is 11
mm with a range of under 7 for babies to over 19 for
tall athletes.
Make sure to include padding around important
navigation tabs (next/continue/enter) to avoid
accidental taps .
S3DR #11
S3DR #11: Buttons, Tabs and More Buttons
. 10/29/14 74Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Importance of Button Sizes
Make butons/tabs large enough to not be missed/avoid mistaps
40 pixels (7 mm by 7mm) GOOD
50 pixels (9mm by 9 mm ) BETTER
30 pixels (5 mm by 5 mm) OK
Give enough buffer around/between tabs/buttons
Padding of at least 2mm (10 pixels) between targets
Source: Microsoft Touch Guidance
. 10/29/14 75Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
One accidental tap could be missing data!
From Michaud, Buskirk and Saunders (2014)
We included slider bars as input options for scale questions. The slider bar
button was made large enough to drag across phone and tablet devices.
On smartphones such as iPhone, the “continue” tab was too close to the
upper answer choices, so people who dragged to either 7 or 8 could have
tapped the continue button by accident on the way to a rating of 9 or 10.
In the radio button version, people attempting to tap an 8 9 or 10 could have
tapped continue on the way there and recorded no data.
Before
adding
padding
of 4
pixels
After
Padding
Added
. 10/29/14 76Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
S3DR #12: Recommendation for Mobile Surveys that have
a Multilingual Audience/Various Language Versions
Remember the higher incidence of mobile users
among Hispanics and Asians in the U.S.
When deciding on spacing of answer choices,
resulting font size implications and layout for a
given set of questions, make choices in light of
the languages that will be presented.
For example, if the Spanish version of the
answer choices are actually longer, then
optimize for these choices first rather than
always optimize for English versions.
S3DR #12
. 10/29/14 77Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Optimizing for English Versions may not be sufficient!
Questions or
their answer
choices (in
English) that
appear to be well
optimized and
presented using
mobile browsers
may not be
optimally
presented in
another language
ENGLISH VERSION SPANISH VERSION
. 10/29/14 78Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Company website development method and approach
may not be the most optimal for survey data collection
Distinguish data/content consumption from data
collection using web
Carefully consider other design alternatives such as
mobile specific survey sites or more adaptive designs
S3DR#13: Choose The Survey Website Development Method Carefully:
The default website method may not work for your survey needs!
S3DR #13
Three main approaches for developing an active mobile browser survey:
Responsive Web Design (RWD) for Survey Pages
Mobile-First RWD
Adaptive Designs
Responsive Design with Server Side Components (RESS)
Separate Mobile Web site developed for Surveys
. 10/29/14 79Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
RWD and Mobile First RWDs for Surveys
Source: http://web3canvas.com/what-is-mobile-first-responsive-design/
RWD for Surveys
would be a strategy
that takes an online
(computer survey)
and thinks about how
to render it down to
tablet and then down
to smartphones.
Mobile First RWD
would take a mobile
survey and
progressively push it
to larger devices with
potentially different
functionality.
. 10/29/14 80Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
It’s Time for Vacation– Let’s Go to Disney!
Responsive Web Design Example: www.disney.com
mini
. 10/29/14 81Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
The other side of RWD…LAPTOP Pinch and Zoom!
Responsive web design can be great at adapting content to smaller
viewports (or devices) but one of the drawbacks for surveys is the
bringing back the pinch and zoom issue to lap/desktops as
illustrated here – a laptop and iphone view of a question from a
sample survey from CheckMarket. https://www.checkmarket.com/mobile-surveys/
Laptop
View
iPhone
View
. 10/29/14 82Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Adaptive Designs
There are various versions of Adaptive web designs but the
approach that has a lot of traction recently is Responsive Web
Design with Server Side Components (RESS)
RESS is sometimes referred to as a hybrid approach between Server-
side implementation (rendering unto Caesar what is Caesar’s) and
RWD.
RESS uses device detection on the server side to send device specific
survey web page content/scripting
The device specific code can include components that allow it to
progressively enhance the survey page depending on browser
features it detects on the respondent’s device.
References:
S. Rieger (2011) http://slidesha.re/1k1HLUH Pratap (2013) http://bit.ly/IK6Q8A
Ronan (2011) http://bit.ly/1tuHNcJ B. Rieger (2011) http://slidesha.re/1pnzBeA
. 10/29/14 83Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Survey Questions
Images used in Survey
Computer Scripting and
HTML Markup (all devices)
www.takesurvey.com
RESPONSIVE WEB
DEISGN – 10000
Feet Overview
. 10/29/14 84Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Survey Questions
Images for
PHONESComputer
Scripting and HTML Markup
(PHONES)
www.takesurvey.com
RESPONSIVE WEB DEISGN
with Server Side Components
10000 Feet Overview
What Kind of
Device is it? User
Agent String
Says:
iPhone
. 10/29/14 85Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Giving your Online Survey Site a Mobile Sibling
Separate mobile version of the site is the ultimate adaptive
strategy
Special scripts are created to program survey functionality specifically
for the mobile devices
The survey website will generally be a stand-alone website and could
potentially require a different web address.
If you know a priori that you are recruiting mobile respondents, then a
unique mobile link and QR code could be provided.
If you are interested in field work or in-person data collection using
mobile devices and you don’t want to use an app, then a separate
mobile web survey optimized for mobile devices could be a cost
effective and viable solution.
Frost (2012) offers a great comparison or responsive and separate
versions of presidential websites from the 2012 election.
. 10/29/14 86Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
. 10/29/14 87Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Focus on a survey design that is technically sound
On mobile websites there is a smaller margin of error for:
Image rendering, browser requests, survey page size, waiting times.
77% of cell internet users say they experience slow download speeds
that prevent things from loading as quickly as they would like. Of those
cell internet users, 46% face slow download speeds weekly or more
frequently. (Pew, 2012)
97% of time on a single web page is spent on the front end – meaning
the loading time of the page (Everts, 2011)
Why do we need to focus on techniques that can optimize
survey websites?
Overall Respondent burden
Lag/Latency Waiting time, Time on Tasks, data consumption/exchange, effort
Break-off, partial completion, full completion
Keep in mind that a web survey is a series of consecutive and linked
web pages (each serving up survey tasks to complete).
. 10/29/14 88Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
There is a great deal of functionality that is consistent
across mobile smartphones- but there is enough
inconsistency to require extensive pre-testing of your
mobile survey across multiple platforms.
Consider using smartphone emulators to pretest the
mobile version of your survey or alternatively a virtual
device laboratory to test mobile versions of your survey.
Test availability of flash content using a resource such
as: http://www.adobe.com/flashplatform/supported_devices/smartphon
es.html
(or on your mobile phone/emulator) http://www.adobe.com/software/flash/about/
S3DR#14: Pre-Testing Mobile Surveys
S3DR #14
. 10/29/14 89Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Mobile Emulator Software Example:Keynote’s DeviceAnywhere
http://www.keynotedeviceanywhere.com/da-free-product-overview.php
Many types of
devices centrally
located in one
source (host of
tablets and
smartphones)
. 10/29/14 90Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Section 4: Resources
for Mobile
Surveys
16:59
. 10/29/14 91Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Resources for Mobile Surveys
An overview of the survey research to date on mobile devices,
capabilities, privacy and legal issues and some discussion of
what’s on the horizon can be found in Chapter 1 of the: AAPOR
Emerging Technologies Task Force Report
Mobile Web Best Practice Guidelines produced by the World Wide
Web Consortium (W3C)
Gives very detailed suggestions on how to create web pages for mobile
devices, in general. Some of the information on touch interfaces, font sizes
and mobile versioning of online websites may directly pertain to survey
development and practice. http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap
Survey Practice eJournal
Generally has a short publishing time from submission to online posting and
contains many shorter articles highlighting experiments conducted with
mobile devices.
. 10/29/14 92Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Resources for Mobile Surveys, Take 2
Mobile Marketing Research Association
Has a Knowledge Center where content on mobile market research is
searchable by keywords. http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects
Mobile Glossaries
uSamp Mobile Glossary Contains detailed definitions for many things
mobile including API, GeoFencing and many other terms relevant to
mobile devices and use.http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations&
The European Society for Opinion and Market Research
(ESOMAR)
Guidance for Mobile Market Research Reporthttp://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-
guidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf
Guidance for Mobile Survey Research Reporthttp://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-
and-Guidelines_Conducting-survey-research-via-mobile-phone.pdf
. 10/29/14 93Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Resources for Mobile Surveys, Take 3
Our Mobile Planet Google Site
Contains data from several countries on mobile behaviors by
basic demographics of the user by year (from 2011 onward)
for a host of countries around the world.
http://think.withgoogle.com/mobileplanet/en/
Mobi-Thinking – Website Offered by dotMobi to
educate practitioners about the mobile arena
Special Section dedicated to Global Mobile Statistics
http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats
Flurry Analytics (Service and Blog)
Track user activity in the app with events that are defined by the app
owner. May be useful to collecting paradata for app related surveys.
Also have a blog that provides regular estimates of app usage across a
variety of platforms.
http://blog.flurry.com/ and http://www.flurry.com
. 10/29/14 94Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Final Comment!
I leave you with this final word from Sir Richard
Branson to encourage all of you to continue to:
invest in experimentation in the mobile survey arena and
share your results (and allow us to LIKE them )
. 10/29/14 95Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Thank you!
TBuskirk@m-s-g.com
314-695-1378
www.m-s-g.com
@trentbuskirk
. 10/29/14 96Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
References
AAPOR Cell Phone Task Force Report, 2010 available at: http://www.aapor.org/Cell_Phone_Task_Force_Report.htm , accessed on October 28, 2011
Attfield, Simon, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. “Towards a Science of User Engagement (Position Paper).” Presented at the WSDM Workshop on User Modelling for Web Applications, Hong Kong, China.
Bailey, Justin and Lorelle Vanno. 2011. “Market Research Association Current Mobile Research Trends.” Webinar for the Florida Chapter of Marketing Research Association.
Bosnjak, Michael, Gottfried Metzger, and Lorenz Gräf. 2010. “Understanding the Willingness to Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective, Hedonic, Social, Self-Expressive, and Trust-Related Factors,” Social Science Computer Review 28: 350-370.
Brick, J. M., P. D. Brick, S. Dipko, S. Presser, C. Tucker, and Y. Yuan. 2007. Cell phone survey feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly 71 (1): 23–39.
Buskirk, T.D., Callegaro, M. and Rao, K. (2010) ‘N the Network'? Using Internet Resources for Predicting Cell Phone Number Status. Social Science Computer Review: Special Issue on Mobile Surveys, Vol. 28, No. 3, 271-286.
Buskirk, T.D. and C. Andrus. (2012a). Smart surveys for smartphone: exploring various approaches for conducting online mobile surveys via smartphones. Survey Practice. Available at: http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/ .
Buskirk, T. D. and Callegaro, M. “Cell Phone Sampling,” in Encyclopedia of Survey Research Methods, Lavrakas, P.J. (editor), Sage, Los Angeles, © 2008, pp. 84-89.
Buskirk, T.D., Gaynor, M., Andrus, C. and Gorrell, C. (2011) “An App a Day Could Keep The Doctor Away: Comparing Mode Effects for a iPhone Survey related to Health App Use.” American Association of Public Opinion Research, Phoenix, AZ.
. 10/29/14 97Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
References (con’t)
Buskirk, T.D. and Andrus, C. (2014) “Making Mobile Surveys Smarter: Results from a randomized
experiment comparing online smartphone and desktop/laptop computers,” Field Methods,
forthcoming. See http://fmx.sagepub.com/content/early/recent
Callegaro, Mario. 2010. “Do You Know Which Device Your Respondent Has Used to Take Your Online
Survey?” Survey Practice, December: www.surveypractice.org.
Callegaro, M. and Macer, T. (2011) “Designing Surveys for Mobile Devices: Pocket-sized Surveys that
Yield Powerful Results,“ Shortcourse Presented at the 66th Annual Meeting of the Americaon
Association of Public Opinion Research, Phoenix, AZ.
Cazes, J., Townsend, L., Rios, H., & Ehler-James, J. (2010). The mobile survey landscape - Today and
Tomorrow. Impacts of mobile devices usage on current and future market research practices.
Retrieved from http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf
comScore (2012) 2012 Mobile Future in Focus. Retrieved from
http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_Fo
cus
Couper, M. P. (2010). Visual design in online surveys: Learnings for the mobile world. Presented at
the Mobile Research Conference 2010, London. Retrieved from
http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdf
Cunningham, JA et al. (2013) “Use of mobile devices to answer online surveys: implications for
research,” BMC Research Notes. Vol 6:258 available online: http://www.biomedcentral.com/1756-
0500/6/258
Duan, Shu, Justin Bailey, and Michael Link. 2011. “Data Collection Method Innovation: A
Crowdsourcing Application to Collect Factual Information in Africa”. Paper presented at the Midwest
Association for Public Opinion Research Conference, Chicago, IL.
. 10/29/14 98Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
Fuchs, Mark, 2008. “Mobile Web Surveys: A Preliminary Discussion of Methodological Implications”. In F.
Conrad & M. Schober (Eds.), Envisioning the Survey Interview of the Future (pp. 77-94). Hoboken, NJ: Wiley.
Graham, P, and C. Cobb (2013). “Comparison of Instantaneous Mobile Time Use Data Collection Methods to
Traditional Time Diary Methods.” Paper presented at the Annual Conference of the American Association for
Public Opinion Research, Boston, MA.
Gregoski, Mathew, Martina Mueller, Alexey Vertegel, et al. (2012). “Development and Validation of a
Smartphone Heart Rate Acquisition Application for Health Promotion and Wellness Telehealth Applications.”
International Journal of Telemedicine and Applications, doi:10.1155/2012/696324.
http://www.hindawi.com/journals/ijta/2012/696324/
Kennedy, C. (2007), Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys,
Public Opinion Quarterly, Vol 71 / Num 5: 750-771.
Koch, P-P. (2010) Smartphone Browser Landscape, A List Apart Article No. 320, available at
http://www.alistapart.com/articles/smartphone-browser-landscape/ (accessed on 10/30/2011)
Kumar, A. (2012) “Pilot study tests wearable sensors and smartphone-based data collection for diabetes,”
iMedicalApps, accessed at http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphone-
diabetes/
Kuntsche, Emmanuel and Benjamin Robert. 2009. “Short Message Service (SMS) Technology in Alcohol
Research – A Feasability Study.” Alcohol &Alcoholism 44: 423-428.
Lai, Jennie W., Lorelle Vanno, Michael Link, Jennie Pearson, Hala Makowska, Karen Benezra, Mark Green.
2010. “Life360: Usability of Mobile Devices for Time Use Surveys” Survey Practice, February:
www.surveypractice.org
9
8
References (con’t)
. 10/29/14 99Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
References (con’t)
Lai, Jennie, Michael Link, and Lorelle Vanno. 2011. “Like? The Effect of Social Networking in the Digital Age
of Mobile Application Research.” Paper presented at the Midwest Association for Public Opinion Research
Conference, Chicago, IL.
Maxl, E., Döring, N., & Wallisch, A. (Eds.). (2009). Mobile market research. Köln: Herbert Von Halem Verlag.
Nitsche, P, et al. (2012) “A strategy on how to utilize smartphones for automatically reconstructing trips in
travel surveys,” Procedia-Social and Behavioral Sciences, Vol. 48, pp. 1033-1046.
Olson, K, and J. Wagner (2013). “A Field Experiment Using GPS devices to measure Interviewer Travel
Behavior.” Paper presented at the annual conference of the American Association for Public Opinion
Research, Boston (May).
Paudela, D., M. Ahmedb, A. Pradhanc, R. Lal Dangold (2013) “Successful use of tablet personal computers
and wireless technologies for the 2011 Nepal Demographic and Health Survey,” Glob Health Sci Pract August
12, 2013 vol. 1 no. 2 p. 277-284; http://www.ghspjournal.org/content/1/2/277.full.pdf+html
Perkins,J, R. Granger, and E. Saleska. 2009. “Data Security Considerations When Using Social Networking
Wesites for Locating and Contacting Sample Members.” Presented at International Field Directors &
Technology Conference.
Peytchev, Andy and Craig A. Hill. 2010. “Experiments in Mobile Web Survey Design: Similarities to Other
Modes and Unique Considerations,” Social Science Computer Review 28: 319-335.
Pew Research Center. 2011. “How People Use Tablets and What It Means for the Future of News.” Report.
Available at: www.pewresearch.org/pubs/2119/tablet-news
Scagnelli, Jeff, Justin Bailey, Michael Link, Hala Moakowska, and Karen Benezra (2012). “On the Run: In the
Moment Smartphone Data Collection.” Paper presented at the annual conference of the American Association
for Public Opinion Research (Orlando, FL).
. 10/29/14 100Trent D. Buskirk, Ph.D. Smarter Smartphone Surveys, Presented at PANJAAPOR
References (con’t)
Schober, Michael, Frederick Conrad, Christopher Antoun, Alison Bowers, Andrew Hupp, Huiying Yan (2013).
“Conversational Interaction and Survey Data Quality in SMS Text Interviews.” Paper presented at the Annual
Conference of the American Association for Public Opinion Research, Boston, MA.
Smith, Aaron. 2011. “Smartphone Adoption and Usage.” Report available at:
www.pewinternet.org/reports/2011/Samartphones.aspx
Steeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field
Methods, 19, 59–75.
van Heerden, A. Shane A. Norris, Stephen M. Tollman, and Linda M. Richter, “Collecting Health Research
Data: Comparing Mobile Phone-Assisted Personal Interviewing to Paper-and-pen Data Collection,” in press,
Field Methods, http://fmx.sagepub.com/content/early/recent
Vanno, Lorelle, Jennie Lai, and Michael Link. 2011. “Want to Collect Behavioral Data in an Engaging Way?
There’s an App for That!” Paper presented at the Southern Association for Public Opinion Research
Conference, Raleigh, NC.
Zahariev, M., Ferneyhough, C., Ryan, C., & Bishop, S. (2009). Best practices in mobile research. Presented at
the ESOMAR Online Research, Chicago, IL: ESOMAR.
Zickuhr, K. (2011). Generations and their gadgets. Pew Internet & the American Life Project. Retrieved from
http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspx
Zuwallack, R. (2009) Piloting Data Collection via Cell Phones: Results, Experiences, and Lessons Learned,
Field Methods, Vol. 21, No. 4, pp. 388–406
Zichermann and Cunningham (2011) Gamification by Design: Implementing Game Mechanics in Web and
Mobile Apps available at: http://www.amazon.com/Gamification-Design-Implementing-Mechanics-
Mobile/dp/1449397670
Andbergen, P. A. (2012). Comparison of WiFi positioning on two mobile devices. Journal Of Location Based
Services, 6(1), 35-50. doi:10.1080/17489725.2011.630038