Mobile Web Survey
19
Transcript of Mobile Web Survey
Mobile Web Survey in the Real World
Marika de Bruijne
Arnaud Wijnant
ESRA conference
Ljubljana
18 July 2013
Research Question & Scope
22-8-2013
3
• What is the effect of conducting a web survey using mobile devices compared to using a computer?
• Mobile devices = smartphones and tablets
• Comparison of:
– Response rates
– Survey means
– Completion time
– Respondent experience
Sample
22-8-2013
4
CENTERPANEL (N=2722)
Smartphone & tablet users (n=766)
Via computer,
web survey ‘as
is’
Pre-measurement (n=2029)
R A N D O M A L L O C A T I O N
(N=223)
Computer version
22-8-2013
5
• Regular web layout
• Grids included
Sample
22-8-2013
6
CENTERPANEL (N=2722)
Smartphone & tablet users (n=766)
Via computer,
web survey ‘as
is’
Via mobile
devices,
adapted layout
Pre-measurement (n=2029)
R A N D O M A L L O C A T I O N
(N=223) (N=217)
Mobile version
22-8-2013
7
• Via mobile browsers
(no app)
• Programmed in C-MoTo: mobile touchscreen interface for Blaise IS
Sample
22-8-2013
8
CENTERPANEL (N=2722)
Smartphone & tablet users (n=766)
Via computer,
web survey ‘as
is’
Via mobile
devices,
adapted layout
Hybrid: via
computer,
adapted layout
Pre-measurement (n=2029)
R A N D O M A L L O C A T I O N
(N=223) (N=221) (N=217)
Hybrid version
22-8-2013
9
• No grids
• Vertical answer scales
Response
• Mobile web response lower than computer web response
• Respondents using incorrect mode excluded
• Invitation by e-mail
22-8-2013
10
Completion Mode
• Not all respondents are willing to use the device requested but use their own device of preference instead
22-8-2013
11
Survey Means
• Only few differences (on p<.05 level)
22-8-2013
12
Completion Time
• Both subjective and objective completion time were longer in the mobile condition
22-8-2013
13
(a) Scale from 1 = very short to 7 = very long (b) Excl. outliers of >3600s *** p<.001
Respondent Experience
• The respondents’ evaluation on mobile questionnaire is similar (slightly, but not significantly, more negative)
• Location mainly at home in all conditions
22-8-2013
14
Conclusions
• Survey means are comparable, but response via mobile devices is lower
• More research on different data quality indicators is needed
• Longer completion time in mobile condition: technical or user related reasons?
• Mobile web survey still needs to mature
22-8-2013
15
Follow-up Research
• How to improve response? -> Experiment with invitations by text message versus e-mail in a smartphone survey
• Optimizing questionnaire design for mobile screens and touchscreen navigation
22-8-2013
16
Future Challenges & Chances
• Scattered landscape ->reconsider and possibly renew online survey strategy
• Need for online multi-platform surveys
• Effects of the more direct & intuitive touchscreen interface on task handling
22-8-2013
17
