Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test:...
Transcript of Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test:...
![Page 1: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/1.jpg)
1RTI International is a trade name of Research Triangle Institute
Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel
Craig A. Hill, PhD
BLS Data Capture Technology ForumMarch 2010
![Page 2: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/2.jpg)
2
![Page 3: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/3.jpg)
3
Full-probability panel with Smartphones
2007: The problem
A potential solution
How will we know if it works?
![Page 4: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/4.jpg)
4
A potential solution
![Page 5: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/5.jpg)
5
![Page 6: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/6.jpg)
6
A potential solution
Feasibility test
Internal (R&D) funding
Full area-prob sample
Blackjack devices
6 interviewers recruited/trained
92 panellists enrolled
Data collection: Jun – Nov 07
![Page 7: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/7.jpg)
7
Data Collection
19 weekly waves Jul – Nov Email w/ hotlink, plus text reminder
3 SMS surveys
National/local issues; health; leisure activities; device usability
![Page 8: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/8.jpg)
8
Evaluation
• Viability of the platform for different question types, formats, and lengths
• Develop and test interviewer training program for recruitment and support
• Develop and test panel recruitment strategies, and incentive packages/related materials• contact (screening) rate• acceptance (recruitment) rate• response rate (over time)• attrition (retention) rate
![Page 9: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/9.jpg)
9
Experiments
Screen “real estate” SMS (“texting”)
Pix
![Page 10: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/10.jpg)
10
Evaluation: Real Estate
![Page 11: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/11.jpg)
11
Evaluation: Real Estate
![Page 12: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/12.jpg)
12
Evaluation: Real Estate
![Page 13: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/13.jpg)
13
Evaluation: Real Estate
![Page 14: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/14.jpg)
14
Evaluation: Real Estate
![Page 15: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/15.jpg)
15
Evaluation: Real Estate
![Page 16: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/16.jpg)
16
Evaluation: Real Estate
![Page 17: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/17.jpg)
17
Published Results
Experiments in Mobile Web Survey Design: Similarities to Other Modes and Unique
Considerations
Andy Peytchev and Craig A. Hill
Social Science Computer Review 000(00) 1-17-2009
http://sscr.sagepub.com
Social Science Computer Review OnlineFirst, published on December 8, 2009 as doi:10.1177/0894439309353037
![Page 18: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/18.jpg)
18
Experiments
Screen “real estate”
SMS (“texting”) Pix
![Page 19: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/19.jpg)
19
Evaluation: SMS/Texting
![Page 20: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/20.jpg)
20
Experiments
Screen “real estate”
SMS (“texting”)
Pix
![Page 21: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/21.jpg)
21
Evaluation: Pictures
![Page 22: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/22.jpg)
22
Evaluation: Pictures
![Page 23: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/23.jpg)
23
Evaluation: Pictures
![Page 24: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/24.jpg)
24
Evaluation
• Viability of the platform for different question types, formats, and lengths
• Develop and test interviewer training program for recruitment and support
• Develop and test panel recruitment strategies, and incentive packages/related materials• contact (screening) rate• acceptance (recruitment) rate• response rate (over time)• attrition (retention) rate
![Page 25: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/25.jpg)
25
Interviewer Training
Sales, not interviewing
In-home training: 2-3x as long
Email set-up difficulty
Provide device to FRs in advance
![Page 26: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/26.jpg)
26
Evaluation
• Viability of the platform for different question types, formats, and lengths
• Develop and test interviewer training program for recruitment and support
• Develop and test panel recruitment strategies, and incentive packages/related materials• contact (screening) rate• acceptance (recruitment) rate• response rate (over time)• attrition (retention) rate
![Page 27: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/27.jpg)
27
![Page 28: Results of a Pilot Test: Smartphone-Enabled, Full-Probability ......Results of a Pilot Test: Smartphone-Enabled, Full-Probability Panel Author Hill, C. Subject Results of a Pilot Test:](https://reader034.fdocuments.us/reader034/viewer/2022042710/5f94e022c47cf4006e05f5e8/html5/thumbnails/28.jpg)
28
Thank you!
For more information:
Craig A. Hill, RTI International
Or, via smartphone