Post on 23-Dec-2015
1
Using the Web Using the Web for for
Surveys of Medical ProvidersSurveys of Medical Providers
Using the Web Using the Web for for
Surveys of Medical ProvidersSurveys of Medical Providers
Vasudha NarayananPresented at
Third International Conference on Establishment SurveysJune 21, 2007
2
AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements
Co-Authors Stephanie Fry, Jennifer Crafts
& Pamela Giambo from Westat
Project Officers David Clark, William Taylor, Gladys Valentin
& Mei Wang from Centers for Medicare & Medicaid Services
This material was prepared by Westat Inc., under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy
3
OverviewOverviewOverviewOverview
• About the Surveys
• Survey Methods
• Questionnaire Design Issues
• Sampling and Contacting Providers
• Completes by Mode
• Factors Affecting Web Response
• Conclusions
4
About MCPSS & QIO SurveyAbout MCPSS & QIO SurveyAbout MCPSS & QIO SurveyAbout MCPSS & QIO Survey
• Two Surveys for Centers for Medicare & Medicaid Services (CMS)– QIO Provider Survey (2004)– MCPSS (2006 & 2007)
• National Sample of Medicare Providers
• Satisfaction Surveys
• Approx. 15 minute survey
• Multi-Mode Data Collection
5
Survey MethodsSurvey MethodsSurvey MethodsSurvey Methods
• Web with telephone follow-up; paper on request
• Telephone and paper surveys to:– Increase response rates– Reduce potential bias
• Importance of:– Screening– High quality mail outs– Professional Interviewers
6
Survey Methods (cont.)Survey Methods (cont.)Survey Methods (cont.)Survey Methods (cont.)
• Field Period (12-16 weeks)– Screener– Notification of web survey– Reminder letters/e-mails– Hard copy questionnaire mailing/fax– Telephone non-response– Response rates (AAPOR-RR3)
• QIO: 90%• MCPSS: 65%
7
Questionnaire Design Questionnaire Design Usability TestingUsability Testing
Questionnaire Design Questionnaire Design Usability TestingUsability Testing
• Purpose of Testing– Ease of navigation– User satisfaction and perceived burden
• Profile of Testers
• Response mode bias
8
Questionnaire Design Questionnaire Design Usability Testing IssuesUsability Testing Issues
Questionnaire Design Questionnaire Design Usability Testing IssuesUsability Testing Issues
• Allow for multiple respondents – Clarity of instructions to “designate a
colleague”
• Clarity of instruction for submitting survey
• Ability to skip sections and questions
9
Elements of the Final DesignElements of the Final DesignElements of the Final DesignElements of the Final Design
• Allow multiple respondents for each survey
• Track section-level designees
• Track section-level status
• Crisp and easy to navigate web survey
• Seamless link between CATI & Web
10
11
12
Completes by ModeCompletes by ModeCompletes by ModeCompletes by Mode
*non-response mail out of hard-copy survey in 2004
• Highest Rates of Web response – Hospitals• Lowest Rates of Web response – Physicians
Web Mail/Fax* Telephone Completes
2004 32.2% 21.9% 45.9% 17,628
2006 40.1% 1.7% 58.1% 16,121
2007 55.1% 0.9% 43.9% 17,747
13
Factors Affecting Web ResponseFactors Affecting Web ResponseFactors Affecting Web ResponseFactors Affecting Web Response
• Percentage Web access– 93% physician practices (2007)– 97% other providers (2007)
Hypotheses:• Ease of Web access at work station
• Ease of survey website navigation
• Ability to obtain help when needed
14
ConclusionsConclusionsConclusionsConclusions
• Feasibility of a Web surveys no longer an issue
• Necessary factors for survey management and achieving high response rates – telephone non-response follow up essential– Frequent interface between data collection
modes– Allow the respondent maximum flexibility
while maintaining survey integrity
15
Contacting the AuthorsContacting the AuthorsContacting the AuthorsContacting the Authors
Vasudha Narayanan (vasudhanarayanan@westat.com)
Stephanie Fry (stephaniefry@westat.com)
Jennifer Crafts (jennifercrafts@westat.com)
Pamela Giambo (pamelagiambo@westat.com)
16
Thank youThank youThank youThank you