1

Using the Web Using the Web for for

Surveys of Medical ProvidersSurveys of Medical Providers

Using the Web Using the Web for for

Surveys of Medical ProvidersSurveys of Medical Providers

Vasudha NarayananPresented at

Third International Conference on Establishment SurveysJune 21, 2007

2

AcknowledgementsAcknowledgementsAcknowledgementsAcknowledgements

Co-Authors Stephanie Fry, Jennifer Crafts

& Pamela Giambo from Westat

Project Officers David Clark, William Taylor, Gladys Valentin

& Mei Wang from Centers for Medicare & Medicaid Services

This material was prepared by Westat Inc., under contract with the Centers for Medicare & Medicaid Services (CMS), an agency of the U.S. Department of Health and Human Services. The contents presented do not necessarily reflect CMS policy

3

OverviewOverviewOverviewOverview

• About the Surveys

• Survey Methods

• Questionnaire Design Issues

• Sampling and Contacting Providers

• Completes by Mode

• Factors Affecting Web Response

• Conclusions

4

About MCPSS & QIO SurveyAbout MCPSS & QIO SurveyAbout MCPSS & QIO SurveyAbout MCPSS & QIO Survey

• Two Surveys for Centers for Medicare & Medicaid Services (CMS)– QIO Provider Survey (2004)– MCPSS (2006 & 2007)

• National Sample of Medicare Providers

• Satisfaction Surveys

• Approx. 15 minute survey

• Multi-Mode Data Collection

5

Survey MethodsSurvey MethodsSurvey MethodsSurvey Methods

• Web with telephone follow-up; paper on request

• Telephone and paper surveys to:– Increase response rates– Reduce potential bias

• Importance of:– Screening– High quality mail outs– Professional Interviewers

6

Survey Methods (cont.)Survey Methods (cont.)Survey Methods (cont.)Survey Methods (cont.)

• Field Period (12-16 weeks)– Screener– Notification of web survey– Reminder letters/e-mails– Hard copy questionnaire mailing/fax– Telephone non-response– Response rates (AAPOR-RR3)

• QIO: 90%• MCPSS: 65%

7

Questionnaire Design Questionnaire Design Usability TestingUsability Testing

Questionnaire Design Questionnaire Design Usability TestingUsability Testing

• Purpose of Testing– Ease of navigation– User satisfaction and perceived burden

• Profile of Testers

• Response mode bias

8

Questionnaire Design Questionnaire Design Usability Testing IssuesUsability Testing Issues

Questionnaire Design Questionnaire Design Usability Testing IssuesUsability Testing Issues

• Allow for multiple respondents – Clarity of instructions to “designate a

colleague”

• Clarity of instruction for submitting survey

• Ability to skip sections and questions

9

Elements of the Final DesignElements of the Final DesignElements of the Final DesignElements of the Final Design

• Allow multiple respondents for each survey

• Track section-level designees

• Track section-level status

• Crisp and easy to navigate web survey

• Seamless link between CATI & Web

10

11

12

Completes by ModeCompletes by ModeCompletes by ModeCompletes by Mode

*non-response mail out of hard-copy survey in 2004

• Highest Rates of Web response – Hospitals• Lowest Rates of Web response – Physicians

Web Mail/Fax* Telephone Completes

2004 32.2% 21.9% 45.9% 17,628

2006 40.1% 1.7% 58.1% 16,121

2007 55.1% 0.9% 43.9% 17,747

13

Factors Affecting Web ResponseFactors Affecting Web ResponseFactors Affecting Web ResponseFactors Affecting Web Response

• Percentage Web access– 93% physician practices (2007)– 97% other providers (2007)

Hypotheses:• Ease of Web access at work station

• Ease of survey website navigation

• Ability to obtain help when needed

14

ConclusionsConclusionsConclusionsConclusions

• Feasibility of a Web surveys no longer an issue

• Necessary factors for survey management and achieving high response rates – telephone non-response follow up essential– Frequent interface between data collection

modes– Allow the respondent maximum flexibility

while maintaining survey integrity

15

Contacting the AuthorsContacting the AuthorsContacting the AuthorsContacting the Authors

Vasudha Narayanan (vasudhanarayanan@westat.com)

Stephanie Fry (stephaniefry@westat.com)

Jennifer Crafts (jennifercrafts@westat.com)

Pamela Giambo (pamelagiambo@westat.com)

16

Thank youThank youThank youThank you