Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training...
-
Upload
anna-dillon -
Category
Documents
-
view
215 -
download
0
Transcript of Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training...
![Page 1: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/1.jpg)
Improving the Effectiveness of Interviewer Administered Surveys
though Refusal Avoidance Training
Grace E. O’Neill
Presented by Anne Russell
U.S. Census Bureau
ICES-III - June 19, 2007
![Page 2: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/2.jpg)
2
Outline of Presentation
• Background
• Respondent focus groups
• Refusal avoidance training
• Training content
• Discussion
• Future developments
![Page 3: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/3.jpg)
3
Background
![Page 4: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/4.jpg)
4
Respondent contact staff
• Interviewers: outgoing calls to gain cooperation and to gather data
• Clerks: incoming calls to provide basic information and resend forms
• Analysts: professional staff who place outgoing calls concerning data errors or nonresponse follow-up
![Page 5: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/5.jpg)
5
Respondent contact training
• Interviewers: Centralized training on telephone skills and refusal avoidance and conversion
• Clerks: Shadow experienced clerks with informal discussion
• Analysts: Basic training on respondent contact techniques
![Page 6: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/6.jpg)
6
Why clerks?
• Self-administered paper/ electronic forms
• Establishment respondents need less encouragement to participate
• Nature of telephone calls
![Page 7: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/7.jpg)
7
However…
Over time, the role of the clerk has evolved into the role of an interviewer
• Less cooperative respondents
• More data collected over the telephone
• More nonresponse follow-up telephone calls
Yet, their training has not evolved
![Page 8: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/8.jpg)
8
Respondent Focus Groups
![Page 9: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/9.jpg)
9
Focus Groups
• Monthly Trade Surveys
• Wholesale and retail firms
• Conducted by outside firm
• Six focus groups, 44 participants
• Respondent’s views and impressions
• Ways to improve survey
![Page 10: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/10.jpg)
10
Focus Groups
Finding
• Inexperienced clerks needed skills to increase participation by respondents, especially on voluntary surveys
Solution
• Refusal avoidance training
![Page 11: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/11.jpg)
11
Refusal Avoidance Training
![Page 12: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/12.jpg)
12
Refusal Avoidance Training
• Groves and McGonagle (2001)
– Assemble respondent concerns
– Develop responses
– Train interviewers to classify concerns
– Train interviewers to provide quick and appropriate responses
![Page 13: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/13.jpg)
13
Refusal Avoidance Training
• Interactive
• Cooperative learning
• Flexible
• Survey specific
• Provides telephone skills, refusal avoidance techniques, and improves communication
![Page 14: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/14.jpg)
14
Training specifics
• Monthly Trade Surveys (MTS) and Quarterly Services Survey (QSS)
• 43 MTS clerks, 14 QSS clerks
• Mix of tenure and survey experience
• Supervisor and survey manager involvement
• Eight hours over two days
![Page 15: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/15.jpg)
15
Training Preparation
• Adapt household- based training to establishment-based survey
• Adapt training to the MTS or QSS
• Identify respondent concerns and develop solutions
• FAQ Job Aid
![Page 16: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/16.jpg)
16
Training Content
![Page 17: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/17.jpg)
17
Refusal avoidance training modules
Module 1: IntroductionModule 2: Survey SpecificModule 3: Shared ExperienceModule 4: Preparing a Telephone CallModule 5: Telephone SkillsModule 6: Identify, Analyzing, and Dealing with
Reluctance and RefusalModule 7: Recovering from Negative CallsModule 8: Wrap-up and Evaluation
![Page 18: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/18.jpg)
18
Module 1
Introduction
• Introduces training to clerks
• Introduces the trainer and clerks to each other
• Provides training schedule
![Page 19: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/19.jpg)
19
Module 2
Survey Specific
• Introduces survey specific content
• Conducted by survey manager or survey staff
![Page 20: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/20.jpg)
20
Module 3
Shared Experience
• Identify clerks’ biggest concerns and difficulties
• Develops solutions
• Concerns are used in a later module
![Page 21: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/21.jpg)
21
Module 3 example
• Trainer: “What are some of the things respondents say when you talk to them?”
• Clerks: “Why should I do this,” “I’m not a wholesaler” etc.
• Group: Identify appropriate responses • Trainer: Make sure identified concerns
are addressed and adds new concerns to list
![Page 22: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/22.jpg)
22
Module 4
Preparing for the Telephone Call
• Asks clerks how they prepare for telephone calls
• Assess what tools clerks use to find information about company and what tools they might need
![Page 23: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/23.jpg)
23
Module 5
Telephone Skills
• Assess clerks active listening skills
• Discuss tone
• Discuss mechanics of placing a telephone call
![Page 24: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/24.jpg)
24
Module 6
Identifying, Analyzing and Dealing with Reluctance and Refusal
• Classify concerns identified in Module 3 as reluctance and/ or refusal
• Further discuss solutions
• Paired practice during class
• Reviewed updated FAQ Job Aid
![Page 25: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/25.jpg)
25
Module 7
Recovering from Negative Calls
• Discusses recovering from refusals and other negative calls
• Helps clerks to evaluate negative experiences
![Page 26: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/26.jpg)
26
Module 8
Wrap-up and Evaluation
• Review main training points
• Clarify any remaining concerns
• Evaluations by clerks
![Page 27: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/27.jpg)
27
Results
![Page 28: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/28.jpg)
28
Results
Clerk evaluation
• Usefulness of workshop
• Usefulness of skills learned
• Increased confidence, preparation, and communication
![Page 29: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/29.jpg)
29
Results
Response rates
Caveats:
• Not experimentally tested
• Confounded by other survey conditions
• Don’t know how many potential refusals clerks prevented
![Page 30: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/30.jpg)
30
Results: MTS
Initially lower from previous response period
• Loss of clerks
• Misclassification of refusals
![Page 31: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/31.jpg)
31
Results: MTS
Over time
• Good response rates, decrease in wholesale refusals
• Continued improvement
• Clerks have more responsibilities
![Page 32: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/32.jpg)
32
Results: QSS
Decline in response rates
Imputation remained stable
• More companies refused
• Larger companies continued to report
![Page 33: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/33.jpg)
33
Results: QSS
• Communication has improved
• Clerks are open about sharing concerns with survey managers
• Bi-weekly telephone calls with call center
• Annual refresher training
![Page 34: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/34.jpg)
34
Discussion
![Page 35: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/35.jpg)
35
Benefits
• Centralized dissemination of skills and information
• Practice occurs in a test environment
• Provides training at regular intervals
• Proactive training instead of reactive training
![Page 36: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/36.jpg)
36
Benefits
• Communication between clerks, supervisors, and survey managers
• Clerks feel invested in data collection process
• Survey managers gain direct insight into data collection process
![Page 37: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/37.jpg)
37
Costs
• Staff time writing and delivering training
• Telephone coverage
• Monetary cost of training
• Difficult to provide conclusive evidence
![Page 38: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/38.jpg)
38
Future Developments
![Page 39: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/39.jpg)
39
Future Developments
• Formalized process for clerk training
• Data capture of call concerns
• Follow-up evaluation by clerks
• Analyst training
![Page 40: Improving the Effectiveness of Interviewer Administered Surveys though Refusal Avoidance Training Grace E. ONeill Presented by Anne Russell U.S. Census.](https://reader036.fdocuments.us/reader036/viewer/2022062511/5514bcb455034640138b5673/html5/thumbnails/40.jpg)
40
Thank You
Please feel free to contact the author at:
Grace E. O’Neill
Email: [email protected]
Phone: 301-763-3537