1

Collecting Data in Multiple Languages: Development of a Methodology

SLAITS National Survey of Children with Special Health Care Needs

Seth T. Brady, Abt Associates Inc.

Larry Osborn, Abt Associates Inc.

Stephen J. Blumberg, National Center for Health Statistics

Lorayn Olson,Abt Associates Inc.

2

Introduction

• Excluding non-English speaking households from RDD telephone surveys can result in biased estimates.

• For many national surveys, the addition of Spanish-language interviewing has been sufficient to reduce this bias.

• But what happens when a study that is expected to screen over 200,000 households is relied upon to produce state-level estimates?

3

NS-CSHCN Spanish-Language Methodology

• Parallel Spanish-language CATI system.

• Telephone center staff of bilingual-Spanish interviewers.

• What about states containing a substantial proportion of households where an additional language is spoken?

4

Methodology for Additional Languages

• Identifying the Most Common Non-English/Non-Spanish Languages for Interviewing

• Translation

• Identification of Cases

• Recruiting and Training Interviewers

• Sample Management/Data Collection

• Quality Assurance

• Data Entry

5

Identifying Languages for Interviewing

• Initial investigation using National Immunization Survey data revealed that 90% of Non-English/Non-Spanish households spoke one of 11 languages:

• Arabic• Cantonese• French• Italian• Japanese• Korean

• Mandarin• Polish• Portuguese• Russian• Vietnamese

6

• NS-CSHCN pretest data confirmed 90% of households spoke one of these 11 languages

• Tagalog determined to be more prevalent than Arabic

• Portuguese later dropped due to interviewer attrition

Identifying Languages for Interviewing cont’d

7

• English questionnaire translated into 10 languages by an independent vendor

• Translated questionnaires were reviewed by • A second vendor• Interviewers• Abt Associates staff fluent in each language (where

possible)

• Response categories not translated unless read to respondent

• Identical pagination across all questionnaires

Translation

8

Recruiting and Training Interviewers

• Recruiting Strategies• Ethnic newspapers• Community groups• Local colleges/universities• Word of mouth

• Language Competence Testing

• Interviewer Training• Basic training• Project training• Mock interviewing• Live interviewing

• Vendor contacts candidates• Scores reported to Abt Associates

9

Identification of Cases

• RDD sample screened for age-eligible households using specialized interviewers and outside interpreters

• Language of respondent identified for households with eligible children

• Case removed for delivery if language was one of 10 identified for interviewing

10

Sample Management/Data Collection

• To ensure sufficient workload for interviewers, cases were allowed to accrue for several months

• Interviewers were given information regarding prior contact dates and any pertinent data gained during screening

• 98% interview completion rate

11

Quality Assurance

• Evaluating Quality of Interview

• Interviews tape recorded• First two• 10% thereafter

• Independent vendor review• Scores interview• Emails feedback to Abt Associates• Project reviews feedback with interviewers

• Review of data• Skip sequences• Key data points

12

Data Entry

• Completed cases were reviewed and sent to Data Preparation Center for entry into the CATI system

• Data entry facilitated by English response choices and identical pagination

13

Results

• 1,304 households screened for special health care needs children

• 1,281 households completed detailed interviews

• 1,233 health insurance interviews • 19 special health care needs interviews

• 29 both a special needs and a health insurance interview

14

Distribution of Screening Languages

20%

14%

14%13%

11%

10%

8%

6%4% 0.3%

Vietnamese

Mandarin

Cantonese

Korean

Russian

Tagalog

French

Japanese

Polish

Italian

15

Results

16

Results

17

Key Outcomes

• 98% interview completion rate

• 1,281 households completed detailed interviews

• 1,304 households screened for special health care needs

18

Lessons Learned

• CAPI capabilities

• Creative recruiting

• Translation preparation

Abt Associates Inc.