Development of Doorstep Introductory Survey Messages for Use across Languages: New Advancements towards Best Practices

Patricia Goerman, Yazmín A. G. Trejo, Anna B. Sandoval Giron, U.S. Census Bureau

Alisu Schoua Glusberg, Research Support Services

Presented at the 72nd annual conference of the American Association for Public Opinion Research (AAPOR)

New Orleans, LA: May 18-21, 2017Disclaimer: This presentation is intended to inform people about research and to encourage discussion. The views expressed are those of the authors and not necessarily those of the U.S. Census Bureau.

1

Introduction: Doorstep Messages

• Messages: statements for face-to-face interviewer to initiate conversation and gain respondent cooperation at the doorstep

Census Specific Messages General Messages

Showing ID card Interviewer introducing self

Purpose and frequency of Census Type of questions to be asked

Census as mandatory Confidentiality

Short length of survey No negative consequences

Literature Review: Interviewer Characteristics and Introductory Messages

• Studies of interviewer characteristics and behavior – Race, gender, SES, interviewer skills, cross cultural differences

• Communication style• Announced v. unannounced visits• Particular messages: eg. non-solicitation• Paradata on doorstep interaction to explain non-response• Little work on messages across languages/ cultural groups

3

Previous U.S. Census Bureau Research

1. Field observation interviews in 8 languages– 2010 observation of 586 CAPI interviews (Pan & Lubkemann, 2013)

2. Expert review (2015) 3. Focus groups: 7 languages (2015)

– Spanish, Chinese, Vietnamese, Korean, Russian, Arabic, English

2. CBAMS: Census Barriers Attitudes and Motivators (2008 & 2011)

4

Overview: Highlighted Findings on Salient Messages-Past Census Bureau Research (2010, 2015)

– All languages (Spanish, Chinese, Korean, Vietnamese, Russian, Arabic, English)• Verifying interviewer identity• Purpose and importance of Census • Advance notification of visit• Confidentiality

– Politeness/honorific markers (Chinese, Korean, Vietnamese, Russian)– Order of messages (Chinese, Vietnamese, Korean)– Appropriate attire (Spanish, Arabic)– Gender matching of interviewer and respondent (Arabic)– Mandatory message - (good: Korean, Russian)

- (bad: Korean, Spanish, Chinese, Arabic)

5

Current Research: 2017 Doorstep Messaging Focus Groups

• 7 Languages: Spanish, Chinese, Vietnamese, Korean, Russian, Arabic, English (6 focus groups in each language)

• Non-English speaking v. bilingual speakers (3 groups each) • Bilinguals: target language dominant speakers• English groups:

– Baseline– Compare bilingual v. English only

6

Four types doorstep interaction videos

1. Language Barrier2. Unaware3. Fear/Mistrust of government4. Low Engagement

Type Probing and Discussion

What did you like about the video? Interviewer tone, appearance, gestures, eye contact?

What did you dislike about the video? Was there anything some people might find difficult to

understand? Anything unnatural/inappropriate for your culture? Would you be willing to participate if you heard these

messages?

8

Project Status

• Majority of focus groups have been completed• Final groups to be completed early June• Analysis underway, report to be completed by Sept 2017• Discussion of preliminary findings

9

10

Chinese Preliminary FindingsMessages Doorstep interactions

How census can benefit Chinese community helpful

Description of short length of survey

Confidentiality and privacy Mandatory message should help

General importance of the Census

Fears related to immigration status, legal issues

Desire for proof of legitimacy such as special attire, uniforms

Importance of building trust between interviewer and respondent.

ID should be bilingual Concerns about language barrier, lack of

understanding

11

Chinese Preliminary FindingsMessages Doorstep interactions

How census can benefit Chinese community helpful

Description of short length of survey

Confidentiality and privacy Mandatory message should help

General importance of the Census

Fears related to immigration status, legal issues

Desire for proof of legitimacy such as special attire, uniforms

Importance of building trust between interviewer and respondent.

ID should be bilingual Concerns about language barrier, lack of

understanding

12

Vietnamese Preliminary FindingsMessages Doorstep interactions

Benefits to the local community (eg. schools, hospitals)

Description of short length of survey

Mandatory message helpful, though not encouraging

Offer to return a different day to complete survey

Assurances: no negative consequences

Fears related to immigration status, legal issues

Lack of trust of someone at doorstep

Concern about effect of participation ongovernment benefits received (eg. food stamps)

Importance of having written materials to look at to help gain trust

Importance of patience and politeness of interviewer, use of honorific markers

13

Vietnamese Preliminary FindingsMessages Doorstep interactions

Benefits to the local community (eg. schools, hospitals)

Description of short length of survey

Mandatory message helpful, though not encouraging

Offer to return a different day to complete survey

Assurances: no negative consequences

Fears related to immigration status, legal issues

Lack of trust of someone at doorstep

Concern about effect of participation ongovernment benefits received (eg. food stamps)

Importance of having written materials to look at to help gain trust

Importance of patience and politeness of interviewer, use of honorific markers

14

Korean Preliminary FindingsMessages Doorstep interactions

Brief and “real” examples of Census benefits for the Korean American community or the community where respondents lived in

Length of the survey is important and should be said upfront.

“Warm expressions” related to “help” rather than mandatory and legal expressions.

Safety is the biggest concern (i.e. opening the door to talk to an interviewer)

Census’ interviewer vs. door to door sales person. (i.e. wearing something distinguishable besides the ID badge)

High quality translations of the forms available, otherwise people prefer to use the English form

15

Korean Preliminary FindingsMessages Doorstep interactions

Brief and “real” examples of Census benefits for the Korean American community or the community where respondents lived in

Length of the survey is important and should be said upfront.

“Warm expressions” related to “help” rather than mandatory and legal expressions.

Safety is the biggest concern (i.e. opening the door to talk to an interviewer)

Census’ interviewer vs. door to door sales person. (i.e. wearing something distinguishable besides the ID badge)

High quality translations of the forms available, otherwise people prefer to use the English form

16

Spanish Preliminary FindingsMessages Doorstep interactions

Good to discuss benefits to community

Description of short length of survey, ease of completion

Mention of confidentialityimportant

Mandatory message should be deemphasized or avoided

Fears related to immigration status, issue of having more residents than allowed on lease

Need for up front friendliness and rapport building

Good to use local interviewers familiar with the community

17

Spanish Preliminary FindingsMessages Doorstep interactions

Good to discuss benefits to community

Description of short length of survey, ease of completion

Mention of confidentialityimportant

Mandatory message should be deemphasized or avoided

Fears related to immigration status, issue of having more residents than allowed on lease

Need for up front friendliness and rapport building

Good to use local interviewers familiar with the community

18

Arabic Preliminary Findings

Messages Doorstep interactions

Census benefits are important to mention

Confidentiality all the information provided remains confidential

Mandatory: “It’s the law” and everyone should participate encourages participation

Emphasize that census does not collect immigration status information

Markers of legitimacy such as ID badge, uniform, business cards, and census bags

Respondents like the use of the security warning card at the doorstep

19

Arabic Preliminary Findings

Messages Doorstep interactions

Census benefits are important to mention

Confidentiality all the information provided remains confidential

Mandatory: “It’s the law” and everyone should participate encourages participation

Emphasize that census does not collect immigration status information

Markers of legitimacy such as ID badge, uniform, business cards, and census bags

Respondents like the use of the security warning card at the doorstep

20

Russian Preliminary Findings

Messages Doorstep interactions Confidentiality all the

information provided remains confidential

Mandatory: It’s the law and required by all those living in the United States

Civic duty filling out the census is a civic duty

Awareness campaigns prior to data collection

Immigration status will not be collected should be stated at the start of interaction

Safety is the biggest concern (i.e. opening the door to talk to an interviewer)

Census’ interviewer markers of legitimacy such as an ID and Census logo bag

21

Russian Preliminary Findings

Messages Doorstep interactions Confidentiality all the

information provided remains confidential

Mandatory: It’s the law and required by all those living in the United States

Civic duty filling out the census is a civic duty

Awareness campaigns prior to data collection

Immigration status will not be collected should be stated at the start of interaction

Safety is the biggest concern (i.e. opening the door to talk to an interviewer)

Census’ interviewer markers of legitimacy such as an ID and Census logo bag

22

English Preliminary Findings

Messages Doorstep interactions Short length of the interview

considered as the most salient messages for encouraging participation

Mandatory “It’s the law”

Census benefits are interpreted as not credible statements

ID badge is easy to see was mentioned as important factor

Feeling uncomfortable with having an interviewer coming to their door (i.e. invasion of space)

Respondents like knowledgeable interviewers with a positive attitude

23

English Preliminary Findings

Messages Doorstep interactions Short length of the interview

considered as the most salient messages for encouraging participation

Mandatory “It’s the law”

Census benefits are interpreted as not credible statements

ID badge is easy to see was mentioned as important factor

Feeling uncomfortable with having an interviewer coming to their door (i.e. invasion of space)

Respondents like knowledgeable interviewers with a positive attitude

Overview of Salient Messages by LanguageChinese Vietnamese Korean Spanish Arabic Russian English

Benefit √ √ √ √ √

Length √ √ √ √ √

Confidentiality √ √ √ √

Mandatory good √ √ √ √

Mandatory bad √ √

Importance √

No negative √

Return other day √

Warm expression √

Civic duty √

Awareness √

Census benefits not credible √

24

Overview of Interaction Issues by LanguageChinese Vietnamese Korean Spanish Arabic Russian English

Showing ID √

Immigration status √ √ √ √ √

Uniforms √ √ √ √

Trust √ √

Language/translation √ √

Receipt of benefits √

Written materials √ √

Politeness √

Safety √ √

Friendliness √

Local interviewers √

Knowledgeable √

Privacy/ personal space √ √

25

Next Steps In depth analysis of findings by language Examination of differences between bilingual v. monolingual

respondents Compilation of best practice recommendations for each

language and specific wording Final project report by Sept 2017

26

Future Plans1. Interviewer Training Looking into development of 2018 and/or 2020 training modules for

bilingual interviewers and/or language barrier situation Interviewer feedback/follow up evaluations

2. Collaboration with 2020 Census advertising research 3. Continued Research through 2020 Census Proposing observation of field interviews during 2020 Census Interviewer and respondent debriefings Collaboration with Decennial Language team and Field operations

27

Development of Doorstep Introductory Survey Messages for Use across Languages: New

Advancements towards Best PracticesPatricia Goerman, Yazmín A. G. Trejo, Anna B. Sandoval Giron,

U.S. Census BureauAlisu Schoua Glusberg, RSS

For more information: E-mail: Patricia.L.Goerman@census.gov