Translation of survey instruments
Alisú Schoua-Glusberg, Ph.D.Research Support Services
DC/AAPOR Presentation – 7/7/04
Research Support Services - 2004
2
Translation methods for survey instruments
This presentation will focus on:
problems involved in questionnaire translation methods and approaches for the translation of survey
instruments,
use of qualitative methods to assess translation quality and the performance of the translated instrument,
providing survey translators the information they need.
Research Support Services - 2004
3
When is Questionnaire Translation Necessary?
Cross-national studies
Cross-cultural studies
National studies by Federal Statistical Agencies
Studies of special populations
Studies where sample falls in areas with significant concentrations of speakers of other languages
Research Support Services - 2004
4
Factors to Consider to Define the Target Audience of Translated Instruments Language minorities Immigrant populations Monolingual (or at least not
bilingual) Language different from designers’ Lower level of education Age at immigration Hybrid culture Different degrees of acculturation
Research Support Services - 2004
5
Source & Target Question Characteristics to Consider
Meaning Style Complexity Source flexibility Cultural aspects Existence of equivalent realities
Research Support Services - 2004
6
Using Previous Translations of Questions vs. Translating Anew
Issues:
Maintaining longitudinal comparability vs. improving questions
Is priority to compare with past translations or with English questionnaire?
Research Support Services - 2004
7
Questionnaires are a Complex Text Type
Little context to clarify ‘intended meaning’
Ambiguous
Measurement properties lead to ‘surveyspeak’ and ‘scalespeak’ (Harkness 1996)
Translators thus often not versed in questionnaire discourse ‘rules’
Special kind of conversation
Research Support Services - 2004
8
Question ‘Equivalence’ is Expected in Translation
Expectation that the translated question… Says the same thing Means the same thing Measures the same thing Measures equivalently (e.g. scales) Imposes same burden on respondents Meets reliability and validity requirements
Research Support Services - 2004
9
Translators’ Job
Translators make decisions about: Semantics Syntax (structure) Pragmatics
They need to understand: Intended “meaning” in order to translate Covert (measurement) intention and
requirements
Research Support Services - 2004
10
Translators therefore…
Have little context and co-text
Have little supporting documentation
Often have no questionnaire ‘author’ to consult
May experience uncertainly about what to “match”
May make decisions based on their experience with other types of text
Research Support Services - 2004
11
Risks of Overly Close Translations Focus on meaning of words rather than
meaning of questions
Rs are inadvertently asked a different question
Processing is more complex
Translated questionnaire sounds unidiomatic
Research Support Services - 2004
12
Steps in producing and testing quex translations Translation Translation review Decisions/Adjudication Quality control Qualitative research Pretesting Documentation
Research Support Services - 2004
13
Survey Translation Approaches One translator - one translation
(direct translation)
Multiple translators – one translation (split committee)
Multiple translators – multiple parallel translations (parallel committee)
Research Support Services - 2004
14
Committee Approach
Three translators prepare translations independently (split or parallel)
Reconciliation meeting with referee
Qualitative research with monolinguals (focus groups and/or cognitive interviews)
Research Support Services - 2004
15
Committee Approach: Reconciliation Meeting Question-by-question review
Reaching consensus when possible
Providing alternatives if no consensus possible
Identifying terms/items for qualitative research
Research Support Services - 2004
16
Committee Approach: Referee’s Role
Resolves style disagreements
Manages interaction
Brings survey researcher perspective
Keeps an eye on source version
Pushes for global decisions
Liaison with research team for consultation and documentation
Research Support Services - 2004
17
Committee Approach: Cognitive Interviews Allow to administer all or part of
instrument
Give a glimpse into thought processes
Allow to see how different alternative terms work
Research Support Services - 2004
18
Committee Approach: Focus Groups
Permit to distinguish what is idiosyncratic
Allow us to listen to how Rs. use language
Allow to include more people in a shorter time
Allow to include different national origin Rs and see if they reach consensus
Research Support Services - 2004
19
Advantages of Committee/ Team Approaches
Group process benefits
Include different varieties of language in translation team
Qualitative research that follows allows to incorporate the target population into process
Relatively low cost
Relatively quick
Research Support Services - 2004
20
Selected Committee Translations: 2000-2004
NSFG (U. Mich.) ATUS (BLS) NHSDA (RTI) ICARIS (Battelle) NMHS (Battelle) SLAITS Asthma Q. (Abt) Job Corps Student Q. (Battelle) Survey of Consumer Attitudes (U.
Mich.) Catholic Voters Political Attitudes
(BRS) McNair Program Evaluation (DIR) L.A. Latino Eye Study (USC)
Women’s Health Initiative Q. (U. Mich.)
WHO Health & Performance Q. (Harvard)
PHDCN (Harvard) Project Bread (UMass-Boston) California Safe Schools
(RAND) CAHPS Dialysis Center
(RAND) National Children’s Study Pilot
(Battelle) Survey of Bioterrorism Pre-
paredness (N.Y. Academy of Medicine)
Research Support Services - 2004
21
Textual assessments Translation appraisalHolistic approaches, e.g., TRAPD, committeeBack translation
Pretesting with bilinguals, e.g.,SplitsDouble administrations
Debriefings Probe interviews Think alouds with Respondent or Translator Focus groups with sample population
Assessing Translations: some procedures(Harkness, Pennell, Schoua-Glusberg – ASA, 2003)
Research Support Services - 2004
22
Backtranslation
One person translates from source into target language.
A second person translates the target language version back into the source language.
A third person compares the original and the backtranslated source language versions.
Discrepancies are investigated.
Research Support Services - 2004
23
What is wrong with back- translation?
It is a black box: we know what went in, we know what came out, yet we know nothing about the adequacy of the target language version.
Research Support Services - 2004
24
Why give Q-by-Q specs to translators?
In the absence of question-by-question specifications, translators make their own decisions…
to resolve ambiguities to figure what “they” mean
These are most often not documented or even explicit. They happen in the translator’s mind.
Research Support Services - 2004
25
Research Conducted
Goal: to examine translators’ decision processes
Vehicle: Recorded a translators’ Committee meeting
Method: Listened to tape searching for decisions made during discussions
Research Support Services - 2004
26
Why this design?
Committee discussion involves:
–Review
–Translation–Adjudication
Makes decisions verbally explicit
Research Support Services - 2004
27
Transcription Detail:
2 hours of committee meeting
11 pages of questionnaire
110 questions
Research Support Services - 2004
28
Example 1
How would your parents feel if they found out you drank alcohol sometimes?
Not at all upset
A little upset
Pretty upset Very upset
Research Support Services - 2004
29
Discussion about “upset”Did “they” mean…
Angry?
Perturbed / Bothered?
Given context, committee decided to translate upset as angry.
Research Support Services - 2004
30
Example 2
I think sometimes it’s okay to cheat at school.
Strongly Agree
Agree
Disagree
Strongly Disagree
Research Support Services - 2004
31
Discussion about ‘cheat’Original Item: I think sometimes it’s okay to cheat at school.
First translation:
Creo que a veces está bien hacer trampa en la escuela.
Referee: “¿Hacer trampa? Everyone agrees?”
Orig. Tr.: To me it refers to copying in exams.
Tr2: “You can also copy when you are doing homework or other schoolwork, that is cheating too, not only in exams.” “And you could cheat in sports, so we need to be specific. In this case this refers to school work, not to playing volleyball.”
Resolution: Creo que a veces está bien copiar o hacer trampa en mis exámenes y tareas escolares.
(I think sometimes it’s okay to copy or cheat in my exams and schoolwork.)
Research Support Services - 2004
32
Example 3
How do you feel about your ability to care for your teen when they are sick or upset?
Discussion about ‘upset’.
In this case, the discussion centered around whether ‘upset’ meant angry, not feeling well, or bothered by something.
The committee went with the latter option.(‘upset’: molesto)
Research Support Services - 2004
33
Example 4How do you feel about your ability to discipline your
teen?
Discussion about ‘discipline’. Do “they” mean:• Punish?• Teach them to behave properly? • Set rules for them?
Checked with client who opted for ‘punish’.
Research Support Services - 2004
34
Example 5
How do you feel about your ability to obtain needed resources for your teen?
Discussion about ‘resources’. Did “they” mean...
Material resources?Assistance/help?
Client asked to preserve the ambiguity as much as possible. Translated as ‘recursos’.
Research Support Services - 2004
35
Example 6
During the past week, how often did you let this teen know you really care about him/her?
Discussion about ‘care’. Did “they” mean…love?concern?
Committee decide to include both:… how much you love him/her and are concerned about him/her
Research Support Services - 2004
36
Providing Specifications:Ideal ModelThree steps:
1. Deliver question-by-question specifications
2. Review original text with translation team member
3. Ask translators to make their decisions explicit and submit them for review.
Research Support Services - 2004
37
Format for Documentation
Q# Original Q. Issue D/QSpecify decision or
question
Research Support Services - 2004
38
Recent/Current Efforts to Standardize Translation Procedures
U.S. Census Bureau translation guidelines
International Social Survey Programme’s translation methods’ work
European Social Survey Implemented Procedures in 20+ countries
European Social Survey and Survey of Health, Aging and Retirement in Europe analyzing translation and documentation outputs
CAHPS Cultural Comparability Task Force
NCHS Translation Issues Forum
CSDI (Comparative Survey Design and Implementation Translation Task Force)
Top Related