Faculdade de Medicina da Universidade do Porto
Curso de Mestrado Integrado em Medicina2010/2011
INTRODUÇÃO À MEDICINA II
TURMA 21REGENTE: Prof Dr. Altamiro Costa Pereira
PROFESSORA E ASSISTENTE: Prof Drª. Cristina Santos e Luís Pinto
http://jpn.icicom.up.pt/imagens/up/up_fmedicina.jpg
Biases in questionnaire construction:
how much do they influence the
answers given?
Types of errorsRandom Errors
Affect questionnaires’ precision
Related to the variability inherent to a phenomenon that is being measured [1]
Systematic Errors Affect
questionnaires’ validity
Related to the wrong association between an exposure anda certain effect [1]
Commonly known as “biases”
1. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641
Scenario Analysis of questionnaires’
results based on false/inaccurate information
Leads To
Descriptions and catalogues of common
biases [2,5]
To inform investigators To minimize biases
Description of biases in literature
2. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-6415. Bernard C.K. Choi and Anita W.P. Pak. A Catalog of Biases in Questionnaires. Preventing Chronic Diseases.
2005; 1; 13
Described biasesQuestion Design
• Ambiguous questions • Technical jargon • Uncommon words
Problems with wording
• Leading questionLeading questions
• Missing intervalFaulty scale
• Sensitive questionsIntrusiveness
• Belief vs. behavior
Missing or inadequate data for intended purpose
Described biasesAdministration of questionnaire
The “Don’t know”, “Undecided”, and “Neutral response” options
• Faking Good
Respondent’s conscious reaction
Described biases
Questionnaire Design
Formatting problem
• Horizontal response format
Research Question Do these so called
‘biases’ really influence the answers given in
questionnaires?
http://static4.depositphotos.com/1000423/471/i/110/depositphotos_4715431-3D-man-with-question-mark.jpg
Check if the way we change a normal question influences the given answers in general.
Be able to sort out at what extent can each of the types of biases mentioned above manipulate the results obtained.
To realize whether there is evidence that the described
biases interfere in questionnaires’ results
Aims
Methods
Two versions of the same
questionnaireA and B
Both with biased and unbiased questions
Target Population
First year students from FMUP
Versions A and B• Randomly assigned
questions to versions A and B of the questionnaire
Questionnaire
arquitecture
• 19 different classes• Distribution prevented them from
detecting two different versions of the same questionnaire.
Questionnaire
Distribution• Gathering the answers given• Creation of a data base in SPSS• Comparison between the supposedly
biased and respective non-biased questions using Qui-square test and T-student test
• Analysing if the documented biases are indeed biases (observing p value)
Questionnaire Analysis
Data AnalysisThe analysis of the answers given was
only possible because we consider that the odds of obtaining a certain answer is similar in both groups of each class.
Our outcome was the rates in the answers given to supposedly biased and unbiased questions, comparing the rates to see if there was any significant differences.
To analyse our results we use the statistical analysis software SPSS 19.0.
Final Results and Discussion
Question A1 vs. B1-2 Biased question (A1) :
Do you lend your tooth paste or tooth brush to other people?
Yes [ ]
No [ ]Unbiased question (B) :
1) Do you lend your tooth paste to other people?Yes [ ]No [ ]
2) Do you lend your tooth brush to other people?Yes [ ]No [ ]
A B1 p
No 77 (77,8%) 29 (30,9%)<0,0001
Yes 22 (22,2%) 65 (69,1)
A B2 p
No 77 (77,8%) 93 (100%)<0,0001
Yes 22 (22,2%) 0 (0%)
Ambiguous question
Question A2 vs. B3 Biased Question (A2):
When you take a bath, do you wash your genital area?
Yes [ ]
No [ ] Unbiased question (B3):
When you take a bath, do you wash your hair?
Yes [ ]
No [ ]
A B p
No 0 (0%) 1 (1%)
0,301Yes 99 (100%) 92 (95,8%)
Missing 0 (0%) 3 (3,2%)
Sensitive question
Question A3 vs. B4Biased question (B4):
You have a shower everyday, don´t you?
Yes [ ]
No [ ]
Unbiased question (A3):
How often do you have a shower in the week: More than 6 days per week[ ]
5-6 days per week [ ]
2-4 days per week [ ]
1 day per week[ ]
I never have a shower [ ]
A B p
(A)More than 6 days per week/(B)Yes
67(67,7%)
82(87,2%)
0.001A)I never have a
shower; 1, 2-4, 5-6 days per week/ (B) No
32(32,3%)
12(12,8%)
Faking good
Question A4 vs. B5Biased question (A4):
Do you reckon that physical exercise is a germane aspect of your health?
Yes [ ]
No [ ] Unbiased question (B5):
Do you consider that physical exercise is a relevant aspect of your health?
Yes [ ]
No [ ]A B Confidence
Intervals
No 7 (7,07%)[-0,00493;0,019125]
3 (3,1%)[-0,00486;0,011062]
_
Yes 85 (85,8%)[0,809155; 0,908845]
91 (94,8%)[0,916201;0,979799]
Statistically Significant
Missing 7 (7,07%)[-0,00493;0,01907]
2 (2,1%)[-0,00446;0,008656]
_
Uncommon word
Question A5 vs. B6Biased question (B6):
Do you think it is important to practice physical exercise at least 3 times a week?
Yes [ ]
No [ ] Unbiased question (A5):Do you practice physical exercise at least 3 times a week?
Yes [ ]No [ ]
A B p
No 69 (69,7%) 15 (16,1%)
<0,0001Yes 30 (30,3%) 78 (83,9%)
Belief vs. behaviour
Question A6 vs. B7Biased question (B7):Your eating habits are: Excellent [ ] Very good [ ] Good [ ] Fair [ ] Somehow deficient [ ] Poor [ ] Very poor [ ]
Unbiased question (A6):Your eating habits are:
Excellent [ ] Very good [ ]Good [ ] Fair [ ] Somehow deficient [ ]Poor [ ] Very poor [ ]
A B p
Very Poor 2 (2%) 0 (0%)
0,463
Poor 2 (2%) 2 (2,1%)
Somehow deficient
5 (5,1%) 2 (2,1%)
Fair 14 (14,1%) 20 (21,3%)
Good 55 (55,6%) 56 (59,6%)
Very Good 17 (17,2%) 11 (11,7%)
Excellent 4 (4%) 3 (3,2%)
Horizontal response format
Question A7 vs. B8Biased question (A7):
Do you agree that McDonalds is responsible for the increase of hypertension rates?
Agree [ ]
Neutral opinion [ ]
Disagree [ ]
Unbiased question (B8):
Do you agree that McDonalds is responsible for the increase of hypertension rates?
Agree [ ]Disagree [ ]
A BConfidence
Intervals
Disagree12 (12,1%)
[-0,00356;0,027759]33 (35,9%)
[0,008297;0,060503] _
Agree49 (49,5%)
[0,018434;0,080566]59 (64,1%)
[0,027091;0,095909]_
“Neutral response” options
Question A8 vs. B9Biased question (B9):
How many days per week do you dedicate to study anatomy?
2 days [ ]
5 days [ ]
7days [ ]
Unbiased question (A8):
How many days per week do you dedicate to study anatomy?
1 day [ ]
2 days [ ]
3 days [ ]
4 days [ ]
5 days [ ]
6 days [ ]
7 days [ ]Mean
Standard deviation
p
A 4,11 1,4350,462
B 3,95 1,649
Faulty scale
Question A9 vs. B10Biased question (B10):
Do you arrange activities with your friends, such as cinema, every week?
Yes [ ]No [ ]
Unbiased question (A9):Do you arrange activities with your friends every week?
Yes [ ]No [ ]
A B p
No 30 (30,9%) 25 (36,6%)
0,509Yes 67 (69,1%) 69 (73,4%)
Leading question
Question A10 vs. B11Biased question (A10):
Did you exercise your Wernicke’s area while responding to this questionnaire?
Yes [ ]
No [ ]
Unbiased question (B11):
Did you understand what you read in this questionnaire?
Yes [ ]
No [ ]
A BConfidence
Intervals
Yes53 (53,5%)
[0,463564;0,606436]93 (96,9%)
[0,944177;0,993823]Statistically Significant
No20 (20,2%)
[0,144497;0,259503]1 (1%)
[-0,00353;0,005527]Statistically Significant
Missing26 (26,3%)
[0,003381;0,049219]2 (2,1%)
[-0,00446;0,008656]_
Technical jargon
ConclusionSumming up, there are a lot of
biases described in literature. Through this study, it has been concluded that some of them might work and others not. We cannot, however, extrapolate these results to the general population.
http://us.cdn2.123rf.com/168nwm/texelart/texelart1011/texelart101100030/8220784-3d-human-by-the-symbol-of-victory-with-his-fingers.jpg
References (Presentation)
1. M. Delgado-Rodríguez and J Llorca, Bias, J. Epidemiol. Community Health 2004;58;635-641
2. Bernard C.K. Choi and Anita W.P. Pak. A Catalog of Biases in Questionnaires. Preventing Chronic Diseases. 2005; 1; 13
3. http://www.statpac.com/surveys/undecided-category.htm
Afonso QuelhasAndré SantosBárbara AraújoCarolina SilvaCatarina MarquesCláudia OliveiraLaura GonçalvesLuís SousaMarta Sanhá Rita CostaTânia Costa
Top Related