Qualitative Methods and Questionnaires Based on slides by Colin Price.
-
Upload
aubrey-merritt -
Category
Documents
-
view
213 -
download
1
Transcript of Qualitative Methods and Questionnaires Based on slides by Colin Price.
Qualitative Methods and Questionnaires
Based on slides by Colin Price
Qua
litat
ive
Res
earc
hS
lide
fro
m
htt
p:/
/cs.
joe
nsu
u.f
i/pa
ge
s/su
ho
ne
n/S
ciM
et2
00
9/w
ee
k3_
qu
alit
ativ
e_
mix
ed
_m
eth
od
.pd
f
The typo is in the original…
Case Study 1 IS 2008-9
How to Create an Immersive Environment for Primary Literacy.
1. Production of a Demonstrator and presented to teachers. Discussion. Voice recording made.
2. Production of Prototype 1 and presented to teachers. Discussion. Voice recording made. Questionnaire administered (Open and Closed).
3. Production of Prototype 2 and presented to teachers. Voice recording made. Questionnaire administered (Open and Closed).
4. Deployment with Children, Observations made.
5. Data analysis, Hypothesis formation.
Case Study 2 IS 2009-10
Visual Semiotics in Games – How does visual content influence players’ movement and choices
1. Questionnaire Administered
2. Observations
1. Player played 2 x 2 minutes of Unreal. Video recording. Researcher made notes.
2. Movie replayed and researcher asked player to explain choices and movement. Audio Recording.
3. Next .. Data analysis …
The Role of the Researcher.
1. Present in the generation of data
2. Can influence the generation of data
3. Is involved in the subjects’ activities
The Qualitative Data generated in both Case Studies.
1. Text response to open questionnaire items
2. Text written by researcher during observations
3. Video recordings
Aims of Qualitative Research
• Understand the phenomena from the point of view of the research participants
• Interpret experiences and meanings of people
• Discover themes and relationships and key words to formulate closed questions.
Characteristics of Qualitative Research
• Participants selected purposefully not randomly
• Involves behaviour of people, opinions and feelings
• Uses details of individual cases in gaining understanding
• Researcher is a “participant observer”
When to use Qualitative Methods
• Concepts need to be developed (little previous research)
• Phenomena cannot be reduced to numerical variables
• Need for novel researcher-devised framework
Qualitative Data, Sampling and Analysis
• Data Types: Text response to questionnaire, transcripts of observations, screen-shots from video recordings
• Purposive sampling to maximize variation
• Snowball Sampling: Select a sample and ask them to recommend next sample
• Data processing : summary, simplification, abstraction
• Data display : tables, charts, networks
Quantitative Qualitative
Meanings derived from numbers.
Meanings expressed through words.
Numerical data. Rich, interesting, data.
Analysis conducted through the use of diagrams and statistics.
Analysis conducted through categorisation into “themes”
Qualitative vs. Quantitative.
Analysing Qualitative Data: Case Study 1
Example. Researcher’s analysis of her own observation notes/transcripts.
1.First reading of transcripts: Look for themes emerging.
2.Second reading: Look for occurrences of each theme in each transcript. Identify concepts.
3.Look for relationships between themes.http://www.researchsupport.com.au/More_than_themes.pdf
Analysing Qualitative Data: Case Study 1
Themes from the researcher’s Observation Transcripts
(Item) Theme Comments
(24) “used quit clock”
(2) “understood how to select a lesson” (8) “use of floor target”
(66) “expressed enjoyment”
(9) “engaged with spellings”
(10) “completed spellings”
(14) “read static instruction”
(17) “used F2”
Analysing Qualitative Data: Case Study 1
Relationships between “Concepts” abstracted from players.
Fun
Learning
Control
Exciting
Adventur
e
Fun 10 9 3
Learning 7 5 6
Control 2 4
Exciting 6
Adventure
Uses both Quantitative and Qualitative Approaches
• Numbers can add precision to words and narrative
• Words and images can add meaning to numbers
Mixed Mode Research
e.g. Qualitative research is used to create a theory. Quantitative research is used to test that theory
e.g. Quantitative research could be used to classify behaviour, then qualitative research could investigate the intentions of the people who showed that behaviour.
• Cost Effective
• Familiar to Most People
• Reduce Bias
• Anonymity may increase truthfulness
• Low response rate
• Cannot probe responses
• Respondent is unknown
The Questionnaire
-
+
• Clarify Goals in introductory explanation – How will you use the data?
• Justify the purpose of each individual question•If you aren’t going to use the responses, don’t ask the question
• Keep it short (two pages?)
• Begin with light and interesting items
• Place the most important items in the first half
• Try it out on a small sample of the target population• And refine it for the full study
General Guidance for Surveys
Types of question (1)
• Open questions: response generated entirely by the respondent– e.g. what do you think of chocolate?
• Closed: respondent selects from a list supplied by the researcher– e.g. Do you have access to a computer at
home? Yes/no– e.g. What is the make of the computer that
you use most? Dell / Apple / HP / Toshiba / Denovo / Sony / other (please specify)
Types of question (2)
• Likert scale: the respondent is asked to rank on a five-point scale, usually in terms of agreement/disagreement with a statement– e.g. the graphics enhanced the game playing
experience for me– e.g. the application loaded quickly– e.g. the web page was easy to navigate
• Keep statements positive• Could also be a scale of
always/often/sometimes/rarely/never
Case Study 1: Justification of Questions
1. “I use Computer Software as an aid to delivering literacy in class”
It is important to understand how many teacher do not use software because it would affect the reliability and therefore the validity of the results
5 “The proposed ideas for this EIE is relevant to teaching literacy at KS2”
Are the ideas about content relevant for the target audience?
11 “The EIE theme is gender neutral”
It is important that neither sex are disadvantaged by the chosen theme which would affect the learning outcomes.
Design of Questionnaire presented to the Teachers.
• Define Research Aims
• Identify Population and Sample
• How to collect replies?
• Questionnaire Design (including Pilot Survey)
• Main survey
• Analysis
Workflow
• Ask Open–ended Questions to generate key words
• Consider your mind – map
• Consider your analysis approach t-test, chi-squared, etc.
• Write your closed questions
Questionnaire Design
What do you think are the reasons for football hooliganism?
Do you think football hooliganism is caused by: (tick if appropriate)
Lack of discipline at home
Players’ behaviour on pitch
Family breakdown
Youth unemployment
Poor schooling
Violence on T.V.
Other (please specify)
Question Types – Open vs. Closed
What is your most usual means of travelling to college? (Tick one box only)
Bus
Car
Bike
Which of the following means do you use to travel to college?
Bus
Car
Bike
Single vs. Multiple Response
Place in order of importance to you the following features of a camping holiday (Indicate by numbering from 1-5 in order where 1 is the most important)
Open air
Mobility
Cost
People
Atmosphere
Ranked Response
(Circle the number under the initials that applies. VI=Very important; I=Important; N=Neutral; U=Unimportant; VU=Very Unimportant).
Indicate your view of the following aspects of a camping holiday
Rated Response
VI I N U VU
Community Life 5 4 3 2 1
Low Cost 5 4 3 2 1
Outdoor Life 5 4 3 2 1
Some general rules can be stated on question wording:
• Be concise and unambiguous
• Avoid double questions
• Avoid questions involving negatives
• Ask for precise answers
• Avoid leading questions
Question Wording
Ask for precise answers if you think the information is available and there are no other constraints (e.g. too intrusive on privacy). For example:
Give your age on 1st September 2001 …… years
is preferable to:
Are you…
Under 18 ….18-65 …...Over 65 ……
Precise Answers
- Evokes the truth. Questions must be non-threatening. - Asks for an answer on only one dimension. - Can accommodate all possible answers. - Follows comfortably from the previous question. - Does not presuppose a certain state of affairs - Does not imply a desired answer. (Leading Question)
Question Wording (1)
- Does not use emotionally loaded or vaguely defined words. (e.g., most, least, majority)
- Does not use unfamiliar words or abbreviations. (CPU, GPU)
- Is not dependent on responses to previous questions. Avoid Branching
- Does not ask the respondent to order or rank a series of more than five items
Question Wording (2)
Question Wording (3)
Example Question:What brand of computer do you own? A. IBM PC B. Apple
- Comment: has mutually exclusive options (but not necessarily a comprehensive set) – and doesn’t cope with a respondent owning multiple computers of different types (or no computer at all).
Example Question:Are you against drug abuse? (circle: Yes or No)
Comment: leading question because of the judgemental word ‘abuse’. Also, ‘abuse’ isn’t defined.
Example Question:Where did you grow up? A. country B. farm C. city
Comment: Produces variability of responses – and may be ambiguous. Most farms are in ‘the country’; people may have moved home during childhood; doesn’t cater for ‘small towns’
Question Wording (4)
Example Question:Are you satisfied with your current auto insurance? (Yes or No)
Comment: presupposes that the respondent has an insured car
Example Questions:1. Do you currently have a life insurance policy ? (Yes or No) If
no, go to question 32. How much is your annual life insurance premium ?
Comment: Branching
Question Wording (5)
Example Question:Do you think football and cricket stimulate social cohesion? (Yes or No)
Comment: Asks for response about two different factors
Example Question:Arrange Littleworth’s eight principles of questionnaire design in order of importance. (1 = most important; 8 = least important)
Comment: Doesn’t state the principles (with which respondents may not be familiar); asks for ranking of more than 5 factors
Further reading
• http://www.chnri.org/resources/1.%20Learning%20Resource%20Material/Qualitative%20Research/Presentations/Inro%20to%20qualitative%20methods.pdf
• http://www.engin.umich.edu/teaching/crltengin/engineering-education-research-resources/ryan-and-bernard-techniques-to-identify-themes.pdf
• http://www.sagepub.com/upm-data/24614_01_Saldana_Ch_01.pdf
End