Qualitative Research

TrustworthinessObservation and Interviewing

Content AnalysisEthnography

Features of Qualitative Research (Hoepfl)

• Natural setting as source of data• Researcher acts as human instrument• Inductive data analysis• Reports are descriptive

– Incorporating “voice”• Interpretive

– Aimed at discovering meaning• Pays attention to unique cases• Emergent design• Judged using special criteria of trustworthiness

Trustworthiness in Qualitative Research

• An important check on the trustworthiness of the researcher’s interpretations in qualitative research is to compare one informant’s description of something with another informant’s description of the same thing.

• Triangulation is a check on trustworthiness by comparing different information on the same topic.

• Triangulation– Data triangulation

• Use of multiple data sources– Students, teachers, administrators, etc.

– Methods triangulation• Interviews, observations, etc.

– Researcher triangulation• Use a team of researchers.

Group Discussion

• Name a research topic for qualitative researchers in which it would be useful to have a team of researchers with diverse backgrounds. Explain why.

Criteria for judging research Quantitative

• Internal validity– Did A cause B?

• External Validity– Are these findings

generalizable?• Reliability

– Are the measures repeatable? • Objectivity

– Are the findings free of researcher bias/values?

Qualitative• Credibility

– Believable from participant’s view

• Transferability– Can this finding be transferred

to other contexts?• Dependability

– Would another researcher come to similar conclusions?

• Confirmability– Can the results be confirmed

or corroborated by others

Judging Qualitative Research

• Role of the reviewer– Coherence

• Does the story make sense?– Consensus

• Do others agree?– Instrumental Utility

• Are the results useful?

Observation and Interviewing

Observation• Certain kinds of research questions can best be

answered by observing how people act or how things look.

• Research role– A relationship acquired by and ascribed to the

researcher in interactive data collection.• There are different roles with regard to

observation:1) Interviewer2) Naturalistic Observer3) Participant Observer4) Participant Researcher5) Inside Observer

Variations in Approaches to Observation

A single observation of limited Multiple observations; long-termduration (e.g., 30 minutes). duration (e.g., months, even years).

Narrow focus: Only a single Broad focus: Holistic view of the activity orelement or characteristic is observed. characteristic being observed and all of

its elements is sought.

The purpose of the The purpose of the No explanation is False explanations areobservation is fully explained observation is given to any of the given; participants are

to all involved. explained to some of participants. deceived about thethe participants. purpose of the

observation.

Full-participant Partial Onlooker;observation participation observer is an outsider

Participants know Some but not Participants do not knowthat observations are being all of the that observations are beingmade and they know who is participants made or that there is

making them. know the observer. someone observing them.

Role of the Observer

How the Observer Is Portrayed to Others

How the Purpose of the Observation Is Portrayed to Others

Duration of the Observations

Focus of the Observations

Participant vs. Non-Participant Observation

• Participant observation studies– Researcher participates as an active member of the group.

• Non-participant observation studies– Researcher does not participate in an activity or situation.– Naturalistic observations and simulations.

• Simulations are created situations in which subjects are asked to act out certain roles.

Observer Effect• The presence of an observer can have a considerable effect

on the behavior of those being observed, and affect the outcome of the study.

• Unless a researcher is concealed, it is quite likely that they will have some form of effect upon the individuals being observed.

• It is for that reason that participants should not be informed of the study’s purpose until after data has been collected.– Does this present ethical problems?– How might a researcher reduce his or her impact on the setting?

Observer Bias• Refers to the possibility that certain

characteristics or ideas of observers may bias what they “see”.

• Observer expectations • Comparing notes or impressions among

other researchers assists in reducing this threat.

Coding Observational Data• Coding scheme

– categories an observer uses to record a person’s or group’s behavior.

• Fixed vs. Open• An observer still must choose what to observe, even

with a fixed coding scheme.• Data are coded into categories that emerge as the

analysis proceeds– What type of research is this?

Interviewing

• Interviewing is an important way for a researcher to check the accuracy of the impressions he or she gained through observation.– Likely the most important data-collection technique for

qualitative research.

Types of Interviews

• Types of interviews: Informal-conversation

Questions emerge from the immediate context

Semi-structured Topics selected in advance Researcher determines sequence and wording during

interview.

Standardized open-ended Exact wording and sequence of questions

predetermined.

Types of Interview Questions

• Six types of interview questions:1) Background/demographic questions2) Knowledge questions3) Experience/behavior questions4) Opinion/values questions5) Feelings questions6) Sensory questions

Interviewing Behavior

– Respect the culture of the group being studied

– Respect the individual being interviewed

– Be natural– Develop an

appropriate rapport with the participant

– Ask one question at a time

– Ask the same question in different ways during the interview

– Ask the interviewee to repeat an answer when in doubt

– Vary who controls the flow of communication

– Avoid leading questions

– Don’t interrupt

Group Discussion

• In your opinion, does the flexibility afforded by semi-structured interviewing open the possibility that the interviewer might bias the interview so that the interviewer obtains results along the lines that he or she already expected to find?– Why or why not?– How might the interviewer reduce potential bias?

• Do you think a fully structured interview reduces the likelihood of this problem?– Why or why not?

Content Analysis

What is Content Analysis?• Content Analysis

– a technique that enables researchers to study human behavior through an analysis of communications.

• Types of communications are:• Textbooks• Essays• Pictures• Songs• Movies• Signs

• A person or group’s conscious and unconscious beliefs, attitudes, or values are often revealed in their communication.

Applications of Content Analysis in Educational Research

• Describe trends in schooling over time

• Understand organizational patterns

• Show how different schools handle the same phenomena differently

• Infer attitudes, values, and cultural patterns in different countries

• Compare the myths that people hold about schools

• Gain a sense of how teachers feel about their work

• Gain some idea of how schools are perceived

• Also, can be used to supplement more direct methods of research

Categorization in Content Analysis

• All procedures at some point convert the descriptive information into categories.

• There are two ways this might be done• The researcher determines the categories before any

analysis begins. These categories are based on previous knowledge, theory, and experience.

• The researcher becomes very familiar with the descriptive information collected and allows the categories to emerge as the analysis continues.

– i.e., grounded theory.

Steps Involved in Content Analysis

• Determine objectives by obtaining information on the following:

– Formulate themes for organization– Check other research findings for validation– Obtain information useful in dealing with educational problems– Investigate possible relationships to test hypothesis

• Define Terms– Clearly define terms before or during the study

• Specify the Unit of Analysis

Steps Involved in Content Analysis

• Locate Relevant Data• Develop a Rationale

• Conceptual link needed to relate data to the objectives

• Develop a Sampling Plan• Techniques could involve:

» Random sample» Stratified sampling» Cluster sampling» Convenience sampling

• Formulate Coding Categories

Steps Involved in Content Analysis

• Checking Reliability and Validity• Test-retest method

• Analyze Data• Counting• Use descriptive statistical procedures such as

frequencies and/or percentages

Sample Tally Sheet

Number Number NumberNewspaper of Editorials Subjective of Pro-Abortion of Anti-AbortionID Number Location Circulation Coded Evaluationa Editorials Editorials

101 A 3,000,000 29 3 0 1102 B 675,000 21 3 1 1103 C 425,000 33 4 2 0104 D 1,000,000 40 1 0 8105 E 550,000 34 5 7 0

aCategories within the subjective evaluation: 1 = very conservative; 2 = somewhat conservative; 3 = middle-of-the-road; 4 = moderately liberal; 5 = very liberal.

Advantages of Content Analysis• The following are considered advantages of

Content Analysis:– Unobtrusive– Useful means of analyzing interview and

observational data– Not limited by time and space to the study of

present events– Relatively simple and economical.

Disadvantages of Content Analysis

• The following are considered disadvantages of Content Analysis:– Usually limited to recorded information– Establishing validity

• Question remains as to the true meaning of the categories themselves

– Historical research findings might not be considered important today

– Temptation to attribute a cause of a phenomenon vs. a reflection of it

Discussion

• Let’s say we wanted to test the belief that poor people are inaccurately and stereotypically portrayed in the media. Describe how you would design a content analysis to study the question. What coding categories do you anticipate?

Ethnographic Research

What is Ethnographic Research?• A description and interpretation of a cultural or social

group• Study of the meanings of behavior, language, and

interactions of a culture-sharing group.• Researcher examines the group’s observable and

learned patterns of behavior.• The key tools are in-depth interviewing and participant

observation.

Ethnographic Procedures

• Wolcott (1987) stated that ethnography consists of:

1) Looking for what people do (behaviors).2) Listening for what they say (language).3) What they make and use (artifacts).

The Unique Value of Ethnographic Research

• Ethnographic research has a particular strength that makes it especially appealing to many researchers.– It can reveal nuances and subtleties that other

methodologies miss.– By going out into the world and observing things

as they occur, we are better able to obtain a more accurate picture.

Ethnographic Concepts

• Culture• Holistic Perspective• Contextualization• An Emic Perspective

• Thick Description• Member Checking• A Nonjudgmental

Orientation

Emic vs. Etic• Local Beliefs and Perceptions and the Ethnographer’s• An emic (native-oriented) approach investigates how

natives think, categorize the world, express thoughts, and interpret stimuli.– Emic = “native viewpoint”– Key cultural consultants are essential for

understanding the emic perspective.• An etic (science-oriented) approach emphasizes the

categories, interpretations, and features that the anthropologist considers important.

Topics that Lend Themselves Well to Ethnographic Research

• Topics that defy simple quantification

• Topics that can be best understood in a natural setting

• Topics that involve the study of individual or group activities over time

• Topics that involve the study of the roles and behaviors associated with those roles

• Topics that involve the study of the activities and behavior of groups as a unit

• Topics involving the study of formal organizations in their totality

Sampling in Ethnographic Research

• Ethnographers attempt to observe everything.• However, no researcher can observe everything

at once.• Samples are small and do not permit

generalization to a larger population.• Their goal is the complete understanding of a

particular situation.

Do Ethnographic Researchers Use A Priori Hypotheses?

• Ethnographers seldom initiate their research with precise hypotheses.

• Attempt to understand an ongoing situation or set of activities that cannot be predicted in advance.

• Ethnographic research relies on both observation and interviewing over time.

Challenges (Creswell)

• Ethnographer needs to have grounding in cultural anthropology.

• Time to collect data is extensive.• Narratives written in a storytelling

approach which may limit audience.• Possibility of “going native”

• And?

Advantages and Disadvantages of Ethnographic Research

• Advantages– Provides comprehensive

perspective – Observes behaviors in their

natural environments

– And?

• Disadvantages– Dependent on the

researcher’s observations and interpretations

– Difficult to check the validity of the researcher’s conclusion

– Observer bias is almost impossible to eliminate

– May lack transferability

– And?