Excellence in Qualitative Research Kathleen A. Knafl, PhD, FAAN Associate Dean for Research Frances...
-
Upload
jemima-blair -
Category
Documents
-
view
215 -
download
2
Transcript of Excellence in Qualitative Research Kathleen A. Knafl, PhD, FAAN Associate Dean for Research Frances...
Excellence in Qualitative Research
Kathleen A. Knafl, Kathleen A. Knafl, PhD, FAANPhD, FAAN
Associate Dean for ResearchFrances Hill Fox Distinguished Professor
The University of North Carolina at Chapel Hill
Learning to Evaluate Qualitative Research
Striving for quality as a researcher
Being evaluated – successful/unsuccessful attempts to publish and secure funding
Evaluating others – proposal and manuscript reviewer
Overview of Presentation
Nature and purpose of qualitative research
Position statements and guidelines for evaluating qualitative research
Common expectations and indicators of excellence
Overall Goal
Avoid an overly simplified or overly complex discussion of issues
Present guidelines that address scientific rigor while recognizing the distinct qualities and contributions of qualitative research
“Creativity must be preserved within qualitative research, but not at the expense of the quality of the science” (Maxwell, 1990)
Nature and Purpose of Qualitative Research
Multiple approaches
Terminological jungle Ethnography
Phenomenology
Hermeneutics
Constructivism
Participatory action research
Grounded theory, etc…
Common Characteristics of Qualitative Research
Focus on subjective perspective of respondents
Emphasis on contextual understanding
Design flexibility; emergent design
Generation of narrative data
Researcher as primary data collection instrument
Differences Across Approaches
Purpose
Philosophical underpinnings
Disciplinary roots
Theoretical orientation
Dominant methods for data collection and analysis
Varying Purposes of Qualitative Research
Description
Conceptualization and theoretical understanding
Empowerment and social change
Description (e.g. descriptive phenomenology, ethnography, qualitative description)
“The purpose of this study is to exploreexplore how spousal carers of people with MS interpreted their lived experience lived experience with their partner, the way in they assigned meaningmeaning to their being in such a situation, and the skills and knowledge they have developed to live with their situation” (Cheung & Hocking, 2004, p. 155).
Conceptualization (e.g. grounded theory, concept development)
“The author investigated decision-making experiences of 20 surrogates who assisted terminally ill family members for this grounded theory study. Findings describe a basic social process basic social process of Seeing them Through with Care and Respect” (Meeker, 2004, p.204).
Empowerment and Change(e.g., feminist, participatory action research)
“In this participatory research study, injured injured
workers and academics togetherworkers and academics together participated in
setting the agenda, determining the research
questions and methods, gathering the data, and
interpreting the results. The process helped all
understand more deeply the complicated reality of
being an injured worker within a set of system
and societal norms (Beardwood, et al., 2005, 33-344).
The Challenge of Evaluating Qualitative Research Evaluative stance – Should the same
criteria apply to all research? All qualitative research?
Reasonable expectations – What should we expect to see included in a qualitative report/proposal?
Appropriate criteria – What standards should we use to judge the merits of a qualitative report/proposal?
Guidelines and Criteria
Guidelines for reporting (Tong, et.all, 2007)
Shared standards (Collingridge & Gantt, 2008; Guba & Lincoln, 1985)
Alternative standards (Cheek, 2007; Cohen & Crabtree, 2008; Davies & Dodd, 2002; Fossey, et al., 2002; Whittemore, et al., 2001)
Shared Standards (Lincoln & Guba, 1985)
Truth Value (internal validity/credibilitycredibility)
Applicability (external validity/transferabilitytransferability)
Consistency (reliability/auditabilityauditability)
Neutrality (objectivity/confirmabilityconfirmability)
Qualitative standard Qualitative standard
Credibility (internal validity)
Defined as adequate representation of group or situation studied
Quality of the data
Demonstrated by prolonged engagement, persistent observation, triangulation, external checks, negative case analysis, member checking
Transferability (external validity)
Defined as the degree of similarity between research site/participants and others
Demonstrated by thick description, reporting of information for judgments about transferability
Joint responsibility of researcher and consumer
Dependability (reliability)
Defined as differentiating real perspective/behavior of respondents from reactive effects of research.
Demonstrated by audit of data collection processes Quality of interaction with subjects
External review of data
Confirmability (objectivity)
Defined as the adequacy of the results, interpretations, and recommendations
Demonstrated by audit of analytic procedures Data processing and reduction
Data review and interpretation
External auditor
Alternative Standards (Davies & Dodd)
“Incorporating the notion of ethics, the authors develop a cluster of terms around which they argue that qualitative research can meaningfully speak about rigor: attentiveness, empathy, carefulness, attentiveness, empathy, carefulness, sensitivity, respect, reflection, sensitivity, respect, reflection, conscientiousness, engagement, awareness conscientiousness, engagement, awareness and opennessand openness” (Davies & Dodd, 2002, p. 279)
Common Evaluative Criteria (Cohen & Crabtree, 2008; Tong, et al., 2007; Whittemore, 2001)
Importance of the research
Use of appropriate methods
Coherence of the research
Clarity of the presentation
Ethical conduct of research
Application of Common Criteria to Research Reports/Proposals
Reasonable expectations
Indicators of excellence
Importance of Research
Reasonable expectations General statement of purpose, research
questions, or aims
Review of relevant literature
Importance of Research
Indicators of excellence Evidence of building on prior research;
addressing an important gap
Pragmatically or theoretically useful
Focus on the research problem, not the method
Making the Case for Importance
“The aim of this study was to identify approaches
primary care providers use to engage older men
in depression care. We focus specifically on
older men because previous work has shown
that they suffer from higher levels of under-they suffer from higher levels of under-
treatmenttreatment and that men differ from women in
gender-specific barriers to help seeking”
(Apesoa-Varano, et al., 2010, p. 587)
Use of Appropriate Methods – Study Design
Reasonable expectations Varied approaches
Emergent design
Indicators of excellence Rationale for selection of approach;
fit with study purpose
Citations in support of the approach
Rationale for Study Design
“We chose grounded theory grounded theory as the method for
data collection and analysis. It is a method for
conceptualizing patterns of behavior patterns of behavior in which
people are engaged (Glaser,1978;1998). In
this study, the patterns of behavior are those in
which nurses engage while caring for palliative
cancer patients in hospitals” (Sandgren, et al., 2006,
p. 80).
Use of Appropriate Methods – Role of Theory (Sandelowski, 2003)
Reasonable expectations Variation in the source, function, and temporal
placement of theory
Different qualitative traditions specify different roles of theory
Prevailing (though misleading) belief that qualitative research is atheoretical
Use of Appropriate Methods – Role of Theory
Indicators of excellence Explicit statement of philosophical beliefs,
concepts, or theories that informed design of study
When a concept/theory is an outcome of the research, comparison to related concepts or theories
Example - Use of Theory
“The philosophical framework philosophical framework for this study was based on interpretive phenomenology” (Cheung, 2004, 155).
“The purpose of this study was to examine older African American women’s perceptions of social and environmental stress in relation to their heart disease through the lens of the through the lens of the weathering conceptual frameworkweathering conceptual framework” (Warren-Findlow, 2007, 234).
Sample Size and Selection
Reasonable expectations Purposive approach; non-probability sample
Relatively small sample
Single case to 50+ participants
Likely range of number of participants
Sample Size and Selection
Indicators of excellence Statement of purposive intent (e.g. maximum
variation, theoretical, intensity)
Specification of unit of interest (individual, group, setting)
Rationale for sample size; invoking the principle of saturation
Purposive Sample – Maximum Variation
“The base line criterion for inclusion in the sample was that all parents were employed and had at least one child whom they identified as disabled. The aim was to generate a sample that The aim was to generate a sample that included a range of situations in which parents included a range of situations in which parents combined employment and care. combined employment and care. This was not intended to achieve generalizability, but to enable examination of issues not related to a specific homogenous group” (Lewis, et. al, 2000,
p.1035).
Purposive Sample – Theoretical
“ As data collection progressedAs data collection progressed, I conducted theoretical sampling to provide data needed to describe the categories thoroughly. For example, because early participants were all reporting that other family members had been very supportive, I sought participants who had experienced conflict (Meeker, 2004,
p.208) .
Data Collection – Interviewing and Observation
Reasonable expectations Use of single or multiple data collection
strategies
Description of:• Setting for data collection
• Duration of data collection
• Identification of data collectors
• Description of observational or interview guides
• Method for recording data
Data Collection – Interviewing and Observation
Indicators of excellence Overall explicitness and thoroughness
• Description of role of the investigators; nature of interactions with the participants
• Options for follow-up contact with participants
• Efforts to assure and monitor data quality
• Evidence of sensitivity to incoming data; emergent design
Example – Interviews
Each participant was told, “I’d like you to tell me
the storystory of your experience with a chronic health
condition. Start at the time your symptoms began
and describe the things that happened one after
another regarding your health condition until
today. I encourage details because whatever is
important to you is of interest to me” (Lee & Poole,
2005, 349).
Example - Observation
“I collected data through participant observation in two cardiac rehabilitation programs. In both, I assisted in the day-to-day work of the staff, I assisted in the day-to-day work of the staff, attended education sessions, and conversed attended education sessions, and conversed with participants on an ongoing basiswith participants on an ongoing basis. When I introduced myself to clients, I indicated I was there to learn about heart disease from the participants’ point of view and that I was carrying out research (Wheatley, 2005, p. 440-441).
Data Processing and Analysis
Reasonable expectations Overview of the steps the researcher took to
break the data into smaller units for the purpose of analysis
Methodological citations that support the approach
Variability in use of software programs to support analysis
Data Processing and Analysis
Indicators of excellence Analytic process is consistent with the
qualitative approach • Line by line coding for grounded theory
• Extraction of significant statements for phenomenology
The unit of study is preserved in the analysis
Evidence of thoroughness and checks on quality
Explicitness
Results
Reasonable expectations Variation in organizing structure for
presenting results• Process
• Essential structure
• Thematic description
• Typology
Use of illustrative quotes or vignettes
Variable use of numbers
Results
Indicators of excellence Vividness
• Compelling presentation
• A good read
• Creative
Coherence• Integration of the data; more than a description
of codes and themes
• Convincingly addresses all study aims
Discussion
Reasonable expectations Possibly integrated with results
Linkages to the body of knowledge in the field
Statement of applied and/or theoretical implications of the findings
Limitations and next steps
Discussion
Indicators of excellence Elaboration of pragmatic/theoretical implications
Limitations that address the unique aspects of qualitative research (e.g. non-probability sample not a limitation)
Explicit consideration of transferability
Ethics
Reasonable expectations Statement that IRB approval has been obtained
Indicators of excellence Evidence of sensitivity to human, social, and
cultural contexts
Recognition of ongoing ethical issues and decisions
Overall Coherence and Consistency
Alignment across all aspects of the study
Completeness and follow through
Explicitness; thoroughness of presentation
Creativity – thoughtful, innovative use of methods; insightful linkages
Balancing Rigor and Creativity
“We can preserve or kill the spirit of qualitative work; we can soften our notion of rigor to include playfulness, soulfulness, imagination, and technique we associate with more artistic endeavors, or we can further harden it by the uncritical application of rules. The choice is ours rigor or rigor mortis” (Sandelowski, 1993).
The University of North Carolina at Chapel Hill