Post on 25-Feb-2016
description
Educational Research 102:
Selecting the Best Study Design for your Research QuestionFrancis S. Nuthalapaty, MD
2010 APGO Faculty Development Seminar
Disclosures
No relevant financial disclosures to declare
Learning Objectives• Describe types of research and study
designs• Understand the characteristics of a
good research question• Understand how to convert the
research question into a hypothesis• Understand how to select the most
appropriate study design to test the hypothesis
What do we already know?Educational Concept True or
False?1. High-fidelity medical simulations facilitate learning.
2. Restriction in resident duty hours improves the quality of patient care.
1. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL and Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher 27, 2, pp 10-28.
2. Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF Jr, Saint S. Systematic review: effects of resident work hours on patient safety. Ann Intern Med. 2004 Dec 7;141(11):851-7.
TRUE
FALSE
Write an educational
research question in which you have
interest
Types of Research
Empirical vs. Non-empirical
Basic vs. Applied
Empirical vs. Non-Empirical Research
Empirical
• Involves collection of data first hand
Non-Empirical
• No first hand data collection
Example: Empirical or Non-empirical?
The Effect of House Staff Working Hours on the Quality of Obstetric
and Gynecologic CareBailit, J et al, Obstet Gynecol 2009
• OBJECTIVE: To measure the effect of house staff working hours reforms on the quality of obstetric and gynecologic care.
Example: Empirical or Non-Empirical?
Features and uses of high-fidelity medical simulations that lead to
effective learning: a BEME systematic review.
Issenberg SB et al, Med Teach 2005
• OBJECTIVE: Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'.
Types of Research
Empirical vs. Non-empirical
Basic vs. Applied
Basic vs. Applied Research
Basic• Results apply to a great
many people and situations
• Results are related to general theory or to a general field of knowledge
• Results need not have immediate or even clear implications
Applied• Results are applicable
only to a specific group of people in a particular situation.
• Results are not necessarily related to a broader field of knowledge
• Results must have immediate and clear implications for practice
Example: Basic or Applied?Assessing Vaginal Surgical Skills
Using Video Motion AnalysisDiwadkar G et al, Obstet Gynecol 2009
• OBJECTIVE: To demonstrate the feasibility of using video motion analysis to quantitate a key step of vaginal hysterectomy and define measurable differences between novice and experienced surgical trainees during vaginal hysterectomy.
Example: Basic or Applied?Effects of a Depression Education Program on Residents’ Knowledge,
Attitudes, and Clinical SkillsLearman L et al, Obstet Gynecol 2003
• OBJECTIVE: To determine whether an interactive educational program would improve obstetrics and gynecology Residents' knowledge, attitudes, confidence, and skills in caring for depressed patients.
Research Methods
What is the difference between Qualitative
Research and Quantitative Research?
Qualitative vs. Quantitative Research
Qualitative
• Descriptive and exploratory focus
• Used to gain insight into attitudes, behaviors, values
• Analysis of unstructured information
• Narrative reporting
Quantitative
• Hypothesis driven
• Used to identify association and/or causation
• Analysis of discrete variables
• Statistical reporting
Quantitative Research• Experimental & Quasi-
experimental• Non-Experimental
–Causal-comparative–Correlational
Quantitative Research• Experimental & Quasi-experimental
– Assess effect of an independent variable on dependent variables
– Comparison of 2 or more groups– Control over ‘treatment’ &
measurement– Randomization– Control group
Quantitative Research Designs
Study Population
Treatment
Control
Randomize
Post-test only Control Group
ExperimentalMeasureme
nt
Measurement
Quantitative Research Designs
Study Population
Treatment
Control
Measurement
Measurement
Randomize
Pre-test/Post-test Control Group
ExperimentalMeasureme
nt
Measurement
Quantitative Research Designs
Study Population
Treatment
Control
Measurement
Measurement
Solomon Four Group
ExperimentalMeasureme
ntMeasureme
nt
Treatment
Measurement
Control Measurement
Randomize
Quantitative Research Designs
Treatment MeasurementStudyPopulation
Quasi-Experimental
One-shotCase Study
Quantitative Research Designs
TreatmentMeasurement MeasurementStudyPopulation
Quasi-Experimental
One-groupPre-test/Post-test
Quantitative Research Designs
Treatment 1 MeasurementStudyGroup 1
Quasi-Experimental
Static Group
Treatment 2 MeasurementStudyGroup 2
Quantitative Research Designs
Treatment 1 MeasurementStudyGroup 1
Quasi-Experimental
Pre-test/Post-test Matched Subjects
Treatment 2 MeasurementStudyGroup 2
Measurement
MeasurementMat
chin
g
Quantitative Research• Non-Experimental
–Causal-comparative–Correlational
Causal-Comparative• Baseline differences are pre-existing
– Non-permutable: Ethnicity, Gender– Permutable: Teaching style
• Determine the cause or consequences of differences
• Associations can be identified• Causality cannot be determined
Quantitative Research Designs
Exposures Outcome
Causal-Comparative
Exploration of Effects
Control Outcome
Quantitative Research Designs
Exposures Outcome
Causal-Comparative
Exploration of Causes
Exposures Control
Quantitative Research Designs
Exposure Outcome
Causal-Comparative
Exploration of Consequences
Outcome
Group 1
Group 2
Correlational Research• Single group of subjects• Describe degree to which 2 or more
quantitative variables are related• Help explain important human
behaviors• Predict likely outcomes• Identifies associations• Causality cannot be determined
Correlational Research
SubjectsObservations
O1 O2
Student A
Student B
Student C
Student D
Student E
Correlational Research• Correlation Coefficient
– Calculated by regression• < .35 = only a slight relationship• .40 - .60 = possible theoretical value• .50 = minimum for crude predictions• >.65 = reasonably accurate predictions• >.85 = close relationship
– Positive correlation = direct association– Negative correlation = inverse
association
Threats to Internal Validity• Subject characteristics• Loss of subjects• Maturation• Repeated measures• Statistical regression• Investigator bias
External Validity• Can the findings from the study be
generalized to larger populations?– Subject characteristics– Environment (lab vs. natural setting)– Did the subjects act differently b/c they
were enrolled in a study (Hawthorne effect)?
What are the steps in the
research process?
The Research ProcessDefine
ResearchQuestion
ConductLiterature
Review
RefineResearchQuestion
DesignStudy
ObtainIRB Approval
CollectAnalyze Data
DisseminateResults
No holein the
literature
Holein the
literature
FINER Research Questions
• F – Feasible• I – Interesting• N – Novel• E – Ethical• R - Relevant
FINER Research Questions• Feasible:
–Can be investigated with available:• Time• Money• Space• Other resources
FINER Research Questions• Interesting:
– Interesting to you– Interesting to others in the field
• Novel:–Will answers to the question
advance knowledge in the field?
FINER Research Questions• Ethical:
– Will any physical or psychological harm come to anyone as a result of the research?• Protect participants from harm• Ensure confidentiality of data• Avoid knowing deception of participants
• Relevant:– How might answers to this question
improve educational practice?
What is a research hypothesis and how is
it different from a research question?
Research Hypothesis
A prediction regardingthe possible outcomes
of the study
Wallen & Fraenkel. Educational Research: A Guide to the Process. 2nd Ed.
Research Hypothesis• A statement which:
–Summarizes the elements of the study• The sample• The design• The predictor and outcome variables
Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
Research Hypothesis• Based upon a FINER research
question• Simple – one difference or
relationship• Specific – clearly stated, defined• Stated in advance – before data
collectionYoder E, MERC Workshop, 2009 AAMC Annual Meeting
Research Hypothesis• May be stated:
– As the alternative hypothesis• Non-directional
– The difference is unknown• Directional hypothesis
– Existing studies/data suggest direction of relationships
– As the null hypothesis
Yoder E, MERC Workshop, 2009 AAMC Annual Meeting
Research Hypothesis• Ha: There is a difference between
groups or variables– Female medical students have a different IQ
than male medical students– μf ≠ μm or μf > μm
• H0: There is no difference between groups or variables– Female medical students have a different IQ
than male medical students– μf = μm Yoder E, MERC Workshop, 2009 AAMC Annual
Meeting
Example: HypothesisImproving Resident Competency in the Management of Shoulder Dystocia With
Simulation TrainingDeering S, et al, Obstet Gynecol 2004
• METHODS: Residents from 2 training programs were randomized by year-group to a training session on shoulder dystocia (SD) management that used an obstetric birthing simulator or to a control group with no specific training. Both groups were subsequently tested on a standardized SD scenario, and a physician grader rated the resident's performance with a standardized evaluation sheet.
Example: Hypothesis• What is the research question?• What is the null hypothesis?• What is the research study design?
– Basic or Applied?– Qualitative or Quantitative?– Experimental or Non-Experimental
Quantitative Research Designs
Study Population
Treatment
Control
Randomize
Post-test only Control Group
ExperimentalMeasureme
nt
Measurement
Small Group Activity
• Share your research questions
• Select one research question to convert to a hypothesis
• Select the most appropriate study design
Give Your Research Impact• Investigate important questions• Connect your study to prior studies
and help build the body of work• Uses appropriate research design• Consult with a biostatistician in the
design phase• Dissemenate your results
Session content, including narrated MS Powerpoint
slides available at:
http://www.obgynknowledgebank.net