September 20, 2012

CONDUCTING A SYSTEMATIC REVIEW:DEFINING A PROBLEM LITERATURE REVIEW

Stages of Systematic Review

1. Define the Problem2. Literature Search3. Data Evaluation4. Data Analysis5. Interpretation of Results6. Presentation of Results

Define the Problem

What research evidence will be relevant to the problem or hypothesis of interest in the review?

• Define variables of interest• Define how variables are related (can be a

hypothesis)• Define search criteria

Define the Problem:Defining Variables

Conceptual vs Operational• Conceptual – describes qualities of the

variable in general or qualitative terms• Example: Intelligence – “a capacity for learning,

reasoning, understanding, and similar forms of mental activity”

• Operational – describes observable events used to measure the variable

• Example: Intelligence – Intelligence Quotient (IQ) score

Defining Variables:Primary research vs systematic review

•Primary research: conceptional and operational definitions must be precisely defined before experiment

•Systematic review: can begin with a broad conceptual definition and known operational measures and further define during course of review

Define the Problem:Relationship between variables

•How do the conceptual and operational definitions fit together in the literature?•Are there multiple operational definitions for one conceptual variable?•Example: intelligence may be measured by many tests (IQ, ACT, SAT, GRE)

Define the Problem:Determine Appropriate Research Designs

1. Should the results of the research be expressed in numbers or narrative?

2. Is the problem or hypothesis seeking to uncover a description of an event, an association between events, or an explanation of an event?

3. Does the problem or hypothesis seek to understand how a process unfolds within an individual unit over time, or what is associated with or explains variation between units or groups of units?

•Descriptive: ask what is happening•Measurement or description of characteristics of an event• Example: reporting of mean IQ score in Tufts freshmen

•Associations: what events happen together•Measurement of a correlation• Example: is IQ score in Tufts freshmen associated with major?

•Explanations: explain why an event happens• Construction of a predictive model• Example: modeling predictive factors in IQ score in Tufts freshmen

Define the Problem:Determine Appropriate Research Designs

Literature SearchWhat procedures should be used to find relevant research?

• Identify sources for research• Identify search terms used

Literature Search:Sources and Search Methods

•Reference databases (pubmed, etc)•Footnote chasing•Consultation with colleagues•Library browsing•Citation searches

Literature Search: Search Terms

•Begin search with broad conceptual definitions as search terms•Want to identify largest number of potentially relevant articles

•Narrow search using operational definitions later• If you find numerous operational definitions in initial search•Which operational definitions to include?

•Example: using term intelligence instead of searching for IQ, SAT, GRE

•Develop exhaustive list of terms used for conceptual definition •Consider different levels of a term: scientific vs lay person•Consider synonyms•Consider spelling in different countries (estrus vs oestrus)•Terms vary by source•Examples?

Literature Search: Search Terms

Logic Models

A systematic, visual representation of a plan

Logic Model for Systematic Review

Input Activities Outputs Outcom

e Impact

Based on definition of problem: VariablesSearch terms

Literature search

Data evaluation

Data analysis

Quantitative or qualitative results of review

Conclusions

Resolve conflict

New information for field

Drive new research

Change in policy

Questions on Articles for Discussion1. How do the authors define the research

problem?2. What variable(s) are defined in the review?a. Conceptual definitionb. Operational definitionc. Effect vs outome

3. Describe the review’s search methodologya. Search terms – exhaustive?b. Sources of information for reviewc. Study eligibility – types of studies included