Outcomes Research Chapter 5 Cummings 5 th ed. Darshni Vira.
-
Upload
clementine-norris -
Category
Documents
-
view
215 -
download
1
Transcript of Outcomes Research Chapter 5 Cummings 5 th ed. Darshni Vira.
Outcomes Outcomes ResearchResearch
Chapter 5Chapter 5
Cummings 5Cummings 5thth ed. ed.
Darshni ViraDarshni Vira
AKA clinical epidemiologyAKA clinical epidemiology Study of the effectiveness of
treatment in a nonrandomized, real-world setting (observational data)
Outcome measures - survival, costs, physiologic measures, QOL
Study OutlineStudy Outline
Pt presents at baseline with a condition
Receives treatment for that condition
Experiences a response to treatment
Bias and ConfoundersBias and Confounders
Bias - “Compared components are not sufficiently similar” Selection bias Treatment bias
Confounders – “VConfounders – “Variable thought to cause an outcome is actually not responsible because of the unseen effects of another variable age, gender, ethnicity, race, comorbidities
Assessment of Baseline Assessment of Baseline ConditionCondition
Definition of diseaseDefinition of disease Inclusion criteriaInclusion criteria
Disease severityDisease severity TNMTNM Sinusitis – Lund-Mackay, Harvard, etc Sinusitis – Lund-Mackay, Harvard, etc
reproducible resultsreproducible results ComorbidityComorbidity
Adult Comorbidity Evaluation 27 (ACE-27) is a validated instrument for evaluating comorbidity in cancer patients
Assessment of TreatmentAssessment of Treatment
Control GroupsControl Groups
Assessment of OutcomesAssessment of Outcomes
EfficacyEfficacy Health intervention, in a controlled
environment, achieves better outcomes than does placebo
Effectiveness Retains its value under usual clinical
circumstances
Study DesignStudy DesignDesignDesign AdvantagesAdvantages DisadvantagesDisadvantages Level of EvidenceLevel of Evidence
Randomized clinical trial (RCT)
Only design to prove causationUnbiased distribution of confounding
Expensive and complexTypically targets efficacy
1, if high-quality RCT2, if low-quality RCT
Observational (cohort) study
Cheaper than RCTClear temporal directionality from treatment to outcome
Difficult to find suitable controlsConfounding
2, with control group4, if no control group
Case-control study
Cheaper than cohort studyEfficient study of rare diseases or delayed outcomes
Must rely on retrospective dataDirectionality between exposure and outcome unclear
33
Case series Cheap and simple No control groupNo causal link betweentreatment and outcome
44
Expert opinion n/an/a n/an/a 55
Grade of Grade of RecommendationRecommendation
(EBM)(EBM)
Level of EvidenceLevel of Evidence
AA 11
BB 2 or 32 or 3
CC 44
DD 55
Measurement of Clinical Measurement of Clinical OutcomesOutcomes
Psychometric Validation Psychometric Validation (questionnaires)(questionnaires) ReliabilityReliability ValidationValidation ResponsivenessResponsiveness BurdenBurden
Categories of OutcomesCategories of Outcomes
Health StatusHealth Status Individual’s physical, emotional, and
social capabilities and limitations Function
How well an individual is able to perform important roles, tasks, or activities
QOL Central focus is on the value that
individuals place on their health status and function
Examples of Outcome Examples of Outcome MeasuresMeasures
Medical Outcomes Study Short Form-36 (SF-36) European Organization for Research and
Treatment of Cancer Quality of Life Questionnaire (EORTC-HN35)
Hearing Handicap Inventory in the Elderly (HHIE)
Sinonasal Outcome Test (SNOT-20) Child Health Questionnaire (CHQ) Voice Handicap Index Functional Outcomes of Sleep Questionnaire
(FOSQ)
Interpreting Interpreting Medical DataMedical Data
Chapter 6Chapter 6
Cummings 5Cummings 5thth ed. ed.
Habits of a Highly Effective Habits of a Highly Effective Data UserData User
1. Check quality before quantity
2. Describe before you analyze
3. Accept the uncertainty of all data
4. Measure error with the right statistical test
5. Put clinical importance before statistical significance
6. Seek the sample source
7. View science as a cumulative process
1. Check Quality before 1. Check Quality before QuantityQuantity
Experimental vs observational studyExperimental vs observational study BiasBias ConfoundersConfounders Control groupControl group Placebo responsePlacebo response Prospective studies measure incidence
(new events) whereas retrospective studies measure prevalence (existing events)
2. Describe Before You 2. Describe Before You AnalyzeAnalyze
Begins by defining the measurement scale that best suits the observations Categorical (qualitative) Numerical (quantitative)
Bell-shaped curve with standard deviation Median Survival curve
CategoricalCategorical
ScaleScale DefinitionDefinition ExampleExample
Dichotomous two mutually exclusive categories
Breastfeeding (yes/no), sex (male/female)
Nominal unordered qualitative categories
Race, religion, country of origin
Ordinal ordered qualitative categories, but with no natural (numerical) distance between their possible values
Hearing loss (none, mild, moderate), patient satisfaction (low, medium, high), age group
Odds ratio with retrospective reviewOdds ratio with retrospective review Relative risk with prospective reviewRelative risk with prospective review Rate difference with prospective trialsRate difference with prospective trials Correlation coefficient with ordinal or Correlation coefficient with ordinal or
numerical datanumerical data Coefficient (r) from 0 to 0.25 indicates little or
no relationship, from 0.25 to 0.49 a fair relationship, from 0.50 to 0.74 a moderate to good relationship, and greater than 0.75 a good to excellent relationship. A perfect linear relationship would yield a coefficient of 1.00
3. Accept the Uncertainty 3. Accept the Uncertainty in All Datain All Data
Precision (repeatability) Should be reported with a 95% confidence
interval Precision may be increased by using a more
reproducible measure, by increasing the number of observations (sample size), or by decreasing the variability among the observations
Accuracy measures nearness to the truth measured in an unbiased manner and reflect
what is truly purported to be measured
4. Measure Error with the 4. Measure Error with the Right Statistical TestRight Statistical Test
All statistical tests measure errorAll statistical tests measure error Choosing the right test is Choosing the right test is
determined by determined by (1) whether the observations come from independent or related samples, (2) whether the purpose is to compare groups or to associate an outcome with one or more predictor variables, and (3) the measurement scale of the variables
Null hypothesisNull hypothesis Results observed in a study, experiment, or test that are no different from what might have occurred due to chance alone
Statistical test Procedure used to reject or accept a null hypothesis
Type I (alpha) error Rejecting a true null hypothesis (false-positive error); declaring that a difference exists when in fact it does not
P value Probability of making a type I error; P < .05 indicates a statistically significant result that is unlikely to be caused by chance
Type II (beta) error Accepting a false null hypothesis (false-negative error); declaring that a difference does not exist when in fact it does
PowerPower Probability that the null hypothesis will be rejected if it is indeed false; mathematically, power is 1.00 minus type II error
5. Putting Clinical 5. Putting Clinical Importance Before Importance Before
Statistical SignificanceStatistical Significance The next logical question after “Is there a
difference?” (statistical significance) is “How big a difference is there?” (clinical importance)
Effect size reflects the magnitude of difference between groups Measured by correlation coefficient
Confidence intervals (CI) are more appropriate measures of clinical importance than P values, because they reflect both magnitude and precision If “significant” results, the lower limit of the 95% CI should
be scrutinized; a value of minimal clinical importance suggests low precision (inadequate sample size)
If “nonsignificant” results, the upper limit of the 95% CI should be scrutinized; a value indicating a potentially important clinical effect suggests low statistical power (false-negative finding)
6. Seek the Sample 6. Seek the Sample SourceSource
A statistical test is valid only when the study sample is random and representative
Identifying the sampling method and selection criteria (inclusion and exclusion criteria) that were applied to the target population to obtain the study sample
When the process appears sound, one concludes that the results are generalizable
7. View Science as a 7. View Science as a Cumulative ProcessCumulative Process
Process of Integration Process of Integration Systemic Reviews (meta-analysis) Systemic Reviews (meta-analysis) Clinical practice guidelinesClinical practice guidelines
Popular Statistical TestsPopular Statistical Tests T-test - T-test - comparing the means of two independent or
matched (related) samples of numerical data ANOVA - three or more independent groups of
continuous data differ significantly with regard to a single factor (oneway ANOVA) or two factors (two-way ANOVA)
Contingency tables - association between two categorical variables by using the chi-square statistic
Survival analysis (Kaplan-Meier) - estimates the probability of an event based on the total period of observation
Multivariate (regression) - Multivariate (regression) - Examines the simultaneous effect of multiple predictor variables (generally three or more) on an outcome of interest
Statistical DeceptionsStatistical Deceptions Standard error is used
instead of standard deviation
Small sample study results are taken at face value
Post hoc P values are used for statistical inference
Some results are “significant” but there are a large number of P values
Subgroups are compared until statistically significant results are found
No significant difference is found between groups in a small sample study
Significant P values are crafted through improper use of hypothesis tests
The EndThe End