Dichotomous Tests Thomas B. Newman, MD, MPH September 27, 2012 Thanks to Josh Galanter and Michael...
-
Upload
magdalene-phelps -
Category
Documents
-
view
215 -
download
0
Transcript of Dichotomous Tests Thomas B. Newman, MD, MPH September 27, 2012 Thanks to Josh Galanter and Michael...
Dichotomous Tests
Thomas B. Newman, MD, MPH
September 27, 2012
Thanks to Josh Galanter and Michael Shlipak
1
Overview Clarifications, chapter 1, chapter 2
material Definitions: sensitivity, specificity, prior
and posterior probability, predictive value, accuracy
2 x 2 table method Likelihood ratios - WOWO Probability and odds FP/FN confusion Test/treat thresholds 2
Clarifications
EBD errata on book website SLUBI= Self limited undiagnosed
benign illness – not a term I use with parents
3
Definitions: Sensitivity and Specificity
4
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
Sensitivity = A/ (A+C)
Specificity = D/ (B+D)
P.I.D. = Positive in Disease
N.I.H.= Negative in Health
Definitions: Positive and Negative Predictive value, Accuracy
5
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
PPV=A/(A+B)NPV=D/(C+D)
Accuracy = (A + D)/(A + B + C + D) = (A+D)/N Accuracy demonstration: screening for brain tumors
Definitions: Pretest (prior) and post-test (posterior) probability
6
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
Pretest probability =
ONLY IF SAMPLING IS “CROSS-SECTIONAL”!
(A+C)/(A+B+C+D)
Posttest probability = A/(A+B) or C/(C+D)
“Cross-sectional” sampling
7
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
PPV=A/(A+B)NPV=D/(C+D)
Subjects are sampled randomly or consecutively, so that the proportion with disease (pretest probability, prevalence) is clinically meaningful
“Case-control” sampling
8
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
PPV=A/(A+B)NPV=D/(C+D)
Subjects with and without disease are sampled separately
Proportion with disease is determined by investigator
Disease status
Has disease
No disease
Total
Test Result
Positive
A B A + B
Negative
C D C + D
Total A + C B + D A + B + C + D
Prevalence vs Pretest probability
Pretest probability is the more general term
For screening tests, pretest probability = prevalence
For diagnostic tests, pretest probability incorporates history and physical exam items
9
Post-test probability vs. Predictive value Posttest probability after a + test is the
same as positive predictive value Posttest probability after a – test is
1– negative predictive value
10
2 2 Table Method
Research vignette
“Tom, you need to call this mother. She’s really upset.”
11
Choroid Plexus Cyst
12
Fill in table
13
Pretest probability 0.0003 Sensitivity 33% Specificity 98.5%
Disease status
Trisomy 18
No Trisomy 18
Total
ChoroidPlexus Cyst
Present
Absent
Total
Likelihood Ratios
14
Likelihood ratios
A ratio of likelihoods:P(Result|Disease)P(Result|No Disease)
WOWO = With Over WithOut
Pretest odds x LR = Posttest odds (Prior odds x LR = Posterior odds)
15
What Tests Do•Their results change the probability of disease
Negative test Positive test
Reasurance TreatmentOrder a Test
•A good test moves us across action thresholds.
0% 100%
HIV+HIV-
16
Likelihood of Disease Depends on 2 Things
1. Where you started from (low, medium, high risk)
2. Length and direction of the “arrow”
Basic paradigm: – What we thought before test result
what we think now
17
Likelihood ratio Effect of test result
Very small (0.01) Greatly decreases P(disease)
Less than 1 (0.5) Decreases P(disease)
One No effect on P(disease)
More than 1 (2) Increases P(disease)
Very big (100) Greatly increases P(disease)
18
Likelihood Ratios Advantages
– Calculation of post-test probability easier (especially when disease is rare)
– Capture information for multi-level and continuous tests (next week)
Disadvantages– If either pretest or posttest probability is high
(~> 10%) you need to use odds (or a slide rule or calculator)
19
Switch to board
LR for the choroid plexus cyst example– Dichotomous test def of LR
Probability and odds
20
Can Use Slide Rule
21
False-negative confusion
Sensitivity of rapid strep test is 85% Therefore, false negative rate is 15% 15% is too high, so always culture to
confirm negative rapid strep tests
22
What’s wrong?Strep No Strep Total
Rapid Test + TP FP TP+FPRapid Test - FN TN TN+FN
TP+FN FP+TN
2 definitions of “false negative rate”– Def #1: 1-sensitivity = FN/(TP+FN). This one is easier because
it’s (assumed to be) constant.– Def #2: 1 - negative predictive value = FN/(FN+TN). This one is
harder because it depends on prior probability, but it is the one that should determine clinical decisions.
23
If prior probability of strep = 20% and specificity is 98%
False negative rate (def #2) = 15/407 = 3.7% NNC (number needed to culture) = 1/.037 = 27
to identify 1 false negative rapid test. (Pre-test probability of 20%)
At some prior probability of strep, culture after negative quick test is not indicated.
Strep No Strep TotalRapid test + 85 8 95Rapid test - 15 392 407Total 100 400 500
24
25
Sensitivity 85% Specificity 98% Prior probability = 20% Rapid test is NEGATIVE LR =
Try it with slide rule!
Similar examples:
Sensitivity of UA for UTI is only 80%, therefore always culture after a negative UA
Sensitivity of CT scan for subarachnoid hemorrhage is only 90%, therefore always do LP after a negative CT
False positive confusion is similar: 1-specificity vs. 1-positive predictive value
26
Test/Treat Thresholds
No test TreatTest
27
“X-Graph”
28
New “X-Graph”
29
Questions?
30