Volvo vida diagnostic tool--volvo vida software install tips
Teaching Tips for Diagnostic Studies
Transcript of Teaching Tips for Diagnostic Studies
Teaching Tips for Diagnostic Studies
Dr. Annette Plüddemann
Nuffield Department of Primary Care Health SciencesCentre for Evidence-Based Medicine
www.oxford.dec.nihr.ac.uk/horizon-scanning
Typically someone with abnormal symptoms consults aphysician, who will obtain a history of their illness and examine them for signs of diseases.
The physician formulates a hypothesis of likely diagnoses and may or may not order further tests to clarify the diagnosis
Diagnosis
• 2/3 legal claims against GPs in UK
• 40,000-80,000 US hospital deaths
from misdiagnosis per year
• Adverse events, negligence cases,
serious disability more likely to be
related to misdiagnosis than drug
errors
• clinical monitoring (such as failure to act upon test results or monitor patients appropriately) – identified as a problem in 31% of preventable deaths
• diagnosis (such as problems with physical examination or failure to seek a specialist opinion) – identified as a problem in 30% of preventable deaths
• drugs or fluid management – identified as a problem in 21% of preventable deaths
Wolf JA, Moreau J, Akilov O, Patton T, English JC, Ho J, Ferris LK.Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection.JAMA Dermatol. 2013 Jan 16:1-4.
40 yr old woman
non smoker
2 children (10 and 16 yrs)
pre-menopausal
aunt developed breast cancer at 63 yrs
what evidence is needed to inform decision?
1. proportion of women like her who have breast cancer (i.e.
prior to having a mammogram)
2. accuracy of mammograms for detecting cancer
Should she have a mammogram?
Rod Jackson, University of Auckland
What evidence is needed to inform decision?
1. proportion of women like her who have breast cancer (i.e.
prior to having a mammogram)
2. accuracy of mammograms for detecting cancer
40 year-old women:
~ 0.1% will have breast cancer
only ~ 1% of 40 year old women with
a positive mammogram have
breast cancer
test accuracy
= 9 (if positive)
0.1
1
5
10
50
95
%
100%
0%
proportion of population
being tested who have condition
40 year-old women:
~ 0.1% will have breast cancer
✖
Fagan nomogram
0.1
1
5
10
50
95
%
100%
0%
proportion of population
being tested who have condition
40 year-old women:
~ 0.1% will have breast cancer
✖
Fagan nomogram
test accuracy
= 9 (if positive)
✖
100
10
1.5
.05
.005
∞
0
0.1
1
5
10
50
95
%
100%
0%
proportion of population being tested who have
condition
40 year-old women:
~ 0.1% will have breast cancer
✖
test accuracy
= 9 (if positive)
✖
100
10
1.5
.05
.005
∞
0
5
10
1
0.1
50
%
90
0%
100%
✖
only ~ 1% of 40 year old women with a positive mammogram have breast
cancer
proportion of those who test positive /
negative who have the condition
Prevalence/pre-test probability
Likelihood ratio
Post-test probability
0.1
1
5
10
50
95
%
100%
0%40 year-old women:
~ 0.1% will have breast cancer
✖
test accuracy
= 0.1 (if negative)
✖
100
10
1.5
.05
.005
∞
00.01
5
10
50
%
90
0%
100%
1
✖
Only ~ 0.01% of 40 year old women with a
negative mammogram have breast
cancer
Diagnostic strategies and what
tests are used for
How do clinicians make diagnoses?
• Aim: identify types and frequency of diagnostic strategies used in primary care
– 6 GPs collected and recorded strategies used on 300 patients.
(Diagnostic strategies used in primary care. Heneghan, et al,. BMJ 2009. 20;338:b9462009)
• Patient history…examination…differential
diagnosis…final diagnosis
Refinement of the
diagnostic causes
•Restricted Rule Outs
•Stepwise refinement
•Probabilistic reasoning
•Pattern recognition fit
•Clinical Prediction Rule
•Spot diagnoses
•Self-labelling
•Presenting complaint
•Pattern recognition
Initiation of the
diagnosis
Defining the final
diagnosis
•Known Diagnosis
•Further tests ordered
•Test of treatment
•Test of time
•No label
(Heneghan et al, BMJ 2009)
Stage Strategies used
Diagnostic stages & strategies
What are tests used for?
• Increase certainty about presence/absence of disease
• Disease severity
• Monitor clinical course
• Assess prognosis – risk/stage within diagnosis
• Plan treatment e.g., location
• Stall for time!
Bossuyt et al BMJ 2006;332:1089–92
• Replacement – new replaces old
– E.g. CT colonography for barium enema
• Triage – new determines need for old
– E.g. B-natriuretic peptide for echocardiography
• Add-on – new combined with old
– E.g. ECG and myocardial perfusion scan
Roles of new tests
• You are a GP with a busy practice and see a lot of
children with influenza-like symptoms during the winter
• You find it difficult to differentiate influenza from other
RTIs and you’ve heard about a diagnostic test for
influenza called ‘near patient testing’ that can be done in
your office
• You decide to assess the evidence to determine how
accurate this test is to help determine if the test will be
useful in your practice
Clinical scenario
• Patient/ProblemHow would I describe a group of patients similar to mine?
• Index testWhich test am I considering?
• Comparator… or …Reference StandardWhat is the best reference standard to diagnose the target condition?
• Outcome….or….Target conditionWhich condition do I want to rule in or rule out?
Defining the clinical question: PICO or PIRT
• Patient/ProblemChildren (<12 years) presenting with cough and fever thought to be
more than a common cold
• Index testNear patient influenza test
• Comparator… or …Reference StandardLaboratory influenza test
• Outcome….or….Target conditionInfluenza - test positive or negative
Defining the clinical question: PICO or PIRT
Harnden et al. Near patient testing for influenza in children in
primary care: comparison with laboratory test. BMJ 2003;326:480
Critical appraisal of a diagnostic
accuracy study
• Validity of a diagnostic study
• Interpret the results
Diagnostic tests: What you need to knowDiagnostic tests: What you need to know
Series of patients
Index test
Reference standard
Compare the results of the index test with the reference standard,
blinded
Diagnostic Accuracy Studies
Are the results valid?
What are the results?
Will they help me look
after my patients?
•Appropriate spectrum of patients?
•Does everyone get the reference standard?
•Is there an independent, blind or objective
comparison with the reference standard?
Appraising diagnostic studies: 3 easy steps
The Ugly 5….
Biases in Diagnostic Accuracy Studies…
1. Appropriate spectrum of patients?
Ideally, test should be performed on a group of
patients in whom it will be applied in the real
world clinical setting
Spectrum bias:
study uses only highly selected
patients…….perhaps those in
whom you would really suspect
have the diagnosis
Case-control vs consecutive
2. Do all patients have the reference standard?
Ideally all patients get the reference standard test
Verification bias:
only some patients get the reference
standard…..probably the ones in whom
you really suspect have the disease
Series of patients
Index test
Compare the results of the index test with the reference standard,
blinded
Partial Reference Bias
Ref. Std. A
Series of patients
Index test
Ref. Std. A
Blinded cross-classification
Differential Reference Bias
Ref. Std. B
Series of patients
Index test
Reference standard….. includes parts of Index test
Blinded cross-classification
Incorporation Bias
Ideally, the reference standard is independent,
blind and objective
3. Independent, blind or objective comparisonwith the reference standard?
Observer bias:
test is very subjective, or
done by person who knows
something about the
patient or samples
Series of patients
Index test
Reference standard
Unblinded cross-classification
Observer Bias
Lijmer, J. G. et al. JAMA 1999;282:1061-1066
Effect of biases on results
Diagnostic Study Example
1. Spectrum
3. Reference standard
4. Blinding
2. Index test
Teaching tips….
40 yr old woman
non smoker
2 children (10 and 16 yrs)
pre-menopausal
aunt developed breast cancer at 63 yrs
what evidence is needed to inform decision?
1. proportion of women like her who have breast cancer (i.e.
prior to having a mammogram)
2. accuracy of mammograms for detecting cancer
Should she have a mammogram?
Start with a clinical scenario
Get tips from other teachers! (Thank you Rod Jackson!)
Rod Jackson, University of Auckland
0.1
1
5
10
50
95
%
100%
0%
proportion of population
being tested who have condition
40 year-old women:
~ 0.1% will have breast cancer
✖
Fagan nomogram
Introduce the nomogram early –how does evidence help make clinical
decisions
Try something new!
• Validity of a diagnostic study
• Interpret the results
Diagnostic tests: What you need to knowDiagnostic tests: What you need to know
Create a relaxed atmosphere;
Humour
Diagnostic Accuracy Studies
Series of patients
Index test
Reference standard
Compare the results of the index test with the reference standard,
blinded
Series of patients
Index test
Ref. Std. A
Blinded cross-classification
Ref. Std. B
In pictures
The Ugly 5….
Biases in Diagnostic Accuracy Studies…
Think about how to help
people remember
Case-control vs consecutive
Use analogies that are not
medical
Lijmer, J. G. et al. JAMA 1999;282:1061-1066
Effect of biases on results
Show some evidence, but don’t make it too complex
Diagnostic Study Example
Interactive; Use an easy
example!…
If you want to use something which shows potential bias, don’t use a
complex test
The Numbers
Are the results valid?
What are the results?
Will they help me look
after my patients?
•Appropriate spectrum of patients?
•Does everyone get the reference standard?
•Is there an independent, blind or objective
comparison with the gold standard?
Appraising diagnostic tests
•Sensitivity, specificity
•Likelihood ratios
•Positive and Negative Predictive Values
Influenza Study Example
Sensitivity and Specificity
Disease
Test
+ -
+
-
True
positives
False
negatives
True
negatives
False
positives
The 2 by 2 table
Disease
Test
+ -
+
-
Sensitivity = a / a + c
Proportion of people
WITH the disease who
have a positive test result.
a
True
positives
c
False
negatives
The 2 by 2 table: Sensitivity
90
10
Sensitivity = 90/100
So, a test with 90%
sensitivity….means that
the test identifies 90 out
of 100 people WITH the
disease
Disease
Test
+ -
+
-
b
False
positives
d
True
negatives
Specificity = d / b + d
Proportion of people
WITHOUT the disease
who have a negative test
result.
The 2 by 2 table: Specificity
75
25
Specificity = 75/100
So, a test with 75%
specificity will be
NEGATIVE in 75 out of
100 people WITHOUT
the disease
The Influenza Example
Disease: Lab Test
Test: Rapid Test
+ -
+
-
27 3
34 93
30
127
1579661
Sensitivity = 27/61 = 0.44 (44%) Specificity = 93/96 = 0.97 (97%)
There were 96 children who did not have influenza… the rapid test was negative in 93 of them
There were 61 children who had influenza…the rapid test was positive in 27 of them
Ruling In and Ruling Out
High Sensitivity
High Specificity
A good test to help in Ruling Out disease
A good test to help in Ruling In disease
High sensitivity means there are very few false negatives – so if the test comes back negative it’s highly unlikely the person has the disease
High specificity means there are very few false positives – so if the test comes back positive it’s highly likely the person has the disease
Disease
Test
+ -
+
-
a
True
positives
c
False
negatives
b
False
positives
d
True
negatives
Specificity = d/b+dSensitivity = a/a+c
Disease: Influenza
Test: Rapid Test
+ -
+
-
27 3
34 93
Sensitivity = 44% Specificity = 97%
SnNOUT
SpPIN
Predictive Values
Disease
Test
+ -
+
-
a
True
positives
c
False
negatives
Positive and Negative Predictive Value
b
False
positives
d
True
negatives
PPV = Proportion of
people with a positive test
who have the disease.
NPV = Proportion of
people with a negative test
who do not have the
disease.
PPV = a / a + b
NPV = d / c + d
The Influenza Example
Disease: Lab Test
Test: Rapid Test
+ -
+
-
27 3
34 93
30
127
1579661
PPV = 27/30 = 90%
NPV = 93/127 = 73%
Your father went to his doctor and was told that his test for a disease was positive. He is really worried, and comes to ask you for help!
Predictive Value: Natural Frequencies
After doing some reading, you find that for men of his age:
The prevalence of the disease is 30%
The test has sensitivity of 50% and specificity of 90%
“Tell me what’s the chance I have this disease?”
• 100% Likely
• 50% Maybe
• 0% Unlikely
Disease has a prevalence of 30%.
The test has sensitivity of 50% and specificity
of 90%.
Predictive Value
2:001:591:581:571:561:551:541:531:521:511:501:491:481:471:461:451:441:431:421:411:401:391:381:371:361:351:341:331:321:311:301:291:281:271:261:251:241:231:221:211:201:191:181:171:161:151:141:131:121:111:101:091:081:071:061:051:041:031:021:011:000:590:580:570:560:550:540:530:520:510:500:490:480:470:460:450:440:430:420:410:400:390:380:370:360:350:340:330:320:310:300:290:280:270:260:250:240:230:220:210:200:190:180:170:160:150:140:130:120:110:100:090:080:070:060:050:040:030:020:01End
Disease has a prevalence of 30%.
The test has sensitivity of 50% and specificity of 90%.
Given a positive test, what is the probability your dad has the disease
Natural Frequencies
30
70
15
7
100
22 people test positive………
of whom 15 have the disease
So, chance of disease is
15/22 = 68%
Disease +ve
Disease -ve
Testing +ve
Sensitivity = 50%
False positive rate = 10%
Prevalence of 30%, Sensitivity of 50%, Specificity of 90%
4
96
2
9.6
100
11.6 people test positive………
of whom 2 have the disease
So, chance of disease is
2/11.6 = 17%
Disease +ve
Disease -ve
Testing +ve
Sensitivity = 50%
False positive rate = 10%
Prevalence of 4%, Sensitivity of 50%, Specificity of 90%
Positive and Negative Predictive Value
•PPV and NPV are not intrinsic to the test – they also depend on
the prevalence!
•NPV and PPV should only be used if the ratio of the number
of patients in the disease group and the number of patients
in the healthy control group is equivalent to the prevalence
of the diseases in the studied population
•Use Likelihood Ratio - does not depend on prevalence
NOTE
Teaching tips….
Use examples from the news, blogs, things that people come across –relevant to everyone;
Suspense…
Find a simple paper with different measures and the actual numbers
Disease
Test
+ -
+
-
Sensitivity = a / a + c
Proportion of people
WITH the disease who
have a positive test result.
a
True
positives
c
False
negatives
The 2 by 2 table: Sensitivity
90
10
Sensitivity = 90/100
So, a test with 90%
sensitivity….means that
the test identifies 90 out
of 100 people WITH the
disease
Explain the concepts in words. Don’t focus on formulas – some like
them (so provide them), but it is more important to understand what the
measures are.
The Influenza Example
Disease: Lab Test
Test: Rapid Test
+ -
+
-
27 3
34 93
30
127
1579661
Sensitivity = 27/61 = 0.44 (44%) Specificity = 93/96 = 0.97 (97%)
There were 96 children who did not have influenza… the rapid test was negative in 93 of them
There were 61 children who had influenza…the rapid test was positive in 27 of them
Use numbers from a paper; simple
language; It’s more important to
understand what it all means than to know
how to calculate
Ruling In and Ruling Out
High Sensitivity
High Specificity
A good test to help in Ruling Out disease
A good test to help in Ruling In disease
High sensitivity means there are very few false negatives – so if the test comes back negative it’s highly unlikely the person has the disease
High specificity means there are very few false positives – so if the test comes back positive it’s highly likely the person has the disease
Disease
Test
+ -
+
-
a
True
positives
c
False
negatives
b
False
positives
d
True
negatives
Specificity = d/b+dSensitivity = a/a+c
Disease: Influenza
Test: Rapid Test
+ -
+
-
27 3
34 93
Sensitivity = 44% Specificity = 97%
SnNOUT
SpPIN
Acronyms help some…but confuse
others
For beginners this may be a step too
far…
Touch on it…then park it and move
on…
Your father went to his doctor and was told that his test for a disease was positive. He is really worried, and comes to ask you for help!
Predictive Value: Natural Frequencies
After doing some reading, you find that for men of his age:
The prevalence of the disease is 30%
The test has sensitivity of 50% and specificity of 90%
“Tell me what’s the chance I have this disease?”
A simple, common scenario everyone
can relate to
• 100% Likely
• 50% Maybe
• 0% Unlikely
Disease has a prevalence of 30%.
The test has sensitivity of 50% and specificity
of 90%.
Predictive Value
Have a go… interactive… safe
environment
2:001:591:581:571:561:551:541:531:521:511:501:491:481:471:461:451:441:431:421:411:401:391:381:371:361:351:341:331:321:311:301:291:281:271:261:251:241:231:221:211:201:191:181:171:161:151:141:131:121:111:101:091:081:071:061:051:041:031:021:011:000:590:580:570:560:550:540:530:520:510:500:490:480:470:460:450:440:430:420:410:400:390:380:370:360:350:340:330:320:310:300:290:280:270:260:250:240:230:220:210:200:190:180:170:160:150:140:130:120:110:100:090:080:070:060:050:040:030:020:01End
Disease has a prevalence of 30%.
The test has sensitivity of 50% and specificity of 90%.
Given a positive test, what is the probability your dad has the disease
Natural Frequencies
Set a time and stick to it!
30
70
15
7
100
22 people test positive………
of whom 15 have the disease
So, chance of disease is
15/22 = 68%
Disease +ve
Disease -ve
Testing +ve
Sensitivity = 50%
False positive rate = 10%
Prevalence of 30%, Sensitivity of 50%, Specificity of 90%
Simple numbers; reinforces
sensitivity and specificity;
No formulas!
4
96
2
9.6
100
11.6 people test positive………
of whom 2 have the disease
So, chance of disease is
2/11.6 = 17%
Disease +ve
Disease -ve
Testing +ve
Sensitivity = 50%
False positive rate = 10%
Prevalence of 4%, Sensitivity of 50%, Specificity of 90%
Change the prevalence, keep other
numbers the same… learning by doing;Good transition to
likelihood ratios
Likelihood Ratios
Likelihood ratios
LR =Probability of clinical finding in patients with disease
Probability of same finding in patients without disease
Positive likelihood ratio (LR+)
How much more likely is a positive test to be found in a person with the disease than in a person without it?
Likelihood ratios
LR+ = sens/(1-spec)
LR- = (1-sens)/(spec)
Negative likelihood ratio (LR-)
How much more likely is a negative test to be found in a person without the disease than in a person with it?
LR>10 = strong
positive test
result
LR<0.1 = strong
negative test
result
LR=1
No diagnostic
value
What do likelihood ratios mean?
Diagnosis of Appendicitis
McBurney’s point If palpation of the left lower quadrant
of a person's abdomen results in more
pain in the right lower quadrant
Rovsing’s sign
Abdominal pain resulting from
passively extending the thigh of a
patient or asking the patient to actively
flex his thigh at the hip
Psoas sign
Ashdown’s sign
Pain when driving over speed bumps
McGee: Evidence based Physical Diagnosis (Saunders Elsevier)
For Example
(LR+ = 3.4)
(LR- = 0.4)
Speed bump test (Ashdown’s sign): LR+ = 1.4LR- = 0.1
Post test ~20%
?Appendicitis:
McBurney tenderness LR+ = 3.4
Pre test 5%
Fagan nomogram
%
%
Post-test odds = Pre-test odds x Likelihood ratio
Post-test odds for disease after onetest become pre-test odds for next
test etc.
Speed bump test LR- = 0.1
Post test ~0.5%
Teaching tips….
Positive likelihood ratio (LR+)
How much more likely is a positive test to be found in a person with the disease than in a person without it?
Likelihood ratios
LR+ = sens/(1-spec)
LR- = (1-sens)/(spec)
Negative likelihood ratio (LR-)
How much more likely is a negative test to be found in a person without the disease than in a person with it?
Calculation in terms of sensitivity/ specificity is simpler and more useful than formula from the 2x2 table
LR>10 = strong
positive test
result
LR<0.1 = strong
negative test
result
LR=1
No diagnostic
value
What do likelihood ratios mean?
Knowing what LRs mean is more
important than how to calculate
Diagnosis of Appendicitis
McBurney’s point If palpation of the left lower quadrant
of a person's abdomen results in more
pain in the right lower quadrant
Rovsing’s sign
Abdominal pain resulting from
passively extending the thigh of a
patient or asking the patient to actively
flex his thigh at the hip
Psoas sign
Ashdown’s sign
Pain when driving over speed bumps
Simple example…
McGee: Evidence based Physical Diagnosis (Saunders Elsevier)
For Example
(LR+ = 3.4)
(LR- = 0.4)
Speed bump test (Ashdown’s sign): LR+ = 1.4LR- = 0.1
Putting numbers on the scale makes it
clearer
Post test ~20%
?Appendicitis:
McBurney tenderness LR+ = 3.4
Pre test 5%
Fagan nomogram
%
%
Post-test odds = Pre-test odds x Likelihood ratio
Post-test odds for disease after onetest become pre-test odds for next
test etc.
Speed bump test LR- = 0.1
Post test ~0.5%
Bring it back to the Nomogram
What about the news story…?
Dementia Prevalence:1.3% of the entire UK population7% of the UK population over 65
Sensitivity: 90%Specificity: 90%
2:001:591:581:571:561:551:541:531:521:511:501:491:481:471:461:451:441:431:421:411:401:391:381:371:361:351:341:331:321:311:301:291:281:271:261:251:241:231:221:211:201:191:181:171:161:151:141:131:121:111:101:091:081:071:061:051:041:031:021:011:000:590:580:570:560:550:540:530:520:510:500:490:480:470:460:450:440:430:420:410:400:390:380:370:360:350:340:330:320:310:300:290:280:270:260:250:240:230:220:210:200:190:180:170:160:150:140:130:120:110:100:090:080:070:060:050:040:030:020:01End
Dementia has a prevalence of 1%.
The test has sensitivity of 90% and specificity of 90%.
Given a positive test, what is the probability the person has “preclinical” Alzheimer’s?
Natural Frequencies
1
99
0.9
9.9
100
11 people test positive………
of whom 1 has the disease
So, chance of disease is 1/11
= 9%
Disease +ve
Disease -ve
Testing +ve
Sensitivity = 90%
False positive rate = 10%
Prevalence of 1%, Sensitivity of 90%, Specificity of 90%
7
93
6
9
100
15 people test positive………
of whom 6 have the disease
So, chance of disease is 6/15
= 40%
Disease +ve
Disease -ve
Testing +ve
Sensitivity = 90%
False positive rate = 10%
Prevalence of 7%, Sensitivity of 90%, Specificity of 90%Over 65 years:
Pre test 1%
Fagan nomogram
%
%
Alzheimer’s test LR+ = 9Post test ~10%
www.xkcd.com
Are the results valid?
What are the results?
Will they help me look
after my patients?
•Appropriate spectrum of patients?
•Does everyone get the gold standard?
•Is there an independent, blind or
objective comparison with the gold
standard?
Appraising diagnostic tests
•Sensitivity, specificity
•Likelihood ratios
•Positive and Negative Predictive Values
•Can I do the test in my setting?
•Do results apply to the mix of patients I see?
•Will the result change my management?
•Costs to patient/health service?
• Reproducibility of the test and interpretation in my setting
• Do results apply to the mix of patients I see?
• Will the results change my management?
• Impact on outcomes that are important to patients?
• Where does the test fit into the diagnostic strategy?
• Costs to patient/health service?
Will the test apply in my setting?
Are the results valid?
What are the results?
Will they help me look
after my patients?
What is the ONE thing I need to remember from today?
Don’t believe everything you are told,
Ask for the Evidence!
Teaching tips….
Bring it back to the news story, reinforce…
• Reproducibility of the test and interpretation in my setting
• Do results apply to the mix of patients I see?
• Will the results change my management?
• Impact on outcomes that are important to patients?
• Where does the test fit into the diagnostic strategy?
• Costs to patient/health service?
Will the test apply in my setting?
There is more to diagnostics than
accuracy!
Are the results valid?
What are the results?
Will they help me look
after my patients?
What is the ONE thing I need to remember from today?
Don’t believe everything you are told,
Ask for the Evidence!
Take home message!
The Diagnostic Process. John Balla. Cambridge Univ. Press
Diagnostic Tests Toolkit. Thompson & Van den Bruel. Wiley-Blackwell.
Evidence base of Clinical Diagnosis. Knottnerus & Buntinx. Wiley-Blackwell
Evidence based Physical Diagnosis. Steven McGee. Saunders
Evidence-based Diagnosis.Newman & Kohn. Cambridge Univ. Press
Useful books on diagnostics
• Bossuyt. Additional patient outcomes and pathways in evaluations of testing.
Med Decis Making 2009
• Heneghan et al. Diagnostic strategies used in primary care. BMJ 2009
• Ferrante di Ruffano. Assessing the value of diagnostic tests: a framework for
designing and evaluating trials. BMJ 2012
• Mallett et al. Interpreting diagnostic accuracy studies for patient care. BMJ 2012
• Bossuyt et al. STARD initiative. Ann Int Med 2003
• Lord et al. Using priniciples of RCT design to guide test evaluation. Med Decis
Making 2009
• Rutjes et al. Evidence of bias and variation in diagnostic accuracy studies.
CMAJ 2006
• Lijmer et al. Proposals for phased evaluation of medical tests. Med Decis
Making 2009
• Whiting et al. QUADAS-2: revised tool for quality assessment of diagnostic
accuracy studies. Ann Int Med 2011
• Halligan S, Altman DG, Mallett S. Disadvantages of using the area under the
receiver operating characteristic curve to assess imaging tests: A discussion and
proposal for an alternative approach. Eur Radiol. 2015
Useful journal articles on diagnostics