Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of...
Transcript of Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of...
![Page 1: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/1.jpg)
Systematic reviews of diagnostic test accuracy
Karen R Steingart, MD, MPHMadhukar Pai, MD, PhD
![Page 2: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/2.jpg)
What is diagnostic test accuracy?What is diagnostic test accuracy?
• DiagnosisDiagnosis- Does this patient have this disease at this
point in time?p• Test accuracy
- What proportion of those with the disease p pdoes the test detect? (sensitivity)- What proportion of those without the
disease get negative test results? (specificity)
![Page 3: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/3.jpg)
Test accuracy may not capture clinical impact
di Ruffano, BMJ 2012
![Page 4: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/4.jpg)
Clinical impact of test results on diagnostic and treatment decisions, and eventually, patient outcomes
Test resultsChange in physician’s
Correct treatment
Improved patientTest results physician’s
decisionstreatment choices
patient outcomes
“Improved accuracy is not always a necessary prerequisite for improving patient health, nor does it guarantee other downstream improvements” [di Ruffano et al. BMJ 2012;344:e686]
![Page 5: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/5.jpg)
Accuracy vs Impact:Rapid measurement of B‐type natriuretic peptidein the emergency diagnosis of heart failure
Maisel et al, N Engl J Med. 2002 Jul 18;347(3):
![Page 6: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/6.jpg)
![Page 7: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/7.jpg)
7
![Page 8: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/8.jpg)
![Page 9: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/9.jpg)
Road map for diagnostic accuracy reviews
Avoid simple pooling of sensand spec; need to use HSROC or bivariate random effects models; do not use funnel plots for publication bias
Stata (metandi command) for bivariate random effects pooling + HSROC)
Use PRISMA for reporting of SR
Pai M et al. Evid Based Med 2004;9:101-103
![Page 10: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/10.jpg)
Key steps in a diagnostic test accuracy ireview
1 Framing focused questions1. Framing focused questions2. Searching for studies3 A i t d lit3. Assessing study quality4. Analyzing the data; undertaking meta-
analyses5. Drawing robust conclusions and g
informative presentation of results
![Page 11: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/11.jpg)
1 Framing focused questions1. Framing focused questions
![Page 12: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/12.jpg)
Begin with a well-framed question,
The objectives of the reviewPICO
Population InterventionInterventionComparison
OutcomeOutcome
+ Study design+ Purpose of the test/strategy
+ Reference standard
Richardson et al. ACP Journal Club 1995;A-12
![Page 13: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/13.jpg)
PICO or PPPICPTR for systematic review f di ti t t ?of diagnostic test accuracy?
• Patients, Presentation, Prior tests• Index test, Comparator tests, p• Purpose: comparative question, role of
testtest• Target condition, Reference standard
![Page 14: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/14.jpg)
![Page 15: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/15.jpg)
2. Searching for studies
![Page 16: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/16.jpg)
Sources of studies for diagnostic i
• MEDLINE, EMBASE, the Cochrane Register of Di ti T t A St di ( d
accuracy reviews
Diagnostic Test Accuracy Studies (under development)
• Search related diagnostic test accuracy reviews g y(for example HTA database, DARE etc)
• Check references of relevant studies/reviews• Use a highly sensitive (broad) search strategy• Use a highly sensitive (broad) search strategy• Use a wide variety of search terms, both text words
and database subject headings (MeSH terms)R ti f h filt h ld ll b• Routine use of search filters should generally be avoided
Bossuyt PM Leeflang MM Cochrane Handbook for Systematic Reviews ofBossuyt PM, Leeflang MM. Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4 [updated September 2008]. The Cochrane Collaboration, 2008
![Page 17: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/17.jpg)
+ Influenza rapid tests: Search strategy
Influenza, Human [Mesh]Influenza A virus [Mesh]
Rapid test, rapid diagnos*, rapiddiagnostic test*, point‐of‐caretest*, antigen detection test*, antigen detection, rapid antigen
Influenza B virus [MeshInfluenza
Flugrippe
test*, immunoassay*, immunochromatographic test*Binax NOW, Directigen Flu, FluOIA, QuickVue Influenza, Rapide grippedetection Flu, SAS Influenza, TRU
FLU, XPECT FLU, Zstat flu
Databases: MEDLINE via Pubmed, EMBASE, Biosis et Web of ScienceMarch 2010, updated december 2011, p
Chartrand C et al. Annals of Int Med 2012
![Page 18: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/18.jpg)
Chartrand C et al. Ann Intern Med 2012
![Page 19: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/19.jpg)
The medical literature can be compared to a jungle. It is fast growing, full of deadwood, sprinkled with hidden treasure and infested with spiders and snakes. Morgan. Can Med Assoc J, 134,Jan 15, 19861986
3 Assessing study quality3. Assessing study quality
![Page 20: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/20.jpg)
Sources of bias in diagnostic studies: 3 key issues
• Inclusion of right spectrum of patients• Verification of patientsp
- choice of reference standard- complete verificationp
• Independent assessment of index test and reference standard (blinding)( g)
![Page 21: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/21.jpg)
Effects of study design, A Rutges CMAJ 2006
![Page 22: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/22.jpg)
The Lancet Infect Dis 2003
Case-control studies had a two-fold higher DOR thantwo-fold higher DOR than cross-sectional studies
![Page 23: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/23.jpg)
QUADAS 2003QUADAS, 2003
QUADAS-2, 2011
![Page 24: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/24.jpg)
Suggested displays – QUADAS-2Suggested displays QUADAS 2
http://www.bris.ac.uk/quadas/
![Page 25: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/25.jpg)
In general, diagnostic studies are poorly done and reported (contacting authors is helpful)
25
![Page 26: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/26.jpg)
4 A l i th d t d t ki t l4. Analyzing the data; undertaking meta-analyses
![Page 27: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/27.jpg)
Key stepsKey steps
• Extract TP, FP, FN, and TN to determine pairedExtract TP, FP, FN, and TN to determine paired estimates of sensitivity and specificity
• Visually examine results of individual studiesy• Calculate overall summary estimates using
HSROC/bivariate meta-analysisy• Look for and investigate possible reasons for
heterogeneity
![Page 28: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/28.jpg)
http://ims.cochrane.org/revman
![Page 29: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/29.jpg)
Forest plot – diagnostic test accuracy review o est p ot d ag ost c test accu acy e e
One row is displayed for each studyExtracted data are presented as TP, FP, FN, TNData shown in the graph are also displayed numerically Each study result is given a box for a point estimateHorizontal line = confidence interval
Steingart, PLoS Med 2011
![Page 30: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/30.jpg)
Statistical models for meta-analysis of diagnostic studiesdiagnostic studies
• Simple, separate pooling of sens and spec should not be done
• Two recommended approaches:– hierarchical summary ROC model (HSROC, Gatsonis and Rutter 2001)– bivariate regression of sensitivity and specificity (Bivariate, Reitsma 2005)
![Page 31: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/31.jpg)
![Page 32: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/32.jpg)
+ Influenza rapid tests
Sensitivity: 62.3% (57.9 – 66.6)Specificity: 98.2% (97.5 – 98.7)LR+: 34.5 (23.8 – 45.2)LR : 0 38 (0 34 0 43)LR‐: 0.38 (0.34 – 0.43)
Chartrand C et al. Ann Intern Med 2012
![Page 33: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/33.jpg)
Stata command, metandiStata command, metandi
![Page 34: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/34.jpg)
Stata output
Pooled sensitivity = 80.8% (95% CI 74.3, 86,0)Pooled specificity = 99.3% (95% CI 97.1, 99.8)
![Page 35: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/35.jpg)
Heterogeneity: very common in diagnostic SRs
• Refers to variation in results among studiesstudies
• May be caused by variation in – test thresholds (unique to meta-analyses of
diagnostic tests)l f di– prevalence of disease
– patient spectrumstudy quality– study quality
– chance variation
35
![Page 36: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/36.jpg)
Variation due to threshold differencesVariation due to threshold differences
• Explicit threshold differences– studies have used different cut-off values
to define positive test resultsp• Implicit threshold differences
– differences in observersdifferences in observers– differences in equipmentConsequence: negative correlation arises• Consequence: negative correlation arises between sensitivity and specificity
J Reitsma, Cochrane DTA Workshop, Amsterdam, Sept 2011
![Page 37: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/37.jpg)
Exploring heterogeneityExploring heterogeneity
• Subgroup analysis
• Meta-regression analysis
![Page 38: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/38.jpg)
Example: subgroup analysis
Chartrand C et al. Ann Intern Med 2012
![Page 39: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/39.jpg)
Meta-regression
• Is a form of linear regression in which studies th it f l iare the unit of analysis
• Aims to relate the size of effect to one or more characteristics of the studies involvedcharacteristics of the studies involved
• DOR is the dependent variable • Covariates that might be associated with theCovariates that might be associated with the
variability in DOR are the independent variables
• Tip: Specify covariates that you want to explore in advance
![Page 40: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/40.jpg)
The threshold effect (-0.21) was significant(p = 0 01) This was also seen in the SROC plot(p = 0.01). This was also seen in the SROC plot,
Ling D et al. PLoS ONE 2008.
Determined using ‘Metareg’ command in Stata
![Page 41: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/41.jpg)
Exploration of heterogeneity – urine LAM ELISA for TB
Minion J et al. ERJ 2011
![Page 42: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/42.jpg)
Publication biasPublication bias
• Formal assessment of publication bias using methods such as funnel plots or regression tests is notregression tests is not recommended for diagnostic test accuracy studies
![Page 43: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/43.jpg)
5. Drawing robust conclusions and informative presentation of results
- summary of findings tables
![Page 44: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/44.jpg)
Issues to discussIssues to discuss
• What are the consequences of using the test in terms of the numbers of TP, FP, FN, and TN?
• How applicable are the results?• To what extent were the primary studies biased?
If serious study limitations were identified, could th i t th lt ?these impact the results?
• What were the limitations of the SR itself?Wh t th i li ti f f t h?• What are the implications for future research?
![Page 45: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/45.jpg)
Steingart draft template
![Page 46: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/46.jpg)
Some general limitations of diagnostic SRs
• Literature search strategies are imperfect and studies b i dcan be missed
• Publication bias is always a concern• Poor quality studies or poorly reported studies• Poor quality studies or poorly reported studies• Unexplained heterogeneity• Not enough studies on clinical impact of testsg p• Industry supported studies or COI of study authors• COI of systematic reviewers• Keeping up to date in rapidly evolving fields
![Page 47: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/47.jpg)
Keeping systematic reviews updated!
2004
2007
2008
2012
![Page 48: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/48.jpg)
Lateral flow urine lipoarabinomannan assay for detecting active tuberculosis in HIV-positive adults
Shah et al, The Cochrane Library 2016 Open accessOpen access
http://onlinelibrary.wiley.com/doi/10.1002/14651858.CD011420.pub2/full
![Page 49: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/49.jpg)
Urine lateral-flow lipoarabinomannan (LF-LAM) Background - 1Background 1
• LAM: 17.5 kilodalton structural component of mycobacterial cell walls
• Detectable in urine of patients with TBM t l i f ELISA LAM*• Meta-analysis of ELISA-LAM*
• increased sensitivity in HIV-positive compared with HIV-positive compared with HIVnegative patients
• Pooled sensitivity 56% (40-71%)• Pooled specificity 95% (77-99%)• Higher sensitivity with more
severe immunosuppressionsevere immunosuppression
*Minion et al. ERJ 2011; Image: Se-Ho Park and Albert Bendelac. Nature 406, 788-792
![Page 50: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/50.jpg)
LF-LAM Background – 2
• Urine LF-LAM is a new diagnostic test that may overcome limitations of other approaches– Point of care, lateral flow
format – Fast (< 20 minutes)– No equipment needed– Low cost (~$3.50)– Accuracy has varied across
published studies
![Page 51: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/51.jpg)
Objectives
• To assess the accuracy of LF-LAM for the diagnosis of active TB disease in HIV-positivediagnosis of active TB disease in HIV positive adults who have signs and symptoms suggestive of TB (TB diagnosis).
• To assess the accuracy of LF-LAM as a screening test for active TB disease in HIV-positive adults irrespective of signs and symptoms suggestive of TB (TB i )TB (TB screening).
![Page 52: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/52.jpg)
Selection criteriaSe ect o c te a
• Eligible study types included randomizedEligible study types included randomized controlled trials, cross-sectional studies, and cohort studies that determined LF-LAM accuracy for TB against a microbiological reference standard (culture or nucleic acid amplification t t f b d it )test from any body site).
• 12 studies were identified:6 f TB i– 6 for TB screening
– 6 for TB diagnosis
![Page 53: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/53.jpg)
Data collection and analysisData collection and analysis
• We determined accuracy of LF-LAM. .. We determined accuracy of LF LAM combined with sputum microscopy or Xpert® MTB/RIF In addition we exploredXpert® MTB/RIF. In addition, we explored the influence of CD4 count on the accuracy estimatesaccuracy estimates.
![Page 54: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/54.jpg)
Risk of bias
Shah Cochrane 2016
![Page 55: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/55.jpg)
Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological reference standard.
The studies are ordered by decreasing sensitivityThe studies are ordered by decreasing sensitivity
TP = True Positive; FP = False Positive; FN = False Negative; TN = True Negative.
![Page 56: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/56.jpg)
Analysis Participants (number of
Pooled estimates (95% CrI)(studies)
Sensitivity SpecificityLF-LAM, ALL 2313 (5
studies)45% (29, 63) 92% (80, 97)
Microscopy 1876 (4 40% (27 54) 95% (94 97)Microscopy alone
1876 (4 studies)
40% (27, 54) 95% (94, 97)
LAM alone 1876 (4 38% (34, 42) 98% (93,100)(studies)
( ) ( )
LAM and microscopy*
1876 (4 studies)
59% (47, 70) 92% (73, 97)microscopy* studies)
* Either test positive
![Page 57: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/57.jpg)
Plots of sensitivity and specificity of urine LAM for TB diagnosis stratified by CD4 count
![Page 58: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/58.jpg)
Summary of Findings Table - 1• Question: what is the diagnostic accuracy of
LF-LAM for diagnosing TB in adults living with HIV?
• Participants: HIV-positive adults with symptoms f TBof TB
• Index test: LF-LAM • Role: a replacement test or test in combination
with sputum smear microscopy or sputum Xpert® MTB/RIFXpert® MTB/RIF
• Reference standard: microbiological (mainly mycobacterial culture)mycobacterial culture)
![Page 59: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/59.jpg)
Summary of Findings Table - 2
• Studies: cross-sectionalS tti i ti t d t ti t• Setting: inpatient and outpatient
• Limitations: the main limitations of the review th f l lit fwere the use of a lower quality reference
standard in most included studies, and the small number of studies and participants included innumber of studies and participants included in the analyses
• Pooled sensitivity: 45% (95% CrI: 29 to 63);Pooled sensitivity: 45% (95% CrI: 29 to 63); pooled specificity: 92% (95% CrI: 80 to 97)
![Page 60: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/60.jpg)
Summary of Findings Table - 3
P l d iti it 45% (29 63)Pooled sensitivity: 45% (29, 63)Pooled specificity: 92% (80, 97)
![Page 61: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/61.jpg)
![Page 62: Systematic reviews of diagnostic test accuracy Karen R ... · Shah Cochrane 2016 Forest plots of urine LAM sensitivity and specificity for TB diagnosis measured against a microbiological](https://reader033.fdocuments.us/reader033/viewer/2022050512/5f9cd26c3acae665f241dd68/html5/thumbnails/62.jpg)
References and ToolsReferences and Tools• Cochrane Diagnostic Test Accuracy Working Group
http://srdta.cochrane.org/- Cochrane Handbook for Diagnostic Test Accuracy Reviews- Macaskill P. Analyzing and Presenting Results. Chapter 10. Cochrane Handbook for Diagnostic Test Accuracy ReviewsCochrane Handbook for Diagnostic Test Accuracy Reviews- http://training.cochrane.org/authors/dta-reviews
• Leeflang. Ann Intern Med. 2008;149:889-897• Whiting PF et al. QUADAS-2: a revised tool for the quality
assessment of diagnostic accuracy studies. Ann Intern Med. 2011 Oct 18;155(8):529-36.; ( )
• www.tbevidence.org• RevMan http://ims.cochrane.org/revman