ANTIMICROBIAL SUSCEPTIBILITY TESTING:...
Transcript of ANTIMICROBIAL SUSCEPTIBILITY TESTING:...
ANTIMICROBIAL
SUSCEPTIBILITY
TESTING: ADVANCED
Romney Humphries, Ph.D., DABMM
Section Chief, Clinical Microbiology
University of California – Los Angeles
Lost Angeles, CA
Susan E. Sharp, Ph.D., DABMM, FAAM
Director, Airport Way Regional Laboratory
Director, Regional Microbiology
Kaiser Permanente
Portland, OR
1. Examine approached to verification studies for
antimicrobial susceptibility test systems.
2. Evaluate approaches to cascade reporting.
3. Discuss the development of an antibiogram.
4. Review the principles of pharmacokinetics and
pharmacodynamics (PK/PD) in relation to how
these apply to MIC interpretive criteria.
OBJECTIVES
Romney Humphries
UCLA Clinical
Microbiology
Los Angeles CA
STRATEGIES FOR VERIFYING
ANTIMICROBIAL
SUSCEPTIBILITY
TESTS AND TEST SYSTEMS
List antimicrobial susceptibility tests (AST) and test systems that are available for use, including their FDA status.
Describe strategies for verifying antimicrobial susceptibility tests and test systems.
Discuss criteria used to determine if verification is successful and options for documenting the verification process.
WHAT WE WILL COVER:
Disk diffusion (Kirby Bauer)
CLSI reference method
Automated zone reader (bioMIC)
Broth microdilution MIC CLSI reference method
Commercial systems
MicroScan (Siemens)
Sensititre (Trek)
AST METHODS/SYSTEMS
Etest
Automated / semi-automated systems MicroScan (Siemens)
Phoenix – (Becton Dickinson)
Sensititre (Trek)
Vitek (bioMerieux)
AST METHODS/SYSTEMS
(CON’T)
WHY DO AST DEVICES REQUIRE
FDA CLEARANCE?
To prove they are
safe and
effective and
pose minimum
risks to patients
Identified Risk…
Administration of
an inappropriate
antimicrobial
agent to a patient
Manufacturer submitted 510k to FDA
Manufacturer demonstrated that results with their AST device are comparable to those generated from a CLSI REFERENCE method.
FDA specifies the data that must be submitted and defines criteria for acceptability.
WHAT DOES IT MEAN FOR A COMMERCIAL
AST DEVICE TO BE FDA CLEARED?
WHAT DATA ARE REQUIRED BY FDA?
No. sites (including 1 in-house) 3
Organisms Fresh or stock (clinical) 100/site
Challenge * 75/one site
Reproducibility 25/site or 10x3x3/site
Interpretive criteria / standards FDA
Stability (3 lots) Real time
QC CLSI strains 20 results/site
Manufacturer’s strains Optional
On-scale At least 1
Inoculum density check QC/reproducibility/fresh
CLSI reference method MIC
http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocu
ments/ucm080564.htm * MICs near “R” breakpoint
WHAT CRITERIA ARE USED TO
DETERMINE IF AST DEVICE RESULTS
ARE ACCEPTABLE?
Criteria Defined as… Acceptable
limits
Essential
Agreement
(EA)
MIC within +/- 1
doubling dilution
of the REF MIC
>89.9 %
Category
Agreement
(CA)
S, I, and R results
agree
>89.9 %
WHAT CRITERIA ARE USED TO
DETERMINE IF AST DEVICE RESULTS ARE
ACCEPTABLE? (CON’T)
Error
Results Acceptable Error
Rate REF AST device
Very major R1 S Based on # of R
orgs tested
Major S2 R ≤ 3 %
Minor S I
Not specified R I
I S
I R
1 only isolates R by REF included in calculating error rate 2 only isolates S by REF included in calculating error rate
Essential agreement (EA) =
# within +/- 1 two-fold dilution of REF MIC
Total # isolates tested
Category agreement (CA) =
# with same S, I, or R result as REF MIC
Total # isolates tested
CALCULATING EA / CA
X
100
X
100
Very major error (VME) =
No. with VME (false R)
Total # “R” isolates tested
Major error (ME) =
No. with ME (false S)
Total # “S” isolates tested
Minor error (mE) =
No. with mE
Total # isolates tested
CALCULATING % ERRORS
X 100
X 100
X 100
Careful!
Data in
literature
sometimes
uses total
# tested
EXAMPLE: CEFAZOLIN
Isolate REF TEST EA CA Error?
1 8 S 8 S yes yes no
2 8 S 2 S no yes no
3 16 I 8 S yes no minor
4 8 S 32 R no no major
5 32 R 8 S no no very major
Breakpoints (µg/ml)
S I R
≤8 16 32
CALCULATING EA / CA
EXAMPLE: CEFAZOLIN
Organism N EA CA
# % # %
E. coli 100 91 91 94 94
C. freundii 94 89 94.6 84 89.41
1unacceptable re: FDA criteria
CALCULATING % ERRORS
EXAMPLE: CEFAZOLIN
Organism N REF
Result
Very
major
major minor
# S # R # % # % # %
E. coli 100 60 30 1 3.3 2 3.3 3 3
C. freundii 94 49 45 0 0 1 2.0 9 9.5
Intended Use
Reagents
Reporting of results
Performance
characteristics
QC
Limitations
Other
PACKAGE INSERT OR “LABELING”
SECTIONS FOR AST DEVICE
Research use only:
For Research Use Only. “Not for use in diagnostic procedures” must be in labeling
Devices are in research phase of development
Investigational use only:
“For Investigational Use Only. The performance characteristics of this product have not been established” must be in labeling
Products being tested or evaluated prior to regular marketing
Reporting:
Report should indicate that test is not FDA-cleared (RUO)
RUO AND IUO – AST DEVICE
You acquire a NEW automated instrument for AST
What should you do to make certain the AST system will produce
accurate and reproducible results?
SCENARIO:
CLIA
493.1253
Verification –
initial assessment of
NEW AST system
Each lab that introduces an unmodified, FDA-cleared or approved test system must do the following before reporting patient test results: Demonstrate that it can obtain performance
specifications comparable to those established by the manufacturer for the following performance characteristics: Accuracy
Precision (reproducibility)
Reportable range of test results for the test system. Verify that the manufacturer's reference intervals (normal values) are appropriate for the laboratory's patient population.
CLIA 493.1253 ESTABLISHMENT AND
VERIFICATION OF PERFORMANCE
SPECIFICATIONS
Accuracy
Precision (reproducibility)
USER MUST DEMONSTRATE THESE PERFORMANCE
SPECIFICATIONS WHEN USING AN FDA-CLEARED TEST
BEFORE USE FOR PATIENT TESTING…
CLIA 493.1253
Each lab that modifies an FDA-cleared or approved test system, or introduces a test system not subject to FDA clearance or approval (including methods developed in-house), or uses a test system in which performance specifications are not provided by the manufacturer must, before reporting patient test results: Establish for each test system the performance specifications
for the following performance characteristics, as applicable: Accuracy Precision Analytical sensitivity Analytical specificity to include interfering substances
Reportable range of test results for the test system. Reference intervals (normal values). Any other performance characteristic required for test
performance.
CLIA 493.1253 ESTABLISHMENT AND
VERIFICATION OF PERFORMANCE
SPECIFICATIONS
Accuracy
Precision (reproducibility)….plus…
Analytical sensitivity (AST, generally ability to
detect “R”)
Analytical specificity (AST, generally ability not to
call false “R”)
Reportable range of patient test results
Reference range(s)
Any characteristic required for test performance
or interpretation of results
USER MUST DEMONSTRATE THESE PERFORMANCE SPECIFICATIONS WHEN USING A TEST THAT IS NOT FDA-CLEARED OR USING A MODIFICATION OF AN FDA -CLEARED TEST BEFORE USE FOR PATIENT TESTING…
CLIA 493.1253 ..,..some do not apply to AST!
USAGE OF TERMS
Initial in-house
assessment of NEW
test
Ongoing evaluation
of IN-USE test
CLIA Verification Validation
CAP Verification Ongoing evaluation
JCAHO Validation Verification
Terms
used in
this talk
Verification Ongoing evaluation
Verification (initial assessment)
Accuracy Reproducibility(P
recision)
Run to run - ATCC strains
Within run - 5 bugs x 3 x 3d
Compare NEW w/ REF method
Manufacturer’s information
Literature
Anecdotal information
In-house studies
Test minimum of 100 isolates
FDA-cleared AST
…Prior to patient testing
Ongoing evaluation
QC e.g., CLSI
ATCC
Competency
PT
Correlation with clinical findings
Equipment PM
Other
FDA-cleared AST
...During patient
testing
What method should we use as REF?
How many bugs?
What kind of bugs? resistance profiles?
What is acceptable?
What if results are not acceptable?
How should we document?
WHAT DO WE HAVE TO CONSIDER WHEN
ESTABLISHING “ACCURACY” FOR USE OF A NEW
AST DEVICE IN OUR LABORATORY?
CLSI broth microdilution or agar dilution reference method
Commercial FDA-cleared broth microdilution method (historically acceptable as REF)
OLD method (that was previously verified)
WHAT METHODS SHOULD WE USE AS
REF?
Accuracy
Studies
Minimum 30; 100 preferred, more if possible
Selection criteria
Represent clinical isolates tested
Variety of susceptibility profiles
Some around breakpoints
Use organism mix on next few slides as guide
Note: some bugs fit multiple “R” criteria
HOW MANY ORGANISMS SHOULD WE
TEST?
Accuracy
Studies
Routine QC strains (CLSI)
Clinical isolates isolated in-house or from other laboratories Fresh
Stock
Resistant and susceptible isolates for each antimicrobial agent
Old PT samples
Organisms provided by manufacturer
WHERE DO WE OBTAIN
ISOLATES?
Accuracy
Studies
EXAMPLE: ORGANISM MIX (N=75)
STAPH/ENTEROCOCCUS PANEL (CON’T)
No
.
Species “R” characteristics R
(pos)
S
(neg)
5 E. faecalis vancomycin 2 3
4 E. faecalis HLAR (Gm) 2 2
4 E. faecalis HLAR (Str) 2 2
6 E. faecium vancomycin 3 3
4 E. faecium HLAR (Gm) 2 2
4 E. faecium HLAR (Str) 2 2
4 Motile
enterococci
low level
vancomycin
2 2
EXAMPLE: ORGANISM MIX (N=75)
STAPH/ENTEROCOCCUS PANEL (CON’T)
No
.
Species “R” characteristics R (pos) S (neg)
20 S. aureus mecA 10 10
2 S. aureus β-lactamase - 2
6 S. aureus inducible clindamycin 3 3
10 CoNS mecA 5 5
6 Other drug dependent*
*e.g., if rifampin reported on S. aureus, test at least 1 R and 1 S
…also
Daptomycin-NS MRSA and E. faecalis
Linezolid-R MRSA, CoNS, enterococcus
EXAMPLE: ORGANISM MIX (N=126)
ENTEROBACTERIACEAE PANEL
No. Species “R” characteristics R
(pos)
S
(neg)
8 E. coli ampicillin 4 4
8 E. coli ESBL 4 4
4 E. coli ESBL scrn +
confirm-
4
8 E. coli gentamicin 4 4
8 E. coli fluoroquinolone 4 4
8 E. coli nitrofurantoin 4 4
EXAMPLE: ORGANISM MIX (N=126)
ENTEROBACTERIACEAE PANEL
No. Species “R” characteristics R
(pos)
S
(neg)
8 C. freundii broad spec β-lac 4 4
8 E. aerogenes broad spec β-lac 4 4
8 E. cloacae broad spec β-lac 4 4
8 E. cloacae gentamicin 4 4
8 M. morganii broad spec β-lac 4 4
8 Providencia spp. broad spec β-lac 4 4
8 S. marcescens broad spec β-lac 4 4
Broad spec β-lacs = one or more from cefotaxime, ceftriaxone,
ceftazidime, cefepime, piperacillin-tazo, carbapenem
EXAMPLE: ORGANISM MIX (N=126)
ENTEROBACTERIACEAE PANEL (CON’T)
No. Species “R” characteristics R
(pos)
S
(neg)
1 K. pneumoniae KPC 1
8 Klebsiella spp. ESBL 4 4
8 P. mirabilis fluoroquinolone 4 4
1 P. mirabilis ESBL 1
4 Salmonella spp fluoroquinolone 1 3
4 Shigella any
…also
additional CRE
Low level FQ R Salmonella (if can be tested on panel)
Tigecycline-R Enterobacteriaceae
EXAMPLE: ORGANISM MIX (N=83)
NON-FERMENTER PANEL
No. Species “R” characteristic R
(pos)
S (neg)
10 P. aeruginosa R- Gm, Tob (S- Amk) 5 5
5 P. aeruginosa R- Gm, Tob, Amk 5
10 P. aeruginosa imipenem 5 5
6 P. aeruginosa piperacillin-tazobactam 3 3
5 P. aeruginosa fluoroquinolone 3 2
10 P. aeruginosa
(mucoid)
if CF patients are seen at
facility
10 Pseudomonas sp.
(not aeruginosa)
any
EXAMPLE: ORGANISM MIX (N=78)
NON-FERMENTER PANEL (CON’T)
No. Species “R” characteristic R (pos) S (neg)
5 Steno. maltophilia trimethoprim-sulfa 1 4
10 Acin. baumannii imipenem = R 5 5
4 Achrom.
xylosoxidans
any
4 Burkholderia sp. any
4 Aeromonas sp. any
VALIDATION DATA - ACCURACY
How many bugs?
What kind of bugs? resistance profiles?
Routine QC strains?
How many replicates? test days?
What is acceptable? ?
What if results are not acceptable? ?
How should we document ??
WHAT DO WE HAVE TO CONSIDER WHEN
ESTABLISHING “REPRODUCIBILIT Y” FOR USE OF A
NEW AST DEVICE IN OUR LABORATORY?
Reproducibility Studies
Routine CLSI ATCC QC strains
Data from routine testing (run to run
reproducibility)
Select 5 clinical isolates (fresh or stock) that
meet as many of the following as possible On-scale endpoints
Some resistance
Mix of high and low MICs for a given drug
Good growth characteristics
Test in triplicate on 3 days (within run and run to run
reproducibility)
Acceptable = 95% within +/- 1 dilution
PRACTICAL STRATEGY FOR
“REPRODUCIBILITY” STUDIES
Reproducibility Studies
SCENARIO #1:
YOUR NEW AST DEVICE* GENERATES THE
FOLLOWING FOR E. COLI AND
GENTAMICIN. WHAT WILL YOU DO?
N
REF Result
EA
CA
Very
major
major minor
# S #I #R # % # % # %
30 23 2 5 83 86.
6
3 60 0 0 1 3.3
*OLD system = commercial broth microdilution
Repeat NEW and REF
1 VME persists
Send to reference lab
(CLSI MIC method) Test more R bugs?
Consider:
Could isolates be clonal?
Any unique growth characteristics?
1 VME persists Agree with NEW
No VME
NOT validated
Stop
Stop
SCENARIO #2:
YOUR NEW AST DEVICE* GENERATES THE FOLLOWING
FOR P. AERUGINOSA AND PIPERACILLIN-TAZOBACTAM.
WHAT WILL YOU DO?
N
REF Result
EA
CA
Very
major
major minor
# S #I #R # % # % # %
25 15 - 10 961 80 1 10 4 26.6 NA NA
1 MIC for only 1 isolate >1 dilution from REF MIC
*OLD system = commercial broth microdilution
SCENARIO #3:
YOUR NEW DEVICE* GENERATES THE
FOLLOWING FOR S. AUREUS AND
OXACILLIN. WILL YOU USE A CEFOXITIN
BACKUP TEST?
N
REF Result
EA
CA
Very
major
major minor
# S #I #R # % # % # %
30 12 - 18 93.3 96.6 0 0 1 8.3 NA NA
*OLD system = cefoxitin disk diffusion
Most widely accepted practice:
Use validation data for panel with “cleared” organisms
Qualify results
SCENARIO #4: YOU WANT TO TEST PASTEURELLA SPP. WITH YOUR COMMERCIAL BROTH MICRODILUTION SYSTEM (PANELS WITH 2.5-5% LYSED HORSE BLOOD) THAT IS NOT FDA-CLEARED FOR TESTING THIS ORGANISM. WHAT SHOULD YOU DO?
amoxicillin 0.5 S
ceftriaxone 0.12 S
doxycycline 0.5 S
levofloxacin 0.06 S
penicillin 0.5 S
SPECIMEN: BLOOD DIAGNOSIS: LEUKEMIA PASTEURELLA MULTOCIDA
MIC (µg/ml)
“Susceptibility testing performed per Dr. Jones request;
Infectious Diseases consult suggested; testing performed
using a system FDA cleared for other similar organisms but
NOT for P. multocida”
No performance specifications from the manufacturer
Lab questioned request and contacted AST “experts”
“Experts” agreed not practical / necessary
One strategy for verifying D zone test: 1. Test CLSI QA strains (S. aureus ATCC BAA-976, BAA-
977)
2. Test 5 staphylococci with known inducible clindamycin resistance and 5 without
3. Place disks at various distances to ensure placement doesn’t cause false S or false R - use ATCC strains in 1)
SCENARIO #5: CLIA SURVEYOR ASKED LAB FOR SENSITIVITY/
SPECIFICITY DATA FOR “D ZONE TEST”
WHAT ERROR RATE IS ACCEPTABLE?
(N≥30)
Error Type
Acceptable
Error Rate
Comment
Very major 1 0
Major < 5 %
Major + Minor 2
≤ 10 %
Combined major and
minor
Essential agreement ≥ 90 %
Categorical agreement ≥ 90 %
Cumitech 31A. ASM Press,
2009.
Reference system not used as comparator
WHAT ERROR RATE IS ACCEPTABLE?
(N≥100)
Error Type
Acceptable
Error Rate
Comment
Very major 1 ≤ 3 % Minimum 35 R isolates
Major ≤ 3 %
Major + Minor 2
≤ 7 %
Combined major and
minor
Essential agreement ≥ 90 %
Categorical agreement ≥ 90 %
Cumitech 31A. ASM Press, 2009.
Arbitrate discrepancies with a Reference system
50% should exhibit some type of “R”
If significant numbers of isolates near breakpoint, CA
my be <90%
“But, do not interpret recommendations
rigidly; consider degree of difficulty
involved in detecting resistance in some
organisms.”
EXAMPLES OF ERRORS AND STRATEGIES FOR DEALING WITH THEM…..
Error Optional Strategy
5/25 (20%) VME ceftazidime / P.
aeruginosa
Unacceptable – use alternate
method
5/25 (20%) ME nitrofurantoin / C.
freundii
Use alternate method for OP urine
C. freundii “R” isolates
16/20 (80%) EA cefazolin / K.
pneumoniae (all errors in “S” range
erring on higher MIC with NEW test)
accept
4/16 (25%) ME pip-tazo / P. aeruginosa Accept...only S and R (no “I”)
breakpoints; educate ID/pharm
about MICs close to breakpoints
8/10 (80%) CA for tobramycin / P.
aeruginosa
Test additional isolates
EXAMPLES OF ERRORS AND STRATEGIES FOR DEALING WITH THEM…..
Error Optional Strategy
8/16 (50%) ME clindamycin / CoNS Do not report…offer test as
“MD request only”..use
alternate method
3/10 (30%) ME for ESBL
confirmatory test for K.
pneumoniae
Use alternate method; check if
isolates might be clonal; check
with other users
2/14 (14.3 %) VME for HLAR
gentamicin and E. faecalis
Use alternate method (sterile
sites; not generally needed
from other sites); check other
Enterococcus spp. carefully
2/16 (12.5%) VME for cefazolin and
E. cloacae
Override all cefazolin-S or –I
results to “R”
EXAMPLE: ENTEROBACTERIACEAE PANEL
Drug Post Validation Comments
Amikacin Failed verification for Serratia spp. Use alternate test
when needed
Ampicillin Edit E. cloacae to “R”
Cefazolin OK
Ceftriaxone OK
Ciprofloxacin OK
Gentamicin OK
Imipenem Only 1 “R” isolate in validation; verify all “R” patient
isolates by alternate method; test more “R”
Nitrofurantoin Failed validation for C. freundii (false R); use alternate
test for “R” urine isolates
Piperacillin-tazo OK
Tobramycin OK
Trimethoprim-sulfa OK
QUESTIONS RE: VERIFICATION
What if ??? Verify performance by….
Company rep does
validation
Showing this correlates with
performance by lab staff.
We get a loaner instrument
for 2 weeks
Testing QC strains (daily) and 25 or
so PT samples and/or previously
tested patient specimens
We relocate instrument Following manufacturer’s package
insert re: critical requirements (setup,
environment, etc.)
Use common sense and
document, document, document!!!
Documentation that a test which has been validated is
repeatedly giving the expected results
Most commonly involves following CLSI QC procedures
Specified by CLIA, CAP, JCAHO
Other components….
ONGOING EVALUATION
Ongoing evaluation
QC e.g., CLSI ATCC
Competency
PT
Correlation with clinical findings
Equipment PM
Other
FDA-cleared AST
During patient
testing
CLIA – sections 493.1253
CAP - MIC.21040, GEN.42020 – GEN.42160
Joint Commission – Quality control sections QC.1.20 –
QC.1.150
WHERE DO WE FIND INFORMATION ON LEGAL
REQUIREMENTS FOR “VERIFICATION” AND
“ONGOING EVALUATION”?
WHAT DOCUMENTS/RECORDS SHOULD
WE MAINTAIN FOR THE VERIFICATION?
Validation
Parameter
Documents
Protocol Written protocol for process used to verify NEW AST
Accuracy Data – NEW AST vs. REFERENCE, summary, conclusions
Reproducibility Data - QC with CLSI ATCC and additional QC strains ,
summary, conclusions
Putting Test IN
USE
All those documenting how test will be introduced and
protocols for ongoing evaluation (e.g., QC, competency,
PT, equipment PM, correlation with clinical findings, etc.)
*Note: good laboratory practice would dictate that records
should be maintained as long as test is in use
ENSURING QUALITY OF AST
10%
10%
10%
10%
10%10%
10%
10%
10%
10% Validation
Ongoing QC
Proficiency tests
Instrument maintenance
Competency
Relevant testingstrategiesVerify unusual results
Sup review of result
Clinical correlation
Keep current withchanges
Verification of ASTs
Also CAP’s BIT
(breakpoint implementation tool)
….but think about impact to the patient..
we must not “rationalize” that an error is OK
nor become obsessed about errors that are
unavoidable
Thank you!