College of American Pathologists1
College of American PathologistsCollege of American Pathologists
Getting the Most Out of Your Proficiency Testing
Laboratory Accreditation Program
Copyright © 2007 College of American Pathologists (CAP). All rights are reserved. Participants are permitted to duplicate the materials for educational use only within their own institution. These materials may not be used for commercial purposes or altered in any way.
Denise K. Driscoll, MS, MT(ASCP)SBB
Laboratory Accreditation Director
Laboratory Accreditation Program
College of American Pathologists2
ObjectivesObjectives
After participating in today’s session, you will be able to:
• Discuss the regulatory aspects and impact of proficiency testing (PT)
• Identify approaches to effectively manage and fully utilize PT in your laboratory
• List investigative techniques used to determine causes of poor PT performance and describe effective corrective actions
• Use PT summary discussions as continuing education opportunities
College of American Pathologists3
Proficiency Testing Do’sProficiency Testing Do’s
• Enrollment
• Participation
• Evaluation of results
• Investigation of failures
• Corrective actions
• Maintenance of records
College of American Pathologists4
Proficiency Testing Don’ts
Proficiency Testing Don’ts
• Test samples differently than patient specimens
• Have same people always do testing• Refer samples to another laboratory• Discuss results before reporting• Use PT samples for other purposes
before submission date deadline
College of American Pathologists5
EnrollmentEnrollment
• Required for all analytes listed in the PT Enrollment Guide– Updated annually– Available at CAP Web site– Included with order renewal forms
College of American Pathologists6
ParticipationParticipation
• In addition to enrolling and testing samples, you must:– Ensure results are sent– Ensure they are sent by the deadline– Ensure all faxed pages are received
• Note: non-participation (failure to receive results) = unsatisfactory
College of American Pathologists7
Evaluation of PT ResultsEvaluation of PT Results
To identify any deviation in proficiency testing from external requirements or facility's expectations that requires investigation and follow-up
College of American Pathologists8
Reading the ReportReading the Report
• Performance on graded (acceptable vs. unacceptable)
• Ungraded results (codes)
• Bias on quantitative testing– Level of bias requiring
action defined in procedureAttachments
A, B, C
College of American Pathologists9
Reported Causes of PT Failures
Reported Causes of PT Failures
51%
24%
12%
7%5% 1%
CLERICAL
NO EXPLANATION
PT MATERIALS
METHODOLOGICAL
TECHNICAL
OTHER
College of American Pathologists10
Clerical ErrorsClerical Errors
• Postanalytic phase
• Same importance as testing errors
• Examples:– Transcription – Method/reagent/instrument codes– Missing information (TNP, etc.)
College of American Pathologists11
Technical IssuesTechnical Issues
Directly attributable to human actions:• Reconstitution/pipetting/dilution errors• Specimen mix-up• Improper specimen handling• Incorrect instrument set-up• Failure to follow testing kit instructions• Morphologic misinterpretation
College of American Pathologists12
Methodologic IssuesMethodologic Issues
• Mechanical difficulties• Instrument software problems• Frequency of calibration• Inadequate reagent performance• Inadequate maintenance/function checks• Other instrument malfunction
(intermittent electric problems)
College of American Pathologists13
Issues with PT Testing Materials
Issues with PT Testing Materials
• Hemolyzed, contaminated
• Unstable PT materials
• Perceived bias
• Matrix effect incompatible with method
• Late shipment
College of American Pathologists14
Investigating PT Failures
• See Attachment “D”Investigation Flowchart
AttachmentD
College of American Pathologists15
InvestigationInvestigation
• Review the all-participant report received from your PT Provider
• Review reporting records• Review sample preparation/handling,
testing• Verify that the PT material was
processed in the correct instrument mode
• Review results from previous survey
College of American Pathologists16
Investigation 2Investigation 2
• Review QC performance, instrument calibration, and reagent performance prior to, during, and after the original analysis of the PT challenge
• Contact the instrument/reagent manufacturer for assistance
• Retest the PT specimen(s), if possible or purchase additional specimens with different reagent lot
College of American Pathologists17
Corrective Actions Corrective Actions
• Design a process to verify clerical entries prior to PT result submission– Verify instrument/method codes
• Retrain testing personnel as to the proper procedures for sample preparation, testing, and reporting
• Modify (narrow) the QC acceptance range to detect problems sooner
College of American Pathologists18
Corrective Actions 2Corrective Actions 2
• Evaluate/increase frequency of calibration
• Perform instrument function verification
• Revise procedure to reflect corrective actions
*For more information, see NCCLS GP-27A “Use of Proficiency Testing to Improve the Clinical Laboratory
College of American Pathologists19
Maintenance of RecordsMaintenance of RecordsOriginal records
• Who performed testing/reported results
• Date/time of testing and reporting
• Worksheets/instrument tapes
• Confirmation results were received
Investigation• QC records
– Date of testing– Before/after
• Maintenance at defined frequency
• Calibration as defined by procedure
Requirement listed in GEN. 11484
College of American Pathologists20
Closing the LoopClosing the Loop
• Include:– Explanation of investigation– Documentation of external validation– Determination of whether there has
been impact on patient care– Implementation of new
processes/procedures to prevent recurrence
College of American Pathologists21
Not Closing the LoopNot Closing the Loop
• May result in sanctions– Additional documentation– Suspension of testing– Cessation of testing
College of American Pathologists22
Satisfactory PT Performance
Satisfactory PT Performance
• Is 80% for regulated analytes
– and other analytes that have at least 5 challenges per PT event)
• Percentage varies for analytes with fewer than 5 challenges
College of American Pathologists23
Unsatisfactory PT Performance
Unsatisfactory PT Performance
• Laboratories that have unsatisfactory performance for 1 or more analytes on an event will receive a PT Exception Summary (PTES) report. Labs must:– Investigate problem – Determine cause – Implement corrective actions
• Response to LAP is not required
College of American Pathologists24
Unsuccessful PT Performance
Unsuccessful PT Performance
• Is unsatisfactory performance on 2 of 3 PT events
• Laboratory will receive a PT Exception Summary (PTES) report
• Laboratory must suspend testing or implement plan of correction
• LAP must approve laboratory’s action
College of American Pathologists25
Case StudyCase Study
AttachmentE, F
Story line:Ineffective investigation and corrective actions led to cessation of testing:2004-C: + bias, all acceptable2005-A: no results received (fax problem?)2005B: + bias, all acceptable2005-C: + bias, 40% acceptable(calibrator problem, almost expired)2006-A: - bias, 60% acceptable(calibrator problem)2006-B: ok
College of American Pathologists26
Alternative AssessmentsAlternative Assessments
• Participation in any formal PT program (for which enrollment is not required) including educational
• Split sample analysis with other laboratories
• Testing against an established method
College of American Pathologists27
Alternative AssessmentsAlternative Assessments• Testing against a reference sample
– product calibrator– previously assayed material (e.g.,
regional pool)• Analysis of patient data
– clinical correlation– delta checking; moving averages
• See NCCLS GP-29A “Assessment of Laboratory Tests When Proficiency Testing is Not Available [2002]”
College of American Pathologists28
Split-Sample Testing: Quantitative
Split-Sample Testing: Quantitative
• Determine type of statistical analysis to use
• Determine criteria for acceptability
• Test samples representing AMR
• Test enough samples
College of American Pathologists29
Split-Sample Testing: Qualitative
Split-Sample Testing: Qualitative
95% confidence
• If only 2 of 3 match, the sample size must be increased to 6
• 5 of the 6 must match to achieve the same level of confidence.
College of American Pathologists30
Continuing EducationContinuing Education
Formal CME/CE– Contact individual PT providers for list
of opportunities
Informal– Discuss performance/investigations
with all techs– Review of summaries/critiques at staff
meetings– Pathologist led CE meeting using PT
readings
College of American Pathologists31
Objective of PT:Objective of PT:
Quality patient care
College of American Pathologists32
Key Points for PT Management
Key Points for PT Management
• Spend more time - a lot more time - going over your PT results
• Be sure PT samples are handled as patient samples
• Be certain that every analyte is covered by formal or alternative assessments– Establish limits of acceptability if the
provider has not
College of American Pathologists33
Additional TipsAdditional Tips
• Keep original worksheets and instrument tapes with the PT file
• Make sure every less-than-acceptable result is followed up and documented
• Resolve systematic biases• Confirm that corrective actions make
sense and actually fix the problem
College of American Pathologists34
Technical AssistanceTechnical Assistance
http://www.cap.org
e-mail: [email protected]
800-323-4040, ext. 6065
College of American Pathologists35
Questions and
Answers
Thank YouThank You
Top Related