Effective Control of the Examination Process
description
Transcript of Effective Control of the Examination Process
![Page 1: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/1.jpg)
Effective Control of theExamination Process
“Reunión de Expertos”
5 Ciclo Internacional De Conferencias De La Calidad
1
![Page 2: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/2.jpg)
Coordinator James Westgard, United States
PARTICIPANTS Laura Mercapide,
Argentina Amadeo Saez, Brasil Aída Porras, Colombia Oscar Martínez, Colombia Enrique Amaya, Perú Margarita Iturriza,
Venezuela Erik Mendoza, México Eduardo Brambila,
México Arturo Terrés, México
![Page 3: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/3.jpg)
![Page 4: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/4.jpg)
Our Focus is on the Examination Process
Pre-analytic and post-analytic errors are also of concern, but our charge is to focus on the analytical part of the examination process
Evidence in scientific literature indicates analytical errors are still major source of problems leading to mistreatment and harm to patients
![Page 5: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/5.jpg)
What errors have been observed in the Total Testing
Process?
Pre-analytic Analytic Post-analytic
• Patient preparation• Specimen acquisition• Specimen processing• Sample transport• Physician test order
• Sample aliquot• Analyzer setup• Test calibration• Quality control• Reportable test
• Test report• Transmittal of report• Receipt of report• Review of test results• Action on test results
60% 15% 25%Plebani & Carraro. Clin Chem 2007:53:1338-42
5
![Page 6: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/6.jpg)
Lab Errors and Patient Care Plebani-Carraro study
Clin Chem 2007:53:1338-42 51,746 tests 393 questionable results 160 confirmed as laboratory errors 46 caused inappropriate patient care 24 of those were analytical errors
Analytical errors are still major cause (over 50% cases) of inappropriate patient care 6
![Page 7: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/7.jpg)
Need for Guidance on “Effective Control of
Examination Process” for Latin American Countries
Important to be relevant and practical for local laboratories
Must consider national and governmental interests and requirements
Must consider professional assessment of needs and practice guidelines
![Page 8: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/8.jpg)
General guidance on “effective control of examination
process”? ISO 15189: Medical Laboratories –
Particular requirements for quality and competence
ISO 15198: Validation of user quality control procedures by the manufacturer
CLSI C24-A3: Statistical Quality Control for Quantitative Measuring Processes – Principles and Definitions
CLSI EP23-P: Laboratory Quality Control based on Risk Management
8
![Page 9: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/9.jpg)
ISO 15189 Guidance for “Assuring Quality” of Examination Process
5.6.1. “The laboratory shall design internal quality control systems that verify the attainment of the intended quality of results…” Medical relevance of laboratory tests
is an important consideration! Comparability of test results is
important for medical relevance! How define medical relevance?
9
![Page 10: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/10.jpg)
ISO 15189 “Assuring quality”
5.6.2 ..determine uncertainty of results, where relevant and possible
5.6.3 …ensure that results are traceable
5.6.4 …participate in interlaboratory comparisons
5.6.5 …if EQA not available, develop a mechanism for determining acceptability
5.6.6 For examinations performed using different procedures or at different sites, define a mechanism for verifying comparability of results
![Page 11: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/11.jpg)
Particular issues assigned to this work group
What frequency of QC is sufficient? How important are recommendations
from manufacturers for QC? Should laboratory modify recommendations?
How often for EQC or PT?
![Page 12: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/12.jpg)
What frequency of QC?
CLSI guidance on defining run length “An analytical run is an interval (i.e., a
period of time or series of measurements) within which the accuracy and precision of the measuring system is expected to be stable; between which events may occur causing the measurement process to be more susceptible (i.e., greater risk) to errors that are important to detect.”
12
![Page 13: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/13.jpg)
Factors Affecting Run Length (1)
Events and non-events Event driven QC
Known, scheduled and expected changes
Non-event driven QC Other things that happen
Parvin’s concepts Ref: Parvin, Gronowski. Effect of
analytical run length on QC performance and the QC planning process. Clin Chem 1997;11:2149-2154.
13
![Page 14: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/14.jpg)
Factors Affecting Run Length (2)
Mode of Operation Batch mode
All patient specimens and control samples are analyzed together
Patient results not reported until control results are validated
Continuous mode Patient results are being reported as
they are determined Control samples are analyzed
periodically14
![Page 15: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/15.jpg)
Factors affecting Run LengthCost-Effectiveness
Number of controls relative to number of patient specimens
Levels of controls
![Page 16: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/16.jpg)
Factors Affecting Run Length (3)
Strategies Manufacturer’s instructions may
provide minimum strategy – e.g., 2 levels per day
+ Event QC to assess significance of changes in the testing process
+ non-event QC to monitor process during routine operation
Multi-stage QC for “startup,” “monitoring,” and “patient data QC”
16
![Page 17: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/17.jpg)
Factors Affecting Run Length (4)
Strategies Sigma QC – relative amount of QC can be
related to method’s sigma-performance Risk analysis and residual risks – guidance
for susceptibility testing Consensus of experts – professional
practice standards Experienced judgment – knowledgeable
analyst has expertise about stability and susceptibility of testing process
17
![Page 18: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/18.jpg)
Run length and frequency of controls
Models
Measures Statistical
Assumptions
Empirical
Economic
Residual risks
Error frequency, f Q/P
Strategies
Sigma
Risk analysis
Consensus of experts
Event+Monitor
Manuf. recommendations
Batch processing
Experienced judgment
Residual risksEvent only
Cost of repeat analysis
Operations
Batch mode
System stability
Continuous mode
Reporting interval
Unexpected events
LIS
Susceptibility
Batch size
Components
Residual risks
Maintenance
Calibration, reagents
Analyte stability
Events
Operators
PartsLab conditions
Stats
QC performance goals
18
![Page 19: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/19.jpg)
What frequency of QC is sufficient?
Recommendations (1) Strategy
Define length of run In terms of time, numbers of patient
samples, mode of operation Importance of “events” or changes that
occur with the process that require verification by controls
Medically important concentrations for controls
General practice to use two levels of controls Sometimes advisable to have three levels
Many factors to consider to optimize run length or frequency of QC
![Page 20: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/20.jpg)
What frequency of QC? Recommendations (2)
For small runs, utilize “batch” strategy QC at beginning QC at end Release results after inspect all controls
and reviewing patient results when necessary
![Page 21: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/21.jpg)
What frequency of QC?Recommendations (3)
For large runs, highly automated systems with continuous reporting of results Controls at beginning of run
Right QC design to detect medically important errors + Controls for events
e.g., Change of reagent lots + Controls to monitor performance during run
Or, possibly use mean or median of patient data to monitor stability during run
+ Controls at end of run
![Page 22: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/22.jpg)
How important are the manufacturer’s QC directions?
Recommendations (4) Provide minimum requirements that the
laboratory must satisfy E.g., calibration, preventive maintenance, etc.
Laboratory is still responsible for design of IQC system Intended clinical use Observed method performance Necessary QC rules and Numbers of
measurements
![Page 23: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/23.jpg)
How important manufacturers QC directions?
Recommendations (5) Need for “independent” control
“Third party control” Traceability is an important responsibility
of manufacturer Calibration materials and process
Verification/validation of method performance is an important responsibility of the laboratory
EQA/PT important responsibility in monitoring/measuring accuracy over time “Commutability” important characteristic of
materials
![Page 24: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/24.jpg)
How often EQA/PT?Recommendations (6)
At least monthly With fast turnaround of results to be useful for
identifying bias and making improvements in the laboratory
Approved EQA program preferred ILAC G13:08/2007 ISO 17043
Most essential information – bias observed vs “assigned value”
![Page 25: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/25.jpg)
Other issues of interest
Medical relevance “Intended use,” “intended quality of test
results” Traceability
Comparability of test results Validation of method performance Design of IQC
Available planning approach, tools
![Page 26: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/26.jpg)
How assure quality?
26
(3) Validate MethodPerformance (CV,bias)
(1) Define Goals for Intended Use
(TEa, Dint)
(3a) Manufacturer’sClaims
(1a) Regulatory &AccreditationRequirements
(1b) Clinical and Medical Applications
(2) Select AnalyticMeasurement
Procedure
(2a) Traceability& Calibration
(2b) Manufacturer’sReference Methods
& Materials
(4) Design SQC(rules, N, F)
(5) FormulateAQC Strategy
(11) Improve AQCEffectiveness
(5b) Lab Evaluationof Residual Risk
(5a) Manufacturer’sRisk Analysis
(6) Develop AQC Plan
(10) Monitor AQCEffectiveness (f), EQA
(7) Implement AQC System
(6a) QC Toolbox
(8) Verify Attainmentof Intended Quality
of Test Results
(9) Measure Quality& Uncertainty
![Page 27: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/27.jpg)
ISO 15198 Validation of QC Procedures
QC procedures shall be validated to assure that failures are not a hazard to patients Recommends use of risk analysis
Conventional statistical quality control procedures (e.g., as described in CLSI C24) are considered adequate Validation may be based on simulated
effects of errors on performance data27
![Page 28: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/28.jpg)
CLSI C24 QC Planning Process
28
Define quality specifications for test
Select appropriate control materials
Determine method performance
Identify quality control strategies
Predict QC performance
Specify goals for QC performance
Select QC to satisfy goals
CalculateSigma
%TEa-%Bias %CV
UtilizeSigma-metricQC Selection
Tool
![Page 29: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/29.jpg)
Sigma-metrics QC Selection Tool
2 Levels Control
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
0.0 1.0 2.0 3.0 4.0
1.65 2.65 3.65 4.65 5.65
13s/2of32s/R4s/31s/6x 0.07 ----- 6 113s/22s/R4s/41s/8x 0.03 ----- 4 213s/22s/R4s/41s 0.03 ----- 4 112.5s 0.04 ----- 4 112.5s 0.03 ----- 2 113s/22s/R4s 0.01 ----- 2 113s 0.00 ----- 2 113.5s 0.00 ----- 2 1
Pfr Ped N R
Pro
bab
ilit
y f
or
Reje
cti
on
(P
)
Systematic Error (SE, multiples of s)
Sigma Scale
DesirableError
Detection
DesirableFalse
Rejection
3s 4s 5s
29
![Page 30: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/30.jpg)
EP22, EP23 on Risk Analysis
Original purpose of CLSI project was to develop scientific approach for defining frequency of QC
Adopted “risk analysis” approach Failure-modes and risk should provide
guidance on need for control mechanisms and frequency of QC
Analytical QC Plan should be the outcome of the risk analysis process
![Page 31: Effective Control of the Examination Process](https://reader036.fdocuments.us/reader036/viewer/2022062408/56814351550346895dafcc46/html5/thumbnails/31.jpg)
Important Considerations in Future QC Systems
Design Is there a scientific basis and approach for
selecting parameters and setting limits on basis of intended quality of results?
Validation Is there an objective approach for assessing
the reliability of technical and medical decisions on control status?
Control Is there a quantitative process for monitoring
and verifying the attainment of intended quality of test results?
31