Romeo & Juliet at San Francisco Ballet_Encore Arts San Francisco
Univ. California San Francisco Medical Center...
Transcript of Univ. California San Francisco Medical Center...
Validation for Histology
Tim Morken, B.A. HTL(ASCP)
Univ. California San Francisco Medical Center
2
Wall Street Journal, January 2008
3
Presentation Goals
1. Explain the term “Validation"
2. Explain how proper validation leads to quality
3. Discuss validation in the context of a "Quality
System”
Upon completion of this presentation,
participants will be able to:
4
Presentation Outline
Quality Management System
CAP / JC / CLIA Requirements
Validation Requirements
Quality Control follows from Validation
5
Key References
Quality Assurance in Anatomic Pathology (book) • Nakhleh and Fitzgibbons, Ed., College of American Pathologists, 2005
College of American Pathologists (CAP) Checklists
• Common
• Lab General
• Anatomic Pathology
Quality Assurance for Design Control and Implementation of Immunohistochemistry Assays; Approved Guideline—Second Edition
• Clinical Laboratory Standards Institute publication I/LA28-A2
Reference handout
6
Quality Management
Quality Management
• Definitions
• Interaction
7
CAP Checklists: Quality Management
General Laboratory Checklist:
• GEN.13806 - 20371
• Quality Management Program
• Quality Monitoring
• GEN.30070 Validation of Accuracy
Anatomic Pathology Checklist
• ANP.10000
• Is Quality Management Program Defined and Documented?
• Procedure Manual: no specific checklist number, but details
what is to be in the procedure manual for each method or process
8
College of American Pathologists
Three Definitions concerning Quality Management:
• 1) Quality Assurance (QA)
• 2) Quality Control (QC)
• 3) Quality Improvement (QI)
Together these constitute a "Quality System"
• Interaction of these elements leads to better quality
• System must be "worked" to succeed.
9
CAP Quality Assurance Definition
1) Quality Assurance
• The practice of assessing performance in
all steps of the laboratory testing cycle
including pre-analytic, analytic and post-
analytic phases to promote excellent
outcomes in medical care.
10
… Deconstructed
"…assessing performance…"
• Requires a set of performance standards.
• Requires comparison of results against those
standards.
"…to promote excellent outcomes…"
• Requires action if standards are not met
11
CAP Quality Control Definition
2) Quality Control
• An integral component of quality assurance
consisting of the aggregate of processes and
techniques to detect, reduce, and correct
deficiencies in an analytical process.
12
… Deconstructed
"…processes and techniques to detect, reduce,
and correct deficiencies…"
• "…Detect…":
• Requires procedures that anticipate possible non-conformances.
• "…Reduce and Correct…":
• Leads to procedures that reduce or eliminate confounding results.
13
CAP Quality Improvement Definition
3) Quality Improvement
• The practice of continuously assessing and
adjusting performance using statistically and
scientifically accepted procedures.
14
… Deconstructed
"…continuously assessing and adjusting…"
• Requires regular review of results data
• Investigation of, and correction of root cause
"..statistically and scientifically…"
• Data trends over a time period
• Hypothesis of root cause
• Eliminating variables
• Testing solutions
15
Quality Management System Summary
Quality Assurance • Describe the quality process • Set standards
Quality Control
• Institute methods to identify problems • Record and report results
Quality Improvement • Track problem trends identified by QC results • Use trend information to correct problems
16
Validation Process
Validation
• Definitions
• Processes
17
Verification vs Validation
Verification:
• The process by which a laboratory determines that an
FDA-cleared/approved test performs according to the
specifications set forth by the manufacturer.
Validation:
• A defined process by which a laboratory confirms that a
laboratory-developed (LDT) or modified FDA-
cleared/approved test performs as intended or claimed.
18
Validation Definition (1)
Establishing documented evidence which
provides a high degree of assurance that
a specific process will consistently
produce a result or product meeting it's
predetermined specifications and quality
attributes
19
Validation Definition (2)
"Establishing documented evidence…"
• Documentation of validation testing is readily available
“…which Provides a high degree of assurance…"
• Studying an adequate number of samples to give
confidence that the new or changed process will work in
your laboratory
20
Validation Definition (3)
"…that a specific process will consistently produce…"
• Identifying areas of actual or potential weaknesses so
improvements can be made prior to implementation.
"…a result or product meeting it's predetermined
specifications and quality attributes."
• Establishing acceptance criteria before initiating the
validation study.
21
Validation and Quality Control
Validation
• Assay Design Specification
• Validation
Verification
Optimization
Standardization
Quality Control
• Controls identified in Validation process
• Standardized procedures defined by Validation
• Identification of process deficiencies
22
Validation Flow Chart
Validation
23
Validation – Optimization - Standardization
Validation
Optimization
Standardization
24
Validation Process
Infinity, with Validation
25
Validation, Analytic Phase Terms (1)
CAP General Validation
CAP GEN.42020-42163 Test Method Validation
• Follows CLIA CFR Sec 493.1253
• Does not apply well to Histology (Histology is usually qualitative)
• But the general principle applies:
• The laboratory must have data on each test's accuracy, precision,
analytic sensitivity, interferences and reportable range
• Unmodified FDA-cleared or approved tests: the lab may use
manufacturer information or published reports but lab must verify outside
data.
• Non-FDA cleared: Lab MUST verify or establish analytic accuracy,
precision, sensitivity, specificity and reportable range.
26
Validation Analytic Terms Applied to Histology
Accuracy:
• Compare results with New test to a previously validated test
Precision:
• Test samples with varying expression
• Intra-run, Inter-run tests, 10 slides each
Sensitivity:
• Lowest level at which the test will demonstrate the target
Interferences [Specificity]:
• What could interfere to give a false positive or negative result?
Reportable Range
• Establish a scoring system
• Definition of a positive result
• Criteria for rejection of the test
27
Validation: Analytical
Validate Accuracy with typical cases
• Sensitivity:
• Expression range of Positive cases, low to high (10-15 cases)
• Specificity
• Positive verses expected negative cases/tissues (10-15 cases)
• Determine best controls
• Range of expression, similar to expected cases (normal or disease?)
• Preferably acquired and processed in your institution
Determine Precision (Reproducibility) • Intra-run: 10 slides in one run
• Inter-run: 10 slides, Ten different runs with one slide each
• Should have similar staining pattern and intensity on all slides
• Test on each instrument (at each site) used to run that test/process
28
Precision and Accuracy
29
Sensitivity
Analytic Sensitivity: • Lowest amount of substance detectable by the test
• Can only be done with controls of known concentration
Diagnostic Sensitivity: • Ability of the test to determine true diagnostic positive verses
false negative (higher % FN = less sensitive)
• Requires comparison to a previously validated test
Histology-related Sensitivity: • Extent to which an test can be modified and still achieve target
recognition.
• NOTE: for antibodies, This is determined by antibody AND detection system!
(adapted from: Theoretical and Practical Aspects of Test Performance, in Immunomicroscopy, Taylor & Cote, 2005)
30
Specificity
Analytic Specificity • Accuracy on tests of known positive and negative controls
• Controls of known concentration
• Determine what could “Interfere" to confound the result
Diagnostic Specificity • Ability of a test to determine true diagnostic negative verses false
positives (Higher % FP = less specific)
• Requires comparison to a previously validated antibody
Histology-related Specificity • Ability of an test procedure to demonstrate its’ particular target in
the absence of staining of other molecules • Or, staining of other structures in addition to target structures/cells
(adapted from: Theoretical and Practical Aspects of Test Performance, in Immunomicroscopy, Taylor & Cote, 2005)
31
Introducing a New Test or Process
We want to introduce or modify a test, instrument,
procedure…
• Who wants it?
• What is the desired result?
• When will we know we achieved the desired result?
• Where will it be used?
• Why do we need it?
• How will it be implemented?
32
Initial Design Specification
Write a Specification:
• The need for the assay
• The "why's"
• Identify special requirements
• Identify suppliers
• Determine the expected results
• Develop the validation procedure
• Optimize the test/process
• Develop the standard protocol
33
Validation: Pre-Analytic
Pre-Analytic example:
• Acquisition: Range of time before sample is put in fixative
• Fixation: Type and Range of time in fixative
• Processing: Several processing schedules
• Sectioning: Section thickness, drying, heating temperature
• Storage: Range of time periods for unstained slide
storage (retention of antigenicity), temperature
• Range of block storage conditions
34
Optimization Overview
Verify Vendor Specifications
• “to ascertain the truth or correctness of, as by examination, research, or comparison”
Vendor literature is a starting point
• Review the datasheet
• Review references given by vendor
• Literature search adds to knowledge
• Compare with other vendors, users
Verify/Optimize:
• Test procedure
• Timing
• Temperature
• Expected results
35
Final Design Specification
Final outcome:
• Validated test/process
• Controls specified
• Specify the expected results
• Specify rejection criteria (tolerance)
• "QC Data Sheet" to guide interpretation
36
Total Test Validation
Pre-Analytic
• Acquisition, Fixation, Processing, Sectioning, Storage
Analytic
• Optimization
• Technologist training
• Instruments
Post-Analytic
• Controls
• Interpretation
• Reporting
• Pathologist performance
(Adapted from: Goldstein NS, et..al., Appl Immunohistochem Mol Morph 15(2):124-133)
37
CAP AP QA/QC and Validation, Routine Histo, SS
ANP.
• .11713 Histologic Prep Quality
• 11734 Slide Quality
• 21395 Special Stains/Studies
• Includes special stains and IHC
• 21450 Special Stain Quality
• 23004 – 23402 Digital Imaging
• Extensive validation and QC requirements
38
Verification or Validation for General Histology
H&E:
• NEW model stainer: Validate for all types of tissue to be stained
• After moving instrument to a new location (verify)
• When changing staining reagent (H or E) (validate)
• When changing timing protocol per reagent (validate)
• Any substantial change in process
Special Stains
• When changing anything in protocol (validate)
• Reagents, Timing, etc)
• When introducing new ancillary equipment (waterbath, heating plate, etc)
(verify)
• When moving from manual to an instrument or from one instrument
to another (validate)
• After instrument move (verify)
39
Control Blocks
Control blocks, Specials or IHC
• Validation of suitable tissue
• Signed by Medical Director/Technical Director
• Target in all slides?
• Stain last slide to be sure
• Multi-tissue necessary (range of expression)?
• Validate storage conditions
• Temperature
• Time period target is viable is mounted sections
• Can you trace-back to original case?
40
Graduated Expression Arrays
Low Medium High
ER
PR
HER2
Courtesy: Pantomics, Ltd, www.pantomics.com
41
Verifcation or Validation for General Histology
Tissue processor
• NEW model: Validate for all types of tissue to be processed
• Move: Verify performance in new site
• Second instrument of same type: Verify with parallel testing on
original instrument.
• Changing protocol (reagents or timing): Validate
Microtome
• NEW model: Validate it cuts with good results
• Move: verify it is functional and cuts same as before move (pre-
move test for comparison!)
• Second instrument of same type: verify with parallel testing of
existing instrument
42
Optimization Method
Optimization
• Vendor-supplied protocols are recommendations
• Ideally work it into your regular protocol
• Test variations in timing, concentration,
temperature
• Variety of positive and negative samples to
determine sensitivity and specificity
• Staining method (Manual or Automated)
43
Optimization Result: General Histology
Write an optimized procedure for each
test/process, including:
• Materials
• Reagents
• Equipment
• Procedure
• Possible pitfalls
• Expected results
44
CAP AP Checklist, IHC
ANP.
• 12425 ASR disclaimer for report
• 21850 Positive and Negative controls for immunofluorescence
• 22250 Procedure for each antibody
• 22300 Documented modifications for fixation other than formalin
• 22500 Buffer pH monitoring
• 22550 Positive control use
• 22570 Negative control use (tissue, reagent)
• 22615 Avidin/Biotin blocking/controls
• 22660 Review/Recording of IHC results
• 22750 Validation of new antibody (except ER, PR, Her2)
• 22760 New lot validation for Antibody and Detection system
• 22800 Automated IHC staining instrument maintenance records
• 22900 Quality of IHC stains
• 22997 Her2 validation (IHC and ISH)
45
Optimization Result: IHC
Write an optimized procedure for each
antibody, including:
• .22250: Test Procedure
• reagents, dilution, HIER, etc
• .22300: Fixation
• .22550: Positive Control
• .22570: Negative Control
• .22625: Avidin-biotin Block (if necessary)
46
Key Take-away
Validation Drives Quality
• Ensures test works as intended
• Develops Standardized Procedures
• Identifies Quality Control methods/materials
Questions?
Thank you.