Validation for Histology

Tim Morken, B.A. HTL(ASCP)

Univ. California San Francisco Medical Center

tim.morken@ucsfmedctr.org

2

Wall Street Journal, January 2008

3

Presentation Goals

1. Explain the term “Validation"

2. Explain how proper validation leads to quality

3. Discuss validation in the context of a "Quality

System”

Upon completion of this presentation,

participants will be able to:

4

Presentation Outline

Quality Management System

CAP / JC / CLIA Requirements

Validation Requirements

Quality Control follows from Validation

5

Key References

Quality Assurance in Anatomic Pathology (book) • Nakhleh and Fitzgibbons, Ed., College of American Pathologists, 2005

College of American Pathologists (CAP) Checklists

• Common

• Lab General

• Anatomic Pathology

Quality Assurance for Design Control and Implementation of Immunohistochemistry Assays; Approved Guideline—Second Edition

• Clinical Laboratory Standards Institute publication I/LA28-A2

Reference handout

6

Quality Management

Quality Management

• Definitions

• Interaction

7

CAP Checklists: Quality Management

General Laboratory Checklist:

• GEN.13806 - 20371

• Quality Management Program

• Quality Monitoring

• GEN.30070 Validation of Accuracy

Anatomic Pathology Checklist

• ANP.10000

• Is Quality Management Program Defined and Documented?

• Procedure Manual: no specific checklist number, but details

what is to be in the procedure manual for each method or process

8

College of American Pathologists

Three Definitions concerning Quality Management:

• 1) Quality Assurance (QA)

• 2) Quality Control (QC)

• 3) Quality Improvement (QI)

Together these constitute a "Quality System"

• Interaction of these elements leads to better quality

• System must be "worked" to succeed.

9

CAP Quality Assurance Definition

1) Quality Assurance

• The practice of assessing performance in

all steps of the laboratory testing cycle

including pre-analytic, analytic and post-

analytic phases to promote excellent

outcomes in medical care.

10

… Deconstructed

"…assessing performance…"

• Requires a set of performance standards.

• Requires comparison of results against those

standards.

"…to promote excellent outcomes…"

• Requires action if standards are not met

11

CAP Quality Control Definition

2) Quality Control

• An integral component of quality assurance

consisting of the aggregate of processes and

techniques to detect, reduce, and correct

deficiencies in an analytical process.

12

… Deconstructed

"…processes and techniques to detect, reduce,

and correct deficiencies…"

• "…Detect…":

• Requires procedures that anticipate possible non-conformances.

• "…Reduce and Correct…":

• Leads to procedures that reduce or eliminate confounding results.

13

CAP Quality Improvement Definition

3) Quality Improvement

• The practice of continuously assessing and

adjusting performance using statistically and

scientifically accepted procedures.

14

… Deconstructed

"…continuously assessing and adjusting…"

• Requires regular review of results data

• Investigation of, and correction of root cause

"..statistically and scientifically…"

• Data trends over a time period

• Hypothesis of root cause

• Eliminating variables

• Testing solutions

15

Quality Management System Summary

Quality Assurance • Describe the quality process • Set standards

Quality Control

• Institute methods to identify problems • Record and report results

Quality Improvement • Track problem trends identified by QC results • Use trend information to correct problems

16

Validation Process

Validation

• Definitions

• Processes

17

Verification vs Validation

Verification:

• The process by which a laboratory determines that an

FDA-cleared/approved test performs according to the

specifications set forth by the manufacturer.

Validation:

• A defined process by which a laboratory confirms that a

laboratory-developed (LDT) or modified FDA-

cleared/approved test performs as intended or claimed.

18

Validation Definition (1)

Establishing documented evidence which

provides a high degree of assurance that

a specific process will consistently

produce a result or product meeting it's

predetermined specifications and quality

attributes

19

Validation Definition (2)

"Establishing documented evidence…"

• Documentation of validation testing is readily available

“…which Provides a high degree of assurance…"

• Studying an adequate number of samples to give

confidence that the new or changed process will work in

your laboratory

20

Validation Definition (3)

"…that a specific process will consistently produce…"

• Identifying areas of actual or potential weaknesses so

improvements can be made prior to implementation.

"…a result or product meeting it's predetermined

specifications and quality attributes."

• Establishing acceptance criteria before initiating the

validation study.

21

Validation and Quality Control

Validation

• Assay Design Specification

• Validation

Verification

Optimization

Standardization

Quality Control

• Controls identified in Validation process

• Standardized procedures defined by Validation

• Identification of process deficiencies

22

Validation Flow Chart

Validation

23

Validation – Optimization - Standardization

Validation

Optimization

Standardization

24

Validation Process

Infinity, with Validation

25

Validation, Analytic Phase Terms (1)

CAP General Validation

CAP GEN.42020-42163 Test Method Validation

• Follows CLIA CFR Sec 493.1253

• Does not apply well to Histology (Histology is usually qualitative)

• But the general principle applies:

• The laboratory must have data on each test's accuracy, precision,

analytic sensitivity, interferences and reportable range

• Unmodified FDA-cleared or approved tests: the lab may use

manufacturer information or published reports but lab must verify outside

data.

• Non-FDA cleared: Lab MUST verify or establish analytic accuracy,

precision, sensitivity, specificity and reportable range.

26

Validation Analytic Terms Applied to Histology

Accuracy:

• Compare results with New test to a previously validated test

Precision:

• Test samples with varying expression

• Intra-run, Inter-run tests, 10 slides each

Sensitivity:

• Lowest level at which the test will demonstrate the target

Interferences [Specificity]:

• What could interfere to give a false positive or negative result?

Reportable Range

• Establish a scoring system

• Definition of a positive result

• Criteria for rejection of the test

27

Validation: Analytical

Validate Accuracy with typical cases

• Sensitivity:

• Expression range of Positive cases, low to high (10-15 cases)

• Specificity

• Positive verses expected negative cases/tissues (10-15 cases)

• Determine best controls

• Range of expression, similar to expected cases (normal or disease?)

• Preferably acquired and processed in your institution

Determine Precision (Reproducibility) • Intra-run: 10 slides in one run

• Inter-run: 10 slides, Ten different runs with one slide each

• Should have similar staining pattern and intensity on all slides

• Test on each instrument (at each site) used to run that test/process

28

Precision and Accuracy

29

Sensitivity

Analytic Sensitivity: • Lowest amount of substance detectable by the test

• Can only be done with controls of known concentration

Diagnostic Sensitivity: • Ability of the test to determine true diagnostic positive verses

false negative (higher % FN = less sensitive)

• Requires comparison to a previously validated test

Histology-related Sensitivity: • Extent to which an test can be modified and still achieve target

recognition.

• NOTE: for antibodies, This is determined by antibody AND detection system!

(adapted from: Theoretical and Practical Aspects of Test Performance, in Immunomicroscopy, Taylor & Cote, 2005)

30

Specificity

Analytic Specificity • Accuracy on tests of known positive and negative controls

• Controls of known concentration

• Determine what could “Interfere" to confound the result

Diagnostic Specificity • Ability of a test to determine true diagnostic negative verses false

positives (Higher % FP = less specific)

• Requires comparison to a previously validated antibody

Histology-related Specificity • Ability of an test procedure to demonstrate its’ particular target in

the absence of staining of other molecules • Or, staining of other structures in addition to target structures/cells

(adapted from: Theoretical and Practical Aspects of Test Performance, in Immunomicroscopy, Taylor & Cote, 2005)

31

Introducing a New Test or Process

We want to introduce or modify a test, instrument,

procedure…

• Who wants it?

• What is the desired result?

• When will we know we achieved the desired result?

• Where will it be used?

• Why do we need it?

• How will it be implemented?

32

Initial Design Specification

Write a Specification:

• The need for the assay

• The "why's"

• Identify special requirements

• Identify suppliers

• Determine the expected results

• Develop the validation procedure

• Optimize the test/process

• Develop the standard protocol

33

Validation: Pre-Analytic

Pre-Analytic example:

• Acquisition: Range of time before sample is put in fixative

• Fixation: Type and Range of time in fixative

• Processing: Several processing schedules

• Sectioning: Section thickness, drying, heating temperature

• Storage: Range of time periods for unstained slide

storage (retention of antigenicity), temperature

• Range of block storage conditions

34

Optimization Overview

Verify Vendor Specifications

• “to ascertain the truth or correctness of, as by examination, research, or comparison”

Vendor literature is a starting point

• Review the datasheet

• Review references given by vendor

• Literature search adds to knowledge

• Compare with other vendors, users

Verify/Optimize:

• Test procedure

• Timing

• Temperature

• Expected results

35

Final Design Specification

Final outcome:

• Validated test/process

• Controls specified

• Specify the expected results

• Specify rejection criteria (tolerance)

• "QC Data Sheet" to guide interpretation

36

Total Test Validation

Pre-Analytic

• Acquisition, Fixation, Processing, Sectioning, Storage

Analytic

• Optimization

• Technologist training

• Instruments

Post-Analytic

• Controls

• Interpretation

• Reporting

• Pathologist performance

(Adapted from: Goldstein NS, et..al., Appl Immunohistochem Mol Morph 15(2):124-133)

37

CAP AP QA/QC and Validation, Routine Histo, SS

ANP.

• .11713 Histologic Prep Quality

• 11734 Slide Quality

• 21395 Special Stains/Studies

• Includes special stains and IHC

• 21450 Special Stain Quality

• 23004 – 23402 Digital Imaging

• Extensive validation and QC requirements

38

Verification or Validation for General Histology

H&E:

• NEW model stainer: Validate for all types of tissue to be stained

• After moving instrument to a new location (verify)

• When changing staining reagent (H or E) (validate)

• When changing timing protocol per reagent (validate)

• Any substantial change in process

Special Stains

• When changing anything in protocol (validate)

• Reagents, Timing, etc)

• When introducing new ancillary equipment (waterbath, heating plate, etc)

(verify)

• When moving from manual to an instrument or from one instrument

to another (validate)

• After instrument move (verify)

39

Control Blocks

Control blocks, Specials or IHC

• Validation of suitable tissue

• Signed by Medical Director/Technical Director

• Target in all slides?

• Stain last slide to be sure

• Multi-tissue necessary (range of expression)?

• Validate storage conditions

• Temperature

• Time period target is viable is mounted sections

• Can you trace-back to original case?

40

Graduated Expression Arrays

Low Medium High

ER

PR

HER2

Courtesy: Pantomics, Ltd, www.pantomics.com

41

Verifcation or Validation for General Histology

Tissue processor

• NEW model: Validate for all types of tissue to be processed

• Move: Verify performance in new site

• Second instrument of same type: Verify with parallel testing on

original instrument.

• Changing protocol (reagents or timing): Validate

Microtome

• NEW model: Validate it cuts with good results

• Move: verify it is functional and cuts same as before move (pre-

move test for comparison!)

• Second instrument of same type: verify with parallel testing of

existing instrument

42

Optimization Method

Optimization

• Vendor-supplied protocols are recommendations

• Ideally work it into your regular protocol

• Test variations in timing, concentration,

temperature

• Variety of positive and negative samples to

determine sensitivity and specificity

• Staining method (Manual or Automated)

43

Optimization Result: General Histology

Write an optimized procedure for each

test/process, including:

• Materials

• Reagents

• Equipment

• Procedure

• Possible pitfalls

• Expected results

44

CAP AP Checklist, IHC

ANP.

• 12425 ASR disclaimer for report

• 21850 Positive and Negative controls for immunofluorescence

• 22250 Procedure for each antibody

• 22300 Documented modifications for fixation other than formalin

• 22500 Buffer pH monitoring

• 22550 Positive control use

• 22570 Negative control use (tissue, reagent)

• 22615 Avidin/Biotin blocking/controls

• 22660 Review/Recording of IHC results

• 22750 Validation of new antibody (except ER, PR, Her2)

• 22760 New lot validation for Antibody and Detection system

• 22800 Automated IHC staining instrument maintenance records

• 22900 Quality of IHC stains

• 22997 Her2 validation (IHC and ISH)

45

Optimization Result: IHC

Write an optimized procedure for each

antibody, including:

• .22250: Test Procedure

• reagents, dilution, HIER, etc

• .22300: Fixation

• .22550: Positive Control

• .22570: Negative Control

• .22625: Avidin-biotin Block (if necessary)

46

Key Take-away

Validation Drives Quality

• Ensures test works as intended

• Develops Standardized Procedures

• Identifies Quality Control methods/materials

Questions?

Thank you.