The Association for Molecular Pathology Education. Innovation and Improved Patient Care. Advocacy.

www.amp.org

Regulation and Validation

Elaine Lyon, PhD FACMG

University of Utah/ARUP Laboratories

April 2013

Disclosure(s)

In accordance with ACCME guidelines, any individual in a position to influence and/or control the content of this ASCP CME activity has disclosed all relevant financial relationships within the past 12 months with commercial interests that provide products and/or services related to the content of this CME activity.

The individual below has disclosed the following financial relationship(s) with commercial interest(s):

• Complete Genomics (advisory board) • Asuragen (reagents/speaking)

• Life Technologies • Invitae

Elaine Lyon, PhD, FACMG

2

3

Clinical Laboratory Improvement Amendments (CLIA)

• Centers for Medicare & Medicaid Services (CMS)

– Division of Laboratory Services

• Objective is to ensure quality laboratory testing

– Standard operating procedures (SOP)

– Proficiency testing

– Laboratory personnel

– Test method verification/validation

– Clinical validity

– Quality control/ quality assessment

– Record and specimen retention

– Calibration

4

Levels of complexity

• High complexity

– Molecular tests typically considered high complexity

• Moderate complexity

• Low complexity

• Waived tests

– Dipstick, glucose monitoring, blood count (for example)

5

College of American Pathology

• Laboratory accreditation

– Inspected by a not-for-profit accrediting organization approved by CMS and requirements equivelent or more stringent than CMS’s regulatory requirements

– Checklist

• General

• Molecular pathology

– Inspections (unannounced)

• Internal

• External

6

Phase I vs. Phase II

• All Phase I and II deficiencies must be addressed.

– Phase I deficiencies: a statement regarding corrective action(s) being taken is sufficient. Recurring Phase I deficiencies will be reviewed critically as to the effectiveness of proposed actions.

– Phase II deficiencies: corrective actions must be accompanied by appropriate documentation, such as policies or procedures, plus supporting records, such as patient reports, log sheets, purchase orders, photographs, memos or minutes from meetings. These records must demonstrate compliance with the policy or procedure. Failure to provide this documentation will delay accreditation.

© 2009 College of American Pathologists. All rights reserved. Modified from Daniel Farkas, PhD

Personnel Director Qualifications Pathologist, board-certified physician in a specialty other than pathology, or doctoral scientist in a biologic science, with specialized training and/or appropriate experience in molecular pathology

Molecular Pathology Checklist © 2010 College of American Pathologists. All rights reserved.

Personnel - Technical Operations The person in charge of technical operations of the molecular pathology laboratory is qualified as one of the following. 1. Person who qualifies as a director 2. MB(ASCP), BS, BA or MLS(ASCP)/MT(ASCP) with at least 4 years experience (at least 1 in molecular

pathology methods) under a qualified director

Training/CME There is an adequate training program for new technologists, and a continuing medical laboratory education program.

Technologist Qualifications Persons performing the technical aspects of molecular pathology qualify as one of the following: 1. Experienced in field under direct supervision of a qualified director or supervisor & qualified to

perform high complexity testing 2. MT(ASCP) certified or equivalent 3. BA or BS degree in biologic sciences with appropriate experience in molecular pathology

Modified from Daniel Farkas, PhD

8

Some Examples

– The laboratory has a document control system.

– The competency of each person to perform his/her assigned duties is assessed.

– Fire drills are conducted at least annually.

– Refrigerator/freezer temperatures are checked and recorded daily.

– The laboratory's current CAP Activity Menu accurately reflects the testing performed.

– There is evidence that the laboratory monitors sample turnaround times and that they are appropriate for the intended purpose of the test.

Modified from Daniel Farkas, PhD

9

Examples

– When appropriate, statistics on molecular pathology test results (e.g. percentages of normal and abnormal findings, allele frequencies) are maintained, and appropriate comparative studies performed.

– Validation studies document test accuracy, analytical sensitivity, analytical specificity and precision.

– New reagent lots and/or shipments are validated before or concurrent with use for patient testing.

Modified from Daniel Farkas, PhD

10

State Regulations

• New York State

– Certificate of Qualification (directors)

– Test specific approval

• California

– Education/training of personnel

• Florida

• Etc

11

Human Subjects Research and Clinical Laboratories

• Use of de-identified residual samples not defined as Human Subjects Research for

– Quality Control

– Quality Improvement

– Validation

• The lines between clinical and research testing may become thin with genome assays

12

Validation/Verification

• High Complexity laboratories must establish and verify method performance characteristics before introducing a new testing procedure or method:

– Verify FDA-cleared reagents

– Laboratory developed method (LDT)

– Modification of a manufacturer’s approved method

– Method not previously cleared by the FDA

13

FDA and LDT’s

• Clinical laboratories as medical device manufacturers when offer LDTs

• Medical devices are the jurisdiction of the FDA

– “Safe and effective” standard

• FDA has historically exercised “enforcement discretion”

• Developing a “framework” for LDT oversight

– LDT’s are increasingly used and some aggressively marketed to physicians and consumers • Many are genetics tests; some are DTC

– CLIA does not provide for oversight of “clinical validity”

14

Validating Clinical Assays

• Analytical Validation

– Familiarity

– Design

– Optimization

– Accuracy

– Robustness (reproducibility)

– Interpretation/Reporting

• Clinical validation

– Clinical sensitivity

– Clinical specificity

15

Analytical Validation

• Accuracy – Samples with known genotypes and controls

– Analytical performance sensitivity/Specificity

• Specificity: False (+) = True negatives/True negatives + False positive

• Sensitivity: False (-) = True positives/True positives + False negatives

• Within-run – Examples of each genotype run in triplicate

• Between-run – Examples of each genotype run on 5 runs, separate days, different

instruments, different technologists

• Other performance characteristics – Sample type, limit of detection (also called sensitivity), inhibiting

subStances (also called specificity)

Huntington Design

• Current testing

– Sizing PCR (CAG)

– Apparent homozygous: Sizing PCR (CCG and CCG+CAG)

– Non-informative or apparent homozygous: Southern analysis

• Triplet-repeat primed PCR

CAG

CCG

CCG+ CAG

Southern

17

HD: Accuracy

• Compare to standard

method

• Sizing PCR and Southern

• Variety of sizes tested

– Normal

– Normal mutable

– Reduced penetrance

– Affected (fully penetrant)

18

HD Accuracy Summary

HD samples type Alleles correctly Genotyped by both assay

Alleles correctly genotyped to within ± 1

Normal: <27 CAG repeats

7 0

Normal mutable: 27-35 CAG repeats

20 0

Reduced penetrance: 36-39 GAG repeats

9 0

Affected: >39 CAG repeats

146 6

Total 246 6

Reproducibility Sample Name

Within run validation Sample Between run validation

NA20207 Allele 19: x̄ 98.96, SD ± 0.05 Allele 21: x̄ 102.88, SD ± 0.04

NA20207 Allele 19: x̄ 96.72, SD ± 0.1 Allele 21: x̄ 102.72, SD ± 0.1

NA20249 Allele 22: x̄ 105.9, SD ± 0 Allele 39: x̄ 155.92, SD ± 0.04

NA20249 Allele 22: x̄ 105.72, SD ± 0.1 Allele 39: x̄ 155.7, SD ± 0.1

NA20252 Allele 22: x̄ 105.86, SD ± 0.05 Allele 65: x̄ 231.36, SD ± 0.2

NA20252 Allele 22: x̄ 105.7, SD ± 0.1 Allele 65: x̄ 230.9, SD ± 0.2

NA20253 Allele 22: x̄ 105.86, SD ± 0.1 Allele 101: x̄ 334.2, SD ± 0.1

NA20253 Allele 22: x̄ 105.68, SD ± 0.1 Allele 101: x̄ 333.6, SD ± 0.4

MGL-14 Allele 19: x̄ 96.88, SD ± 0.04 Allele: Expanded

MGL-14 Allele 19: x̄ 96.74, SD ± 0.1 Allele: Expanded

Allele denoted in CAG repeat size, x̄= mean size in base pairs, DS=Standard deviation

20

Sequencing Assays:

• Reference sequence

• Alternative transcripts

• Homology checks – pseudogenes

• Inheritance

• Databases – Locus specific

• Known benign variants

Gene GBK file (analysis

GBK file (reporting)

Mutation database

numbering differences

MLPA exon numbering differences

GVIE- ARUP Wiki

CDS Inheritance

PTEN NC_000010.10 NM_000314.4 None NO No A.dominant

21

Regions Interrogated

• Targeted exons

– Example: MEN2

• All coding exons

– ‘Full gene or full sequence analysis’

• Intron/exon boundaries

– +20-+50

• Known deep intronic mutations

• Regulatory regions

– 5’ UTR, promoter

– 3’ UTR

22

Workflow

• M13 tagged primers

• Workflow

– Low throughput – per sample

– High throughput – per exon

• Primer plate

23

Primer Design

• Often per exon

• Design around pseudogenes

• Avoid known variants

– Interfere with PCR

• All at same PCR conditions?

• Tailed PCR primers?

24

Difficult Regions

• High GC content

– Optimization

• Secondary structure

– Optimization or avoidance

• Benign Insertions/deletions

– Example: CFTR GATT

• Pseudogenes

– Example: PMS2

• Repeat motifs

– Example: CFTR intron 8 TG/T

– Example: Homopolymers

CFTR GATT

• Design Long and Short amplicons

• Cover all regions

Primer Design

CFTR intron 8 TG/T region F and R primers for Long amplicon

Primer Design

CFTR intron 8 TG/T region F and R primers for Short amplicon

27

Primer Design

• Loop-out/masking

28

Analytical Validation

• Performance characteristics

– *Accuracy

29

Quality Checks

• Trace scores: average quality score

• Signal intensity

• Signal to noise ratio

• %QV20+ :percentage of bases with quality values > 20

M13R R To be sequenced F M13F

Total amplicon length

Amplicon length for %QV20+ calculation

Reproducibility – PCR product

*Intra-run variability

Re-design of exon 3

Reproducibility – PCR products

*Inter-run

All reactions

32

Proficiency Testing

• CLIA approved PT providers: – http://www.cms.gov/Regulations-and-

Guidance/Legislation/CLIA/Downloads/ptlist.pdf

• CAP or CAP/ACMG – “A means of evaluating a laboratory’s performance under controlled conditions relative

to a given set of criteria through analysis of unknown samples provided by an external source” CAP

• External Quality Assurance twice a year

• Alternative assessment

– Interlaboratory exchange facilitated by CAP

– Interlaboratory exchange (lab managed)

• Internal assessment (blinded sample)

33

Surveys

• Molecular Genetics (MGL1-5)

• Inherited metabolic disease by DNA (IMD)

• Molecular Genetic Sequencing (SEC1)

• Pharmacogenetics (PGX)

• Molecular Oncology – Solid Tumors (MSI, SARC,

• Molecular Oncology – Hematologic (MHO-MHO5, MRD)

• Molecular Microbiology (VR1, DFA, HCV, HIV, VLS, HC, HPV, BSTE, ID)

• HLA Molecular Typing (ML, DL, DML)

34

Current Procedure Terminology (CPT)

• Molecular microbiology codes

– Included with microbiology codes

• Molecular Pathology Codes (2013)

– Oncology and Inherited diseases

– General principles • Accepted clinical practice

• Multiple labs

• Typical test

• Increased transparency

• Tier 1 – frequently ordered

• Tier 2

• Unlisted

Tier 1 • Examples:

• Transparent

• Inherited disease

– 30% of test menu

– 87% of test volume

• Somatic diseases

– 41% of test menu

– 63% of test volume

• Represents ‘mature’ tests

• Often simpler, less expensive tests

81206: BCR/ABL1 (t(9:22)) (eg, chronic myelogenous leukemia translocation analysis; major breakpoint, qualitative or quantitative 81207: > minor breakpoint, qualitative of quantitative 81208: > other breakpoint, qualitative or quantitative 81220: CFTR (cystic fibrosis transmembrane conductance regulator) (eg, cystic fibrosis) gene analysis, common variants (eg ACMG/ACOG guidelines) (when intron 8 poly-T analysis is performed in conjuction with 81220 in a R117H positive patient, do not report 81224) 81221: > known familial variants 81222: > duplication/deletion variants 81223: >full gene sequence 81224: >intron 8 poly-T analysis (eg, male infertility)

Tier 2

• Examples:

• Not complete transparent – Amount of work

• Laboratories do not ‘self-assign’

• Inherited disease – 32% of our test menu

– 6% of test volume (genetic),

• Somatic disease – 38% of test volume

– 20% of test volume

• Costs reflected in level

• Codes may move to Tier 1 as become routine

81401: Level 2 (eg, 2-10 SNPS, 1 methylated variant or 1 somatic variant (typically using nonsequencing target variant analysis) or detection of a dynamic mutation disorder/triplet repeat) ABL (c-abl oncogene I, receptor tyrosine kinase) (eg, acquired imatinib resistance). T315I variant 81407: Level 8 (eg, analysis of 26-50 exons by DNA sequence analysis, mutation scanning or duplication/deletion variatns of >50 exons, sequence analysis of multiple genes on one platform) > F8 (coagulation facotr VIII) (eg hemophilia A), full gene sequence

Unlisted: 81479 • Examples:

• Blood antigens • IL28B • Translocations

• BCL-2/JH (t(14;18), API2-MALT1 t(11;18), SYT-SSX, t(X;18),

• Rare genetic diseases • mitochondrial genome • Exome

• Inherited disease • 31% of test menu • 4% of test volume

• Somatic disease • 7% of test menu • 14% of test volume

• Percentages may differ significantly for specialty laboratories

38

Challenges in Fitting Codes to Existing Tests

• Descriptions (e.g) are worded to not need an exact fit

• Description of one variant, lab does several

– Laboratories not recompensed for extra variant(s)

• Description includes several variants, lab does one

– Concern of misrepresenting

– Tier 1 APC known familial variants

81220: CFTR (cystic fibrosis transmembrane conductance regulator) (eg, cystic fibrosis) gene analysis, common variants (eg ACMG/ACOG guidelines) 81201: APC (adenomatous polyposis coli) (eg, familial adenomatosis polyposis (FAP), attenuated FAP) gene analysis, full gene sequence 81202: > known familial variants

39

Challenges

• Combined analysis

– Two samples for comparison purposes but one code – Bone marrow transplantation

monitoring

– Maternal cell contamination

– Need to track different samples under one order

– Change the way it is ordered?

81265: Comparative analysis using Short Tandem Repeat (STR) markers: patient and comparative specimen (eg, pre-transplant recipient and donor germline testing, …..) + 81266 each additional specimen 81267 Chimerism (engraftment analysis, post transplantation specimen….. Without cell seletion 81268 > will cell selection (eg CD3, CD33) each cell type

40

DNA Extraction

• Technology/methods had separate code for DNA extraction

• Included in the molecular pathology code

• Laboratories receive blood, tissue etc. to extract for future testing

– Laboratories act as an informal DNA banking facility

41

Family specific mutation codes

• Tier 1

– 81221 CFTR Known familial variants

– 81293 MLH1 Known familial variants

– 81303 MECP2 Known familial variant

• One code for genes listed in tier 2

• Unlisted

• Our laboratory has one test number for all FSM

– Add gene to use

42

Next Generation Sequencing

• Methods code not available

• Oncology/inherited/infectious disease

• Gene panels, exomes, genomes

• Intepretation

Example: Cancer Panel • 50 genes

• 2855 Mutations

• codes: – 81210 (BRAF)

– 81235 (EGFR)

– 81245 (FLT3)

– 81270 (JAK2)

– 81275 (KRAS, ex 12-13)

– 81310 (NPM1)

– 81402 (MPL)

– 81403 x5 (ABL1, HRAS, IDH1, IDH2, KRAS ex 3)

– 81404 x6 (FGFR2, FGFR3, KIT, NRAS, PDGFRA, RET)

– 81405 (TP53)

– 81479 x1 (the rest)

ABL1 AKT1 ALK APC ATM BRAF CDH1 CDKN2A CSF1R CTNNB1 EGFR ERBB2 ERBB4 EZH2 FBXW7 FGFR1 FGFR2 FGFR3 FLT3 GNA11 GNAQ GNAS HNF1A HRAS IDH1 IDH2

JAK2 JAK3 KDR KIT KRAS MET MLH1 MPL NOTCH1 NPM1 NRAS PDGFRA PIK3CA PTEN PTPN11 RB1 RET SMAD4 SMARCB1 SMO SRC STK11 TP53 VHL

Or 81479

44

• Group of connective tissue disorders that include aneurysms, dilation, dissections/ruptures, and tortuosity of the aorta.

• Syndromic and familial connective tissue disorders caused by mutations in multiple genes – Marfan Syndrome

– Loeys-Dietz Syndrome

– Ehler Danlos Syndrome Type IV

– Arterial Tortuosity Syndrome

– Thoracic Aortic Aneurysms & Dissections

Cummings et al., 1998 JBJS

Aortopathies

Courtesy of Dr. Pinar Bayrak-Toydemir

45

• 66 exons

• Mutation Positive ( 10% positivity rate)

– Includes known pathogenic and suspected pathogenic

– 56%: diagnosis based on clinical phenotype

– 44%: suspected diagnosis of Marfan disease

• 4% Variants of Uncertain Clinical Significance

– 64%: Suspected diagnosis of Marfan

– 37%: Diagnosis based on clinical phenotype

Marfan Single Gene Assay

p. Arg545Cys

Aortopathies Gene Panel Gene Protein Disease Exons

FBN1 Fibrillin AD 66

TGFBR1 Transforming growth factor, beta receptor 1 AD 9

TGFBR2 Transforming growth factor, beta receptor 2 AD 7

COL3A1 Collagen, type III, alpha 1 AD 51

MYH11 Myosin, heavy chain 11, smooth muscle AD 41

ACTA2 Actin, alpha 2, smooth muscle, aorta AD 9

SLC2A10 Solute carrier family 2 (facilitated glucose transporter), member 10

AR 5

SMAD3 Mother against decapentaplegic homolog 3 AD 9

MYLK Myosin light chain kinase AD 34

FBN2 Fibrillin 2 AD 71

TOTAL 302 Courtesy of Dr. Pinar Bayrak-Toydemir

47

Assay Validation

• NGS assay of 10 genes

• Confirmed positives by Sanger sequencing

• Identified mutations in FBN1, FBN2, SMAD

– FBN1: c.7111T>A; p.Trp2371Arg (pathogenic)

– FBN1: c.1585C>CT, p.R529X (pathogenic)

– SMAD3 c.1229G>T, p.Val410Gly (suspected pathogenic)

– FBN2 c.T1644G, p.D548E (VUS)

Courtesy of Dr. Pinar Bayrak-Toydemir

48

Utility of NGS Gene Panel Assays

• Ideal for syndromes with overlapping phenotypes or genetically heterogeneous disorders:

– Marfan Syndrome

– Loeys-Dietz Syndrome

– Ehler Danlos Syndrome Type IV

– Arterial Tortuosity Syndrome

– Thoracic Aortic Aneurysms & Dissections

• Ideal for patients:

– Eliminate repeated or non-essential medical evaluations

– Eliminates time and cost of sequential sequencing of genes

Courtesy of Dr. Pinar Bayrak-Toydemir

Coding for Aortopathies Genes (current codes)

FBN1 (81408 tier 2)

TGFBR1 (81405 tier 2)

TGFBR2 (81405 tier 2)

COL3A1

MYH11

ACTA2

SLC2A10

SMAD3

MYLK

FBN2

81479 unlisted x1

Or 81479

50

Association for Molecular Pathology GSP Codes

• Proposal to address CPT coding for Genomic Sequencing Procedures (March 2013)

– Association for Molecular Pathology Economic Affairs Committee • Jeff Kant, former chair, Jan Nowak, co‐chair, Aaron Bossler,

co‐chair, Dara Aisner, Sam Caughron, Jill Hagenkord, Roger Klein, Elaine Lyon, Paul Raslavicus, Linda Sabatini, Michelle Schoonmaker, Ester Stein, Kathryn Tynan

• http://www.amp.org/documents/AMPProposaltoAddressCodingforGenomicSequencingProcedures_FINAL.pdf

51

NGS Considerations

• Panels: gene content

• Exomes: used as gene panel, 2nd analysis of exome if 1st analysis not informative

• Genomes: Analzyed as panel or exome, 2nd analysis of genome if 1st analysis not informative

• Symptom-guided analysis vs incidental findings

• Coding for re-analysis/re-intepretation?

52

Proposed Strategy

• Level of work

– targeted multiple gene sequencing

– exome sequencing

– Genome sequencing

• Descriptor

– clear indication of the clinical question to be addressed

• Report and Interpretation

• Re-analysis

53

Inherited Disease Gene Panel

• GSAX3:

– Aortic dysfunction or dilation (eg, FBN1, TGFBR1, TGFBR2, COL3A1, MYH11, ACTA2, SLC2A10, SMAD3, MYLK, FBN2) (eg, Marfan syndrome, Loeys Dietz syndrome, Ehler Danlos syndrome type IV, arterial tortuosity syndrome),genomic sequence analysis

• GSAX4:

– Report and interpretation

54

Quantitative Genomic

• QGSAX1:

– Fetal chromosomal aneuploidy (eg, chromosome 13, 18, 21) (eg, trisomy 21, trisomy 18 and trisomy 13) from circulating cell‐free fetal DNA, genomic sequence analysis

• QGSAX2:

– Report and interpretation

55

Exome Genomic Sequence Analysis

• GSAX90:

– Exome (eg. Unexplained constitutional or heritable disorder or syndrome) genomic sequence analysis

• GSAX91:

– Re‐analysis for unrelated condition or syndrome

• GSAX92:

– Report and interpretation

56

Genome Genomic Sequence Analysis

• GSAX96:

– Genome (eg, unexplained constitutional or heritable disorder or syndrome) genomic sequence analysis

• GSAX97:

– Re‐analysis for unrelated condition or syndrome

• GSAX98:

– Report and interpretation

Requesting Codes

• Request cpt code from AMA

– http://www.ama-assn.org/ama/pub/physician-resources/solutions-managing-your-practice/coding-billing-insurance/cpt/applying-cpt-codes.page • Molecular Pathology Procedures/Multianalyte Assays with Algorithmic

Analysis (MAAA)

• Information requested

– FDA-cleared?

– Number of laboratories performing test

– Test volume

– Clinical efficacy and utility

– Specify recommended terminology

• Review

58

Reimbursement

• Unlisted code not equivalent to denial or under payment

• CPT codes do not ensure reimbursement

– Received notification that some pharmacogenetic applications (cytochrome enzymes) will not be covered

– Others (CFTR, Long QT) require pre-authorization

59

Patient Access to Lab Results

• “CLIA Program and HIPAA Privacy Rule; Patients’ Access to Test Reports” (September 14, 2011) – Permit laboratories, upon a patient’s request, to

provide access to competed test reports • Further engages patients in their own care

– Pre-empts any State laws to the contrary

– Does not prescribe the request process (relies upon the HIPAA authorization process) • Failure to authenticate excepts labs from compliance

Association for Molecular Pathology 9650 Rockville Pike

Bethesda, MD 20814

AMPEducation@amp.org

www.amp.org

© Association for Molecular Pathology, 2013