Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation...

27
Clinical-Grade Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic Labs Oregon Health & Science University Portland, OR

Transcript of Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation...

Page 1: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Clinical-Grade Next-Generation Sequencing:

Assay Design & Validation

Richard D. Press, MD, PhDDept of Pathology

Knight Cancer InstituteKnight Diagnostic Labs

Oregon Health & Science UniversityPortland, OR

Page 2: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

NGS Assay Design ConsiderationsCustomization is Key

• Coverage: medium, large, or X-large

• If not whole exome / genome, then:– Which genes to sequence? – Which regions?

• What cancers to cover?– Organ-specific?– Carcinoma vs sarcoma vs

hematopoeitic?– All cancers?

What size are you?

Page 3: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Assay Validation Questions(I don’t have the answers)

• How to validate all known mutation possibilities?

• Tumor cellularity requirements?• Minimal mutant allele burden detection?• Quantitative or Qualitative reporting?• Quality control: read depth? Other

parameters?• Unknown variants?• Many other questions and variables

Page 4: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

What’s All the Fuss About NGS?- Broader coverage, including tumor suppressors- Better sensitivity, through deeper reads- ? Lower costs, through multiplexing- More comprehensive cancer genome characterization for

targeted therapeutic (and diagnostic) discovery

Single geneassays

ABI

Multiplexedhotspots

Multigenepanels

Ion Torrent PGMSequenom MassArray

Wholeexome

454

Wholegenome

Illumina HiSeq

Page 5: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Targeted Therapeutics

SHC GRB2SOS

NRASKRAS

BRAF

MEK

ERKPSTAT

P

PP

P

P PP

PP

P

PI3KPDK

AKT

P

mTOR

S6K

P

P

P

PTEN

Receptor tyrosine kinases

p53

ALK

ErlotinibLapatinibPF299804AfatinibImatinib RAF265

Vemurafenib

AZD6244PD0325901ARRY162

EverolimusTemsirolimus

BEZ235

BKM120BGT226BYL719

Crizotinib

MK2206SR13668

Page 6: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Coverage: How High Do You Want to Go?

• Mutation Hotspots• Disease-Specific

Gene Panels• Generic “Cancer”

Gene Panels• Whole Exome• Whole Genome

Higher Coverage means:• More complexity & cost• More unknown variants• ? Overkill for clinical

care: who cares if its not drugable?

Page 7: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Ion Torrent PGM

• Moderate throughput• Massively parallel 3rd

generation sequencing• Performed on a semi-

conductor chip

Page 8: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Ion Torrent PGM

Each well is a tiny solid-state pH meter

Page 9: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Library Preparation:Hybridization-Capture Approaches

NimblegenAgilentIon Torrent

50 to 1,000 ngGenomic DNA

Shear, End Repair,Ligate adapters

Gel purify at ~180 bp

Add biotinylated probesto genes/exons of interest

b

b

b

Purify hybridizedRNA probes with magnetic beads

b

b

Treat with NaOHto remove RNA Emulsion PCR

Page 10: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Amplify gDNA targets190-Amplicon multiplex PCRIon AmpliSeqTM Cancer Primer Pool

Remove genomic DNA template and primersPartially digest primers

Ligate barcode adaptersNick-translate and amplify

Genomic DNA

A BC

P1

A BC P1

Emulsion PCREnrichment of templated IonSphere particlesSequence

Ampliseq primer pool

Emulsion PCR

NGS Library Prep

Amplicon-Based Approach

Page 11: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

AmpliSeq Cancer Panel• 739 hotspots covered by 190 amplicons• Single tube amplification • Average amplicon length: 119 bp (100-169 bp)• Input DNA: 10 ng (Fresh or FFPE)• Turn-around time: 48 hours • 46 genes:

ABL1, AKT1, ALK, APC, ATM, BRAF, CDH1, CDKN2A, CSF1R, CTNNB1, EGFR, ERBB2, ERBB4, FBXW7, FGFR1, FGFR2, FGFR3, FLT3, GNAS, HNF1A, HRAS, IDH1, JAK2, JAK3, KDR, KIT, KRAS, MET, MLH1, MPL, NOTCH1, NPM1, NRAS, PDGFRA, PIK3CA, PTEN, PTPN11, RB1, RET, SMAD4, SMARCB1, SMO, SRC, STK11, TP53, VHL

Page 12: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Mass Spectrometry-Based Detection of Genomic Mutations

PCR targets of interest~ 100 bp amplicons

Clean-up steps

Primer extension reaction

De-salting step

MALDI-TOF Mass spectrometry

AT

GC

TddA

CddG

G A

Page 13: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Mass spec profile for a 9-plex reaction

Multiplex of 9 assays

PIK3CA E542K wt 542Emutant 542Kunextendedprimer

Page 14: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Mass Spec Leukemia Panel: 370 mutations / 31 genes

ABL FLT3 KRASAKT1 FMS METAKT2 GATA1 MPLAKT3 HRAS NOTCH1BRAF IDH1 NPM1CBL IDH2 NRASCBLB JAK1 NTRK1FBXW7 JAK2 PAX5FES JAK3 PDGFRBFGFR4 KIT PTPN11

SOS1

Page 15: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Mutation Spectrum in AML (108 OHSU cases)

Normal cytogenetics: 78% mutation frequencyAbnormal cytogenetics: 43% mutation frequency

Mutation discovery with 31-gene mass spec panel plus single gene in-del assays (FLT3, CEBPA, KIT)

J Dunlap, in press

Page 16: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Ampliseq Validation Study

• 45 FFPE tumor DNA samples with known mutations previously quantitated on Sequenom MassArray (mass spectrometry)– 53 point mutations– 19 in/dels (range 4 - 63 bp)

• 7 unmatched FFPE normal tissue DNA• 100 bp single-end sequencing runs

– 22 samples run singly on 314 chips– All samples run as 4-plexes on 316 chips

Page 17: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

TP53 R273H

PG1-48

Colonadenoc.

• General background is low• Some homopolymers (e.g. a run of C’s) lead to false positive calls

Page 18: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Read Coverage Distribution:190 Amplicons in each of 45 Tumor Samples

(Normalized to 400,000 reads)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0

1

2

3

4

5

6

7

8

9

10

0-99

100-

199

200-

299

300-

399

400-

499

500-

599

600-

699

700-

799

800-

899

900-

999

1000

-109

911

00-1

199

1200

-129

913

00-1

399

1400

-149

915

00-1

599

1600

-169

917

00-1

799

1800

-189

919

00-1

999

2000

-209

921

00-2

199

2200

-229

923

00-2

399

2400

-249

925

00-2

599

2600

-269

927

00-2

799

2800

-289

929

00-2

999

3000

-309

931

00-3

199

3200

-329

933

00-3

399

3400

-349

935

00-3

599

3600

-369

937

00-3

799

3800

-389

939

00-3

999

4000

-409

941

00-4

199

4200

-429

943

00-4

399

4400

-449

945

00-4

599

4600

-469

947

00-4

799

4800

-489

9

Perc

enta

ge o

f tar

gets

≥ re

ad d

epth

Num

ber o

f tar

get r

egio

ns w

ith re

ad d

epth

Read depth

Average = 1948 reads

•95% of amplicons average >400 reads •91% of amplicons >650 reads (estimated 5% sensitivity) • 78% of amplicons with >1200 reads (estimated 1% sensitivity) Beadling, submitted 2012

Page 19: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Sequencing Performance

• 4-plex samples run on 316 chips (6.2 million wells)– Avg 4 million beads loaded– Avg 3.7 million beads had library templates (92%)– Avg 1.7 million beads yielded quality sequence

• For individual samples– Avg 428,000 reads/sample (range 178K - 710K)– Mean read length 76 bp– On-target reads: >95%

• Across all samples (normalized to 400K reads)– Avg 1,941 reads per amplicon – 95% of amplicons with > 400 reads

Page 20: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

What is the Optimal Mutant Allele Burden Cutoff?53 known point mutations in 45 tumor samples

All 53 “detected” by both NGS & Mass Spectrometry

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

0.00 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 0.90 1.00

True

pos

itive

rate

(sen

sitiv

ity)

False positive rate (1-specificity)

8% 1% variant allele5%

10%15%

50%

Receiver Operator Characteristic (ROC) Analysis

If lower limit of detection set at 8% mutant allele:- 100% Sensitivity- 94% Specificity

But are these few discordants really “false positives”?Or variants missed by less sensitive methods?

Beadling, submitted 2012

Page 21: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Summary of Variants• All 53 known point mutations were identified by the

variant caller software• 26 new mutations were also identified

– APC in colon ca; PTEN in endometrial ca; STK11 in lung ca

• 19 in/dels were included in the analysis; range: 4-63 bp – 2 called exactly– 5 flagged as point mutations– 12 visible on manual inspection but not flagged

• 54 ‘variants’ turned up reproducibly in both tumor and normal DNA, likely reflecting sequencing aberrations

Page 22: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Is NGS Data Quantitative?Allele Ratios: MassArray vs AmpliSeq

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Red = 1:1 equivalence line

N = 53 mutationsSlope = 0.782Bias = minimal (3.4%) (Ampliseq lower than Sequenom)R = 0.92

Ion Torrent Ampliseq Allele Burden

Sequ

enom

Alle

le B

urde

n

(P < 0.0001)

Beadling, submitted 2012

Page 23: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

NGS Clinical Validation: Status Report (OHSU; May ‘12)Ampliseq 46 gene panel (multiplex PCR library

prep)– Analytical validation essentially complete– 100% sensitive compared to “gold standards”– Variants of unknown function? – Low-level variants?

• We have validated an 8% mutant allele threshold– Indels remain a challenge for variant caller

• New software any better?– Aiming for summer ‘12 launch in our clinical lab

Page 24: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

The Future: Disease-Specific Gene Panels

Cancer Site

Target genes

# Exons Kilobases Ampli-cons

New Genes(not in Ampliseq)

Lung 23 224 34.3 502 9

Colon 16 157 31.8 405 6

Melanoma 21 113 13.9 231 9

AML / ALL / MDS

42 342 62.6 863 28

• Ion Torrent custom primer design software used to design primers for library prep

• Proprietary primer modifications allow massive multiplexing: 2-4 PCR reactions per library prep

Page 25: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Many Unanswered Questions• How to re-validate assays given continuing

rapid pace of improvements to chemistries, hardware, and software?

• Sanger sequencing confirmation?• Unknown variants? • Matched normal tissue required?• Quality control? • Gene patent implications?• FDA? Can you image the approval process?• Will anyone pay us for this service?

Page 26: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

AcknowledgementsOHSU Knight Diagnostic Labs

Carol Beadling, PhDTanaya NeffAndrea WarrickFei YangJennifer Dunlap MDChris Corless MD, PhD

Page 27: Clinical-Grade Next-Generation Sequencing: … Next-Generation Sequencing: Assay Design & Validation Richard D. Press, MD, PhD Dept of Pathology Knight Cancer Institute Knight Diagnostic

Oregon Health & Science University