Emerging diagnostic technologies proving the clinical application through gold standard data...

Emerging Diagnostic Technologies:

Proving The Clinical Application

Through Gold Standard Data

Lyssa Friedman

1Lyssa Friedman3/25/14

OutlineAsking the right questions: a stair-step approach to product development and selecting the right technical platform

Focus on novel diagnostics

Key components for success

Case studies

The role of clinical operations

3/25/14 Lyssa Friedman 2

Next Generation Sequencing:

The right tool for the diagnostic job?


Next Generation Sequencing (NGS)

2013


Nho, K. et al. Whole-exome sequencing and imaging

genetics identify functional variants for rate of change

in hippocampal volume in mild cognitive impairment

Braggio E, et al. Lessons from next-

generation sequencing analysis in

hematological malignancies

Mille FA, et al. Testing personalized

medicine: patient and physician

expectations of next-generation genomic

sequencing in late-stage cancer care

NGS in the news

3/25/14

Lyssa Friedman

5

Mar 17 2014

Mar 18 2014

Jan 14 2014 (Regeneron)

Mar 8 2012 (Complete Genomics)

Jan 26 2014

Oct 24 2013

NGS in the regulatory landscape


First high-throughput DNA sequencing analyzer

to receive U.S. Food and Drug Administration

(FDA) premarket clearance

MiSeqDxNov 2013

http://www.illumina.com

NGS in the clinic


• Indication: thyroid cancer

• ThyroSeq – 12 genes

• Ion AmpliSeq™

• Cystic fibrosis (CF), two products:

• CF 139-Variant Assay – 139 variants

• CF Clinical Sequencing Assay – additional variants

• Prenatal testing: verifitest – 7 fetal chromosomal abnormalities

• MiSeqDx

• Two products/indications:

• NeoTYPE MDS/CMML Profile – 16 genes, myelodysplastic

syndrome/chronic myelomonocytic leukemia (MDS/CMML)

• Solid tumor profile – 48 driver genes, multiple tumor types

• 7 common KRAS mutations, multiple cancer types

+ Laboratory Developed Tests (LDTs)

+ New platform for existing clinical indication

± Lower cost of goods sold (COGS)

Back to basics: What is the right tool for the job?

Honda Civic Jaguar XJ

Does the job for the price ✚ ✚

Safe and compliant ✚ ✚

Sexy and stylish ✚

The next best thing ✚

Because I can afford it ✚


Othertechnologies

NGS

✖

Choose the technical platform that suits the

business, clinical and performance needs

The tail wagging the dog

Broad clinical question

Target biomarkers

Selected platform

Business case

Clinical need

Reimbursement strategy

Commercial strategy


Avoid retrofitting the business, clinical, reimbursement and

commercial strategies to an early platform or assay decision

Instead, follow a stair-step approach


Business case

Clinical assessment

Biomarker

discovery

Risk assessment

•Technical

•Regulatory

Technical platform

and assay selection

Product

development

Product

launch

Setting the stage for successful product

development and launch


Define

• Market

• Unmet clinical need

• Gold standard Conduct early discussions

• Regulatory strategy

• Commercial strategy

• Reimbursement strategy Stay focused

• Clinical indication

• Discovery or in-licensing plan Remain agnostic regarding technical platform and assay requirements –

choose based on

• Clinical and market needs

• Test performance requirements

• Regulatory risks

• Technical risks

• COGS Allow the chosen technical platform and assay to serve the clinical,

business, commercial and reimbursement case

Consider these key components early

Gold standard

Regulatory strategy

Study design

Publication strategy

Reimbursement strategy


LDT vs FDA


LDT FDA

Companion diagnostics✔

Kit for distribution✔

Sole-source CLIA lab✔ ✔

Define the gold standard

• How do key stakeholders define the gold standard?– Physician community

– Literature/guidelines

– FDA

– Payers

• Is local diagnosis sufficient or is “expert” diagnosis needed? For expert diagnosis:– Individual vs. panel

– Blinded vs. conference

– Expert name recognition

• Apply gold standard label early in the discovery process – avoid waiting for development phase – Risk that labels will change changes in performance


Publications drive reimbursement

Well-designed studies with pre-defined

• Samples

• Clinical annotation

• Gold standard label

• Performance requirements

High-impact peer-reviewed publications

Positive coverage decisions


Studies Publications ReimbursementThe ACCE Evaluation Process for Genetic Testing


http://www.cdc.gov/genomics/gtesting/EGAPP/index.htm

http://www.cdc.gov/genomics/gtesting/ACCE/index.htm

• ACCE Model Project:

• Analytic validity

• Clinical validity

• Clinical utility

• Ethical/legal/social

• Publications in these areas

help build the reimbursement

case

Centers for Disease Control (CDC) Office of Public Health Genomics

Evaluation of Genomic Applications in Practice and Prevention (EGAPP™) Initiative

What does success look like?

Case Studies


Three success stories


Publications

LDTMulti-

gene

Novel/

licensedNGS?

Clinical

validation

Analytic

verification

Clinical

utility

Cost

effectiveness

Genomic

Health

OncotypeDx✔ ✔ Novel No ✔ ✔ ✔ ✔

Veracyte

Afirma Thyroid

FNA Analysis✔ ✔ Novel No ✔ ✔ ✔ ✔

Illumina

verifi prenatal

test✔ ✔ Licensed Yes ✔ ✔ ✔ ✔

Cancer assays

Risk score (continuum)


•Samples: formalin-fixed paraffin-embedded tissue

•Platform: Reverse transcription polymerase chain reaction (RT-PCR),

mRNA

•Gold standard: centralized pathology

•Regulatory: CLIA LDT

Breast cancer

Risk of

recurrence

21 genes

Colon cancer

Risk of

recurrence

12 genes

Prostate cancer

Disease risk

assessment

17 genes

Bast RC & Hortobagyi. N Engl J Med 2004

Genomic Health Company Timeline


FFPE: formalin-fixed paraffin-embedded; RT-PCR: reverse transcription polymerase chain reaction; NSABP: National Surgical Adjuvant

Breast and Bowel Project; NCI: National Cancer Institute; ASCO: American Society of Clinical Oncology; SABCS: San Antonio Breast Cancer

Symposium; NEJM: New England Journal of Medicine; NCCN: National Comprehensive Cancer Network; DCIS: ductal carcinoma in situ

• Series A

• RNA extraction

protocol

• TaqMan (RT-PCR)

• Proof of concept

• Series B&C

• Commercial

strategy

• Series D

• ASCO,

SABCS

• Clinical

validation

• CLIA lab

• Breast assay

launch

• Clinical

validation

publication

(NEJM)

• Economic/cost

effectiveness

publication

• IPO

• Medicare

• Aetna

• Colon assay

development

• United

Healthcare

• NCCN

guidelines

• Colon assay

clinical

validation

• Breast assay

clinical utility

publications

• Colon assay

launch

• Social media

campaign

• Prostate

program

• Smartphone

app

• Breast NGS

data

• DCIS test

launch

• Prostate

clinical

validation

• NGS in

development

2000 ‘01 ‘02 ‘03 ‘04 ‘05 ‘06 ‘07 ‘08 ‘09 ‘10 ‘11 ‘12 ‘13

• Prostate

launch

http://www.genomichealth.com, http://www.oncotypedx.com

Suspicious thyroid neoplasm

Benign vs continued suspicious

result


Thyroid neoplasm with

indeterminate cytopathology

142 genes

•Samples: RNA-preserved fine needle aspirate biopsy

•Platform: Custom microarray (Affymetrix), mRNA

•Gold standard: centralized pathology

•Regulatory: CLIA LDT on FDA-cleared microarray platform

Alexander EK, et al. N Engl J Med 2012

Veracyte Company Timeline


• Series A

• RNA extraction

protocol

• Clinical validation

data


publication

• Series B

• Thyroid benign classifier

launch

• Cost effectiveness

publication


publication (NEJM)

• Analytic verification

publication

• Clinical utility

publication

• Medicare

• NCCN guidelines

• 2nd clinical utility

publication

• Gene discovery IPF

• United Healthcare

• Aetna

• IPO

2008 2009 2010 2011 2012 2013 2014

• Thyroid

malignancy

classifier launch

NCCN: National Comprehensive Cancer Network; IPF: idiopathic pulmonary fibrosis

http://veracyte.com, http://afirma.com

Fetal chromosomal abnormalities


(Verinata)

Blood screen

7 fetal chromosomal abnormalities

•Samples: maternal blood

•Platform: massively parallel sequencing (MPS) of maternal plasma

cell-free DNA (cfDNA)

•Gold standard: previously existing and accepted prenatal aneuploidy

screening

•Regulatory: CLIA LDT on FDA-cleared DNA sequencing analyzer

Bianchi DW, et al. N Engl J Med 2014

Verinata Company Timeline


• Stanford license


publication

• Clinical utility publication

• Name change (fr. Artemis

Health)

• Series C

• Launch

• Product expansion

• Cost effectiveness

publication

• Fundraising

• Series D

• Acquisition (Illimuna)

2009 2011 2012 2013 2014


publication (NEJM)

http://www.illumina.com, http://www.verifitest.com

Poised for success:

Stay laser-focused on goals

Take a stair-step approach

Novel diagnostics


GoalsStaying on target is a continuous process


Clearly define

•Success

•Milestones

•Timelines

•Risks Stay

•Focused

•Flexible

•Nimble Communicate with transparency

•Speedbumps

•Roadblocks

•Brick walls Mitigate risk

Question assumptions (avoid drinking the Kool-Aid)

Learn quickly from failures

Successful companies choose the technical platform

that suits business, clinical and test performance needs


Business case

Clinical assessment

Biomarker

discovery

Risk assessment

•Technical

•Regulatory

Technical platform

and assay selection

Product

development

Product

launch

While managing many of these processes

non-sequentially and simultaneously


Business case

• Market

– Who are your key customers?

– What is the clinical indication?

– Where is your geographical target over the next 3 years? 5 years?

10 years?

– What is the payer landscape?

• Early consideration of risks

– Who are your investors?

– What return do your investors expect on their investment in 3 years?

5 years? 10 years?

– Who are your competitors?

– What is the regulatory landscape? Potential minefields?

– What is the technical landscape? Potential minefields?

• Performance requirements and gold standard– What does the market need in order to trust the test

• Equipment and reagents– How competitive is the space? FDA-cleared vs.

research use only (RUO)? How will this affect• Supply agreements: cost, room to negotiate

• Reagents: cost, availability, stability, need for verification

• Software verification and validation (V&V)– What capital purchases and internal resources are

needed?

– How will V&V affect time and cost?


Business case, cont’d

• Unmet clinical need

– What is the unmet clinical need?

– What are the pain points? How can you ease your

physician customer’s pain?

– What is the current clinical workflow? How can you

minimize disruption of that workflow?

– What will it take to engender physician trust in your

test? Performance? Gold standard? Publications?

Guidelines? Influential KOLs? FDA-clearance?


Clinical assessment

• Gold standard

– Is there label consistency from discovery through

development?

– Is there stakeholder buy-in for the gold standard?

• Physician community

• Literature/guidelines

• FDA

• Payers


Clinical assessment, cont’d

• Target sample: lessons learned from prospective

sample collection

– How are they collected and handled?

– Where are they collected? Clinic, radiology, pathology?

– What is the impact on sample collection on the patient?

Special procedures, extra visits, discomfort, anesthesia?

– How disruptive will collection and handling be to current

workflow?

– Will special training be needed?

– Will special storage facilities be needed?

– How many specialties will need to be involved? Clinic

physician, radiology, pathology?



• KOLs – how are they selected and engaged?– Protocol development

– Development of key clinical questions

– Publications

– Guideline committees

– Influence: US/ex-US

– Study participation

• Non-KOL investigators– Peer influence

– Early adopters

– Patient population

– Clinic access - workflow



• Clinical utility: so what?

– Patient outcomes

– Societal cost-effectiveness

– Savings to payers

• Performance requirements

• Publication strategy

• Reimbursement strategy



Disease

Test Present Absent Total

PositiveTrue positive

A

False positive

BA + B

NegativeFalse negative

C

True negative

DC + D

Total A + C B + D A + B + C + D

SensitivityA

A+CProbability test positive when disease present

SpecificityD

B+DProbability test negative when disease not present

Positive predictive value

PPV

A

A+BProbability disease present when test positive

Negative predictive value

NPV

D

C+DProbability disease absent when test negative

PrevalenceA+C

A+B+C+D

Performance requirements: what type of test?


Performance requirements: things to

think about

• Will the test performance lead to actionable patient results?

– What difference does the test make?

• What performance will the community accept?

– What are the barriers to uptake? How much education will the community need in order to understand test performance?

• How well does the gold standard perform?

– The test cannot outperform the gold standard


Publication and reimbursement strategy


Publications

Protocol

Clinical validation study

Recognized gold standard

Independent test set

Clinical

validity

Analytic

validity

Clinical

utility

Clinical

guidelines

Positive

coverage

decisions

Samples

Discovery

Samples

Study planning

Samples

Analytic testing

Samples

Validation


•Simultaneous and interdependent processes

•Continuously re-assess risks and re-focus priorities

throughout the continuum

Processes are not sequential

• Use same gold standard from discovery through development

– Changes can affect test performance

• Continue other activities in parallel

– Stability and kit testing

– Planning: validation, product development, reimbursement, publication and commercial strategies

• Ongoing and continuous risk assessment with focused and aggressive timelines

– Building the bicycle while riding it (downhill at 60 mph!)


Biomarker discovery

• Will platform change from discovery to

product development? If so, anticipate

changes, with associated delays, to

– Gene set

– Equipment and reagents

– Algorithm development

– Test performance


Technical risk assessment

• LDT vs FDA clearance– LDT

• CLIA lab: buy or build?

• Equipment

• Documentation

• Staffing

– FDA• Documentation – design control

• Early meetings with agency

• Kit and partnership strategies

• Equipment and reagents– FDA-cleared vs RUO


Regulatory risk assessment

• Clearer platform decision at this stage with

a stair-step approach

– FDA-cleared vs. RUO

– Reagents

– Cost of goods

– Regulatory pathway


Technical platform and assay selection

Technical platform: you get what you

pay for


DataDevelopment

costs

What assay and technical

platform?


RT-PCR: reverse transcriptase polymerase chain reaction; FISH: fluorescent in

situ hybridization; SKI; spectral karyotype imaging; cDNA: complementary DNA

Genomic information Indications

PCRMultiple copies of targeted DNA

sequenceLarge DNA targets

RT-PCR Amplified RNARNA viruses

mRNA or large DNA translocations

qPCR Real-time PCRReal-time quantification of multiple

copies

FISHOligonucleotide probes attached to

cDNA targetOncogene amplification, translocations

Microarrays Thousands of transcripts and probes

Gene expression “fingerprints”

Single nucleotide polymorphisms

(SNPs)

Sequencing Precise order of nucleotides within DNAMolecular biology, evolutionary biology,

metagenomiccosts

Netto GJ, et al. Proc (Bayl Univ Med Cent) 2003

NGS: things to think about

• Equipment is costly – can it be leveraged across projects in lab?

• Computational challenges

– Sequencing produces raw data requiring assembly into longer sequences

– Sequences may not be assigned to specific chromosomes

• When transferring existing assay to new platform full validation is needed – technology transfer not sufficient


• Transfer assay from R&D – conduct studies on platform to be used commercially

• LDT: conduct studies in CLIA lab by approved personnel– Stability – collection, handling transport

– Analytic sensitivity – input, dilution

– Analytic specificity – interfering substances

– Assay robustness

– Reproducibility

– Limits of detection

– Quality control


Product development

• Clinical sites = early adopters

• Partner with commercial group for

knowledge transfer

– Customer pain points and workflow

– Nuances of patient population and physician

specialty

– Key contact people at customer sites


Product launch

Clinical operations:

An integral part of the whole


Clinical operations: not just for accruing

samples…

Sample accrual

Pristine annotation

Gold standard labels

Thorough data management and data auditing

Prospective protocols conducted according to Good Clinical Practices


…We are the first ear to the voice of the

customer and thus a key to success


• Articulate the clinical question– Unmet need

– Pain points

– Workflow challenges

• Define the gold standard

• Grow influential KOL relationship, which are critical to– Publications

– Physician uptake

– Guideline acceptance

• Cultivate PI/site relationships: early commercial adopters

• Develop critical protocols

• Assess and mitigate risk

• Define clinical utility

• Publish abstracts and manuscripts that drive guideline and

physician uptake and reimbursement

Questions?

Lyssa Friedman

[email protected] / 415-250-8356

mailto:[email protected]

Emerging diagnostic technologies proving the clinical application through gold standard data...

Health & Medicine

Transcript of Emerging diagnostic technologies proving the clinical application through gold standard data...