Use of Archived Tissue in Use of Archived Tissue in Evaluating the Medical Evaluating the Medical Utility of Prognostic & Utility of Prognostic & Predictive BiomarkersPredictive Biomarkers

Richard Simon, D.Sc.Richard Simon, D.Sc.Chief, Biometric Research BranchChief, Biometric Research Branch

National Cancer InstituteNational Cancer Institutehttp://brb.nci.nih.govhttp://brb.nci.nih.gov

Prognostic & Predictive Prognostic & Predictive BiomarkersBiomarkers

Most cancer treatments benefit only a minority Most cancer treatments benefit only a minority of patients to whom they are administeredof patients to whom they are administered

Being able to predict which patients are likely Being able to predict which patients are likely to benefit would to benefit would Save patients from unnecessary toxicity, and Save patients from unnecessary toxicity, and

enhance their chance of receiving a drug that helps enhance their chance of receiving a drug that helps themthem

Control medical costs Control medical costs Improve the success rate of clinical drug Improve the success rate of clinical drug

developmentdevelopment

Predictive & Prognostic Predictive & Prognostic BiomarkersBiomarkers

Predictive biomarkersPredictive biomarkers Measured before treatment to identify who will or Measured before treatment to identify who will or

will not benefit from a particular treatmentwill not benefit from a particular treatment ER, HER2, KRASER, HER2, KRAS

Prognostic biomarkersPrognostic biomarkers Measured before treatment to indicate long-term Measured before treatment to indicate long-term

outcome for patients untreated or receiving outcome for patients untreated or receiving standard treatmentstandard treatment

Used to identify who does not require more intensive Used to identify who does not require more intensive treatmenttreatment

OncotypeDxOncotypeDx

Prognostic and Predictive Prognostic and Predictive Biomarkers in OncologyBiomarkers in Oncology

Single gene or protein measurementSingle gene or protein measurement ER protein expressionER protein expression HER2 amplificationHER2 amplification KRAS mutationKRAS mutation

Index or classifier that summarizes Index or classifier that summarizes expression levels of multiple genesexpression levels of multiple genes OncotypeDx recurrence scoreOncotypeDx recurrence score

Types of Validation for Types of Validation for Prognostic and Predictive Prognostic and Predictive

BiomarkersBiomarkers Analytical validationAnalytical validation

Accuracy compared to gold-standard assayAccuracy compared to gold-standard assay Robust and reproducible if there is no gold-standardRobust and reproducible if there is no gold-standard Pre-analytical robustness to tissue handling and Pre-analytical robustness to tissue handling and

preparationpreparation Clinical validation/correlationClinical validation/correlation

Does the biomarker predict what it’s supposed to Does the biomarker predict what it’s supposed to predict for independent datapredict for independent data

Clinical/Medical utilityClinical/Medical utility Does use of the biomarker result in patient benefitDoes use of the biomarker result in patient benefit

Generally by improving treatment decisionsGenerally by improving treatment decisions Is it “actionable”Is it “actionable”

Prognostic Factors in OncologyPrognostic Factors in Oncology

Most prognostic factors are not used Most prognostic factors are not used because although they correlate with a because although they correlate with a clinical endpoint, they have no clinical endpoint, they have no demonstrated medical utilitydemonstrated medical utility They are developed in unfocused studies that They are developed in unfocused studies that

use convenience samples of heterogeneous use convenience samples of heterogeneous patients for whom tissue is availablepatients for whom tissue is available

They are not reliable because they are They are not reliable because they are exploratory and not prospectively focused on exploratory and not prospectively focused on a single factora single factor

Predictive BiomarkersPredictive Biomarkers

In the past often studied as un-In the past often studied as un-focused post-hoc subset analyses of focused post-hoc subset analyses of RCTs.RCTs. Numerous subsets examinedNumerous subsets examined No focused pre-specified hypothesisNo focused pre-specified hypothesis No control of type I errorNo control of type I error

Prospective Validation of the Prospective Validation of the Medical Utility of a Medical Utility of a PrognosticPrognostic

BiomarkerBiomarker Develop biomarker predictive of Develop biomarker predictive of

outcome for low stage patients outcome for low stage patients without chemotherapy. Thenwithout chemotherapy. Then

Select low stage patients who are Select low stage patients who are considered good prognosis by considered good prognosis by biomarkerbiomarker Randomize them to standard Randomize them to standard

chemotherapy or to no chemotherapychemotherapy or to no chemotherapy e.g. MINDACTe.g. MINDACT

If predicted risk of recurrence If predicted risk of recurrence without chemotherapy is sufficiently without chemotherapy is sufficiently low, then randomization might be low, then randomization might be omitted omited and chemotherapy omitted omited and chemotherapy withheldwithheld If their outcome is sufficiently good If their outcome is sufficiently good

without chemotherapy, absolute benefit without chemotherapy, absolute benefit of chemotherapy would be very smallof chemotherapy would be very small

TAILORxTAILORx

Prospective Evaluation of a Prospective Evaluation of a PredictivePredictive Biomarker Biomarker

Perform Test with Predictive Biomarker

Predicted Non-responsive to New Rx

Predicted ResponsiveTo New Rx

ControlNew RX Control

New RX

Prospective Evaluation of Prospective Evaluation of PredictivePredictive BiomarkerBiomarker

The RCTThe RCT

focused on a single pre-specified biomarkerfocused on a single pre-specified biomarker sized with sufficient marker + and marker – sized with sufficient marker + and marker –

patients for adequately powered separate patients for adequately powered separate analysis of T vs C differences analysis of T vs C differences

Evaluating a predictive biomarker does Evaluating a predictive biomarker does notnot involve comparison of outcome of involve comparison of outcome of marker + vs marker – patientmarker + vs marker – patient

The purpose of the RCT is not to re-derive The purpose of the RCT is not to re-derive or refine the definition of the biomarkeror refine the definition of the biomarker

R Simon. Using genomics in clinical trial R Simon. Using genomics in clinical trial design, Clinical Cancer Research 14:5984-93, design, Clinical Cancer Research 14:5984-93, 20082008

R Simon and A Maitournim . Evaluating the R Simon and A Maitournim . Evaluating the efficiency of targeted designs for randomized efficiency of targeted designs for randomized clinical trials. Clinical Cancer Research clinical trials. Clinical Cancer Research 10:6759-63, 2004.10:6759-63, 2004.

R Simon. Roadmap for developing and R Simon. Roadmap for developing and validating therapeutically relevant genomic validating therapeutically relevant genomic classifiers. Journal of Clinical Oncology classifiers. Journal of Clinical Oncology 23:7332-41, 2005.23:7332-41, 2005.

It may be infeasible or unethical to It may be infeasible or unethical to conduct a new prospective trial to conduct a new prospective trial to

test a hypothesis about a test a hypothesis about a prognostic or predictive biomarkerprognostic or predictive biomarker

For tests relating to the use of approved For tests relating to the use of approved therapeuticstherapeutics KRAS for anti-EGFR antibodies in colorectal cancerKRAS for anti-EGFR antibodies in colorectal cancer HER2 for doxorubicinHER2 for doxorubicin

TAILORx and MINDACT trials require many TAILORx and MINDACT trials require many thousands of patientsthousands of patients Genomic test strategy designs which randomize whether Genomic test strategy designs which randomize whether

to perform or not perform the test often require tens of to perform or not perform the test often require tens of thousands of patientsthousands of patients

In some cases the benefits of a In some cases the benefits of a prospective trial can be closely prospective trial can be closely achieved by the carefully planned achieved by the carefully planned use of archived tissue from a use of archived tissue from a previously conducted randomized previously conducted randomized clinical trial clinical trial

Use of Archived Specimens in Evaluation of Use of Archived Specimens in Evaluation of Prognostic and Predictive BiomarkersPrognostic and Predictive Biomarkers

Richard M. Simon, Soonmyung Paik and Daniel F. HayesRichard M. Simon, Soonmyung Paik and Daniel F. Hayes

Claims of medical utility for prognostic and Claims of medical utility for prognostic and predictive biomarkers based on analysis of archived predictive biomarkers based on analysis of archived tissues can be considered to have either a high or tissues can be considered to have either a high or low level of evidence depending on several key low level of evidence depending on several key factors. factors.

Studies using archived tissues, when conducted Studies using archived tissues, when conducted under ideal conditions and independently confirmed under ideal conditions and independently confirmed can provide the highest level of evidence. can provide the highest level of evidence.

Traditional analyses of prognostic or predictive Traditional analyses of prognostic or predictive factors, using non analytically validated assays on a factors, using non analytically validated assays on a convenience sample of tissues and conducted in an convenience sample of tissues and conducted in an exploratory and unfocused manner provide a very exploratory and unfocused manner provide a very low level of evidence for clinical utility. low level of evidence for clinical utility.

For Level I EvidenceFor Level I Evidence

Archived tissue adequate for a successful assay must be Archived tissue adequate for a successful assay must be available on a sufficiently large number of patients from a available on a sufficiently large number of patients from a phase III trial with a design that enables the appropriate phase III trial with a design that enables the appropriate analysesanalyses e.g. For predictive marker, RCT comparing T to Control e.g. For predictive marker, RCT comparing T to Control Adequate statistical powerAdequate statistical power The patients included in the evaluation are clearly The patients included in the evaluation are clearly

representative of the patients in the trial. representative of the patients in the trial. The test should be analytically and pre-analytically validated The test should be analytically and pre-analytically validated

for use with archived tissue.for use with archived tissue. An analysis plan for the biomarker evaluation should be An analysis plan for the biomarker evaluation should be

completely specified in writing prior to the performance of completely specified in writing prior to the performance of the biomarker assays on archived tissue and should be the biomarker assays on archived tissue and should be focused on evaluation of a single completely defined focused on evaluation of a single completely defined classifier.classifier.

The results of the analysis should be validated using The results of the analysis should be validated using specimens from a similar, but separate, studyspecimens from a similar, but separate, study

Does It Matter If the Does It Matter If the Randomization in the RCT Was Randomization in the RCT Was

Not “Stratified” By the Test?Not “Stratified” By the Test?

NoNo Stratification improves balance of Stratification improves balance of

stratification factors in overall stratification factors in overall comparisonscomparisons

Stratification does not improve Stratification does not improve comparability of treatment (T) and control comparability of treatment (T) and control (C) groups within test positive patients or (C) groups within test positive patients or within test negative patientswithin test negative patients

What Proportion of the Patients What Proportion of the Patients Should Have Adequate Archived Should Have Adequate Archived

Tissue Available for a Valid Tissue Available for a Valid Analysis?Analysis?

No minimum percentage can be adequately No minimum percentage can be adequately defendeddefended

For treatment versus control comparisons within For treatment versus control comparisons within test + patients (or within test – patients), the test + patients (or within test – patients), the analysis is internally valid regardless of the analysis is internally valid regardless of the proportion with available tissue or the selection proportion with available tissue or the selection factors affecting tissue availabilityfactors affecting tissue availability

External validity (generalization of results to the External validity (generalization of results to the broad population of patients) depends on broad population of patients) depends on representativeness of the full study group as well representativeness of the full study group as well as the proportion with tissue availableas the proportion with tissue available

Does the RCT From Which the Does the RCT From Which the Archived Tissues are Taken Need Archived Tissues are Taken Need to Be Significant Overall for the T to Be Significant Overall for the T

vs C Treatment Comparison?vs C Treatment Comparison?

No No In a fully prospective trial of T vs C with a In a fully prospective trial of T vs C with a

companion test it is incorrect to require that the companion test it is incorrect to require that the overall T vs C comparison be significant to claim overall T vs C comparison be significant to claim that T is better than C for test + patients but not that T is better than C for test + patients but not for test – patientsfor test – patients That requirement has been traditionally used to protect That requirement has been traditionally used to protect

against data dredging. It is inappropriate for focused against data dredging. It is inappropriate for focused trials of a treatment with a companion test.trials of a treatment with a companion test.

Since the requirement is inappropriate for a fully Since the requirement is inappropriate for a fully prospective trial, it is also inappropriate for a prospective trial, it is also inappropriate for a properly designed prospective-retrospective trialproperly designed prospective-retrospective trial

FactorFactor   

AA BB CC DD

Clinical trialClinical trial RCT* designed to RCT* designed to address tumor address tumor

markermarker

RCT* not designed to address RCT* not designed to address tumor marker, but design tumor marker, but design

accommodates tumor marker accommodates tumor marker utility.utility.

Prospective Prospective observational registry, observational registry, treatment and followup treatment and followup

not dictatednot dictated

No prospective aspect to studyNo prospective aspect to study

Patients and Patients and patient datapatient data

Prospectively Prospectively enrolled, treated, enrolled, treated, and followed in and followed in

RCT*RCT*

Prospectively enrolled, treated, Prospectively enrolled, treated, and followed in RCT*and followed in RCT*

Prospectively enrolled Prospectively enrolled in registry, but in registry, but

treatment and followup treatment and followup standard of carestandard of care

No prospective stipulation of No prospective stipulation of treatment or followup; patient treatment or followup; patient data collected by retrospective data collected by retrospective

chart reviewchart review

Specimen Specimen collection, collection, processing, processing, and archivaland archival

Specimens Specimens collected, processed collected, processed

and assayed for and assayed for specific marker in specific marker in

real timereal time

Specimens collected, Specimens collected, processed, and archived processed, and archived

prospectively using generic prospectively using generic SOPs. Assayed after trial SOPs. Assayed after trial

completed with analytically completed with analytically validated test validated test

Specimens collected, Specimens collected, processed, and processed, and

archived prospectively archived prospectively using generic SOPs. using generic SOPs.

Assayed after trial Assayed after trial completedcompleted

Specimens collected, Specimens collected, processed and archived with processed and archived with

no prospective SOPsno prospective SOPs

Statistical Statistical Design and Design and analysisanalysis

Study powered to Study powered to address tumor address tumor marker question.marker question.

Study powered to address Study powered to address therapeutic question; therapeutic question;

underpowered to address tumor underpowered to address tumor marker question.marker question.

Focused analysis plan for Focused analysis plan for marker question developed marker question developed

prior to doing assaysprior to doing assays

Study not prospectively Study not prospectively powered at all. powered at all. Retrospective study Retrospective study design confounded by design confounded by selection of specimens selection of specimens for study.for study.Focused analysis plan Focused analysis plan for marker question for marker question developed prior to developed prior to doing assaysdoing assays

Study not prospectively Study not prospectively powered at all. Retrospective powered at all. Retrospective study design confounded by study design confounded by selection of specimens for selection of specimens for study.study.No focused analysis plan for No focused analysis plan for marker question developed marker question developed prior to doing assaysprior to doing assays

ValidationValidation Although preferred, Although preferred, validation not validation not

requiredrequired

Requires one or more Requires one or more additional validation studiesadditional validation studies

Result very likely to be Result very likely to be play of chance. play of chance.

Requires subsequent Requires subsequent validation studiesvalidation studies

Result very likely to be play Result very likely to be play of chance. of chance.

Requires subsequent Requires subsequent validationvalidation

TerminologyTerminology ProspectiveProspective Prospective using archived Prospective using archived samplessamples

Prospective Prospective /observational/observational

Retrospective/observationalRetrospective/observational

* RCT required for predictive biomarker

Revised Levels of Evidence for Tumor Revised Levels of Evidence for Tumor

Marker StudiesMarker Studies Level of EvidenceLevel of Evidence Category from Table 1Category from Table 1 Validation StudiesValidation Studies

AvailableAvailable

II AA None requiredNone required

II BB One or more with consistent resultsOne or more with consistent results

IIII BB NoneNoneoror

Inconsistent resultsInconsistent results

IIII CC 2 or more with consistent results2 or more with consistent results

IIIIII CC NoneNoneoror

1 with consistent results1 with consistent resultsoror

Inconsistent resultsInconsistent results

IV-VIV-V DD NANA

Barriers to the Barriers to the Development of Prognostic Development of Prognostic and Predictive Biomarkers and Predictive Biomarkers

for Clinical Usefor Clinical Use Increased complexity and cost of developing new drugs with Increased complexity and cost of developing new drugs with

predictive biomarkerspredictive biomarkers Difficulty of identifying the appropriate predictive biomarker Difficulty of identifying the appropriate predictive biomarker

prior to conduct of phase III clinical trials of new drugsprior to conduct of phase III clinical trials of new drugs Inadequate appreciation of the distinction between establishing Inadequate appreciation of the distinction between establishing

clinical correlation and medical utility for prognostic markersclinical correlation and medical utility for prognostic markers Inadequate appreciation of the importance of focused testing of Inadequate appreciation of the importance of focused testing of

pre-specified marker hypothesespre-specified marker hypotheses High costs and time required for prospective validation of High costs and time required for prospective validation of

prognostic factors for withholding chemotherapy in low-risk prognostic factors for withholding chemotherapy in low-risk populationspopulations

Inadequate focus on analytically validating testsInadequate focus on analytically validating tests Unavailability of adequate archived tissue from patients on key Unavailability of adequate archived tissue from patients on key

RCT’sRCT’s

Barriers to the Barriers to the Development of Prognostic Development of Prognostic and Predictive Biomarkers and Predictive Biomarkers

for Clinical Usefor Clinical Use Lack of regulatory guidance for approval of in-Lack of regulatory guidance for approval of in-

vitro diagnostics and of treatments for restricted vitro diagnostics and of treatments for restricted populations thatpopulations that is appropriate to the inherent heterogeneity of human is appropriate to the inherent heterogeneity of human

cancerscancers encourages the development of predictive biomarkersencourages the development of predictive biomarkers is cognizant of the infeasibility of always addressing is cognizant of the infeasibility of always addressing

treatment by subset questions prospectivelytreatment by subset questions prospectively e.g. Difficulty of approving even restriction of a labeling e.g. Difficulty of approving even restriction of a labeling

indication based on a compelling prospective-retrospective indication based on a compelling prospective-retrospective analyses of KRAS in multiple randomized colorectal cancer analyses of KRAS in multiple randomized colorectal cancer trials trials

ConclusionsConclusions New biotechnology and knowledge of tumor New biotechnology and knowledge of tumor

biology provide important opportunities to biology provide important opportunities to improve therapeutic decision-makingimprove therapeutic decision-making

Treatment of broad populations with Treatment of broad populations with regimens that do not benefit most patients is regimens that do not benefit most patients is no longer necessary nor economically no longer necessary nor economically sustainable sustainable

The established molecular heterogeneity of The established molecular heterogeneity of tumors requires new approaches to the tumors requires new approaches to the development of cancer treatmentsdevelopment of cancer treatments The traditional assumption underlying most The traditional assumption underlying most

clinical trial design that treatment by subset clinical trial design that treatment by subset interactions are unlikely is often inappropriate interactions are unlikely is often inappropriate todaytoday

For establishing medical utility of prognostic and For establishing medical utility of prognostic and predictive tests, fully prospective trial designs predictive tests, fully prospective trial designs are desirableare desirable

It is often not possible, however, to identify the It is often not possible, however, to identify the appropriate predictive marker for a drug prior to appropriate predictive marker for a drug prior to the conduct of the pivotal trials of that drugthe conduct of the pivotal trials of that drug

It is often not feasible or ethical to conduct It is often not feasible or ethical to conduct prospective RCT’s for validating predictive prospective RCT’s for validating predictive markers of approved drugsmarkers of approved drugs

It is very expensive and time consuming to It is very expensive and time consuming to conduct prospective trials to evaluate prognostic conduct prospective trials to evaluate prognostic markers that indicate withholding standard markers that indicate withholding standard chemotherapy regimens from good risk patientschemotherapy regimens from good risk patients

Studies using archived tissues, when Studies using archived tissues, when conducted under ideal conditions conducted under ideal conditions and independently confirmed can and independently confirmed can provide the highest level of evidenceprovide the highest level of evidence

Barriers to the development, Barriers to the development, approval and use of effective approval and use of effective prognostic and predictive prognostic and predictive biomarkers that benefit patients and biomarkers that benefit patients and society are substantialsociety are substantial

AcknowledgementsAcknowledgements

Soonmyung Paik, NSABPSoonmyung Paik, NSABP Daniel Hayes, U. MichiganDaniel Hayes, U. Michigan