1

Harmonizing Regulatory Approaches to Clinical Trials to Improve Quality and Compliance

Leslie K. Ball, MDDirector, Division of Scientific Investigations

Office of Compliance, CDER, FDAFDA Enforcement Summit October 22, 2009

2

Disclaimer

The views expressed are my own.

3

Quality and Compliance in Clinical Trials: Overview

• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement

• Where are we going?– Quality risk management

• Sponsors and clinical investigators• FDA and other regulatory authorities

4

Historical Perspective: Role of FDA in Ensuring High Quality Research

• 1961: 1st FDA inspection of clinical investigator in Maryland– CI convicted after it was determined that most of results produced

at kitchen table (> 25 sponsors) • Early 1960’s; MER-29 (cholesterol lowering agent)

approved– Subsequent human use revealed balding, impotence, cataracts,

and liver damage– Sponsor found guilty of falsifying source data and failing to submit

data on toxicity• These high profile cases of data fraud by clinical

investigators and sponsors* led to the recognition that FDA had responsibility for ensuring data quality via inspections

*http://www.fda.gov/downloads/AboutFDA/CentersOffices/CDER/UCM164151.pdf

5

Role of FDA In Ensuring High Quality Research

• Approvals and labeling of a new drug or biologic products are risk-benefit decisions

• Based on scientific and clinical review of data collected and presented by Sponsors

• Relies on public participation in clinical trials and public confidence in the clinical trial enterprise

6

FDA Bioresearch Monitoring Inspection Program

• Inspections of clinical investigators, sponsors, IRBs, GLP and BE– Evaluate data quality and integrity– Assess whether rights, safety and welfare of subjects have been

protected• Types of inspection:

– Data validation in support of NDAs/BLAs/ANDAs• Clinical Investigators• Sponsors/Contract Research Organizations• Bioequivalence

– Surveillance (Systems-based inspections)• GLP• Institutional Review Boards

– For cause (complaints)• All of the above

7

Quality and Compliance in Clinical Trials: Overview

• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement

• Where are we going?– Quality risk management

• Sponsors and clinical investigators• FDA and other regulatory authorities

8

Public Expectations:How effective is the FDA*?

*http://www.gallup.com/poll/121886/cdc-tops-agency-ratings-federal-reserve-board-lowest.aspx

9

Public Expectations (cont.)• Congress

– House Energy and Commerce Committee/Govt Oversight Subcommittee

• Ketek (February 2008)• Independent IRBs (March 2009)

• OIG– September 2007: FDA’s Oversight of Clinical Investigators– January 2009: FDA’s Oversight of Clinical Investigators’ Financial

Information– Ongoing: FDA’s Oversight of Foreign Clinical Trials

• GAO– March 2009 Independent IRBs– Report to be released soon: FDA’s Oversight of Clinical

Investigators (DQ process)

10

Enforcement

11

Enforcement (cont.)“…A strong FDA has credibility with the public.A strong FDA is transparent in explaining its

decisions.A strong FDA pursues creative solutions to

longstanding problems and is always looking for novel ways to prevent illness and promote health.

And a strong FDA enforces the law…”

Margaret Hamburg, M.D., Commissioner of Food and DrugsFDLI "Effective Enforcement and Benefits to Public Health"August 6, 2009

12

0

100

200

300400

500

600

700

800

FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08

Spon

CIIRB/RDRC

BIOEQGLP

CDER BIMO Inspections* FY 2000-2008

549 556690723 672 667647 674690

*Based on inspection start date 2/5/09

13

CDER Clinical InvestigatorOAI Actions (Warning/NIDPOE Letters*)

02468

1012141618

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

WL

NIDPOE

*Based on Letter Issued (FY) 9-23-09

14

Enforcement • Is misconduct by clinical investigators

increasing?– ORA and DSI inspection data indicate that

rate of significant non-compliance (as evidenced by inspections with field classifications of OAI) has been stable, but rate of enforcement (final classifications of OAI including WLs) has been increasing

15

Quality and Compliance in Clinical Trials: Overview

• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement

• Where are we going?– Quality risk management

• Sponsors and clinical investigators• FDA and other regulatory authorities

16

Data Validation Inspections for New Drug Applications: Old Paradigm

• Sites selected for inspection using qualitative risk-based approach

– Most often selected highest enrolling sites for inspection

• Typically 4-5 sites inspected per application• If significant violations identified at 1-2 sites

such that data deemed unreliable, FDA performed sensitivity analysis excluding those sites from efficacy analysis

• At remaining sites in application, data assumed to be reliable

17

The World Has Changed• Increase in multiregional, multicenter

studies with no single site driving efficacy• Limiting enrollment at clinical sites as risk-

mitigation strategy• If pivotal study has 200 sites and 2 of 4

inspected sites reveal significant GCP violations impacting data integrity, are data unreliable from 50% of sites across application?

18

Data Validation Inspections for New Drug Applications: Current Paradigm

• Use risk-based approach to select clinical investigator sites for inspection

• If first round on inspections of clinical investigator sites reveal significant violations affecting data integrity, conduct second round of inspections

• Conduct systems-based inspection of sponsor and/or CRO?– Were identified problems corrected in real time?

• Request 3rd Party audit of representative sample of sites• Compare 3rd Party audit with sponsor monitoring reports and FDA

inspection results• Assess whether violations were limited to sites inspected by FDA or

extend across other sites as determined by 3rd party audit• Hold sponsor and CRO accountable for inadequate monitoring,

failure of quality systems (Including rejection of data, WarningLetters, Follow up Inspections)

19

Limitations of Current Approach• Can’t “inspect in” quality after clinical trial

is completed• Labor and time intensive

– May lead to delays in product approval– Costly in terms of inspectional resources

• Lacks adequate incentives to prevent or correct non-compliance in real time– “Compliance is a reverse lottery that sponsors

choose not to play”

20

• How can we collectively ensure that clinical trials produce high quality data and protect the rights, safety, and welfare of research subjects?

21

What is Quality?• Definition of quality:

– In clinical trial context: The ability to effectively and efficiently answer the intended question about the benefits and risks of a medical product (therapeutic or diagnostic) or procedure while assuring protection of human subjects.*

*Clinical Trials Transformation Initiative at http: //www.trialstransformation.org/scope

22

Data Quality and Data Integrity• Data Quality: Basic Elements (ALCOA)

– Attributable– Legible– Contemporaneous– Original– Accurate

• Data integrity: A dimension of data contributing to trustworthiness and pertaining to the systems and processes for data capture, correction, maintenance, transmission, and retention*

• Focus on significant endpoint data supporting safety and efficacy determinations in FDA decision-making

*http://www.cdisc.org/glossary/CDISC2008GlossaryVersion7.0.pdf

23

High Quality Data• No perfect dataset• Alternate definition of high quality data: data that

sufficiently supports conclusions and interpretations equivalent to those derived from error-free data* (data fit for purpose)– Sufficiently accurate to support FDA regulatory

decisions, sponsor claims about a product, labeling• Data quality and integrity together determine

fitness for purpose*Assuring Data Quality and Validity in Clinical Trials for Regulatory DecisionMaking: Workshop Report, IOM 1999 at http://www.iom.edu/CMS/3740/5583.aspx

24

Implementation of Quality Systems*• Can’t “inspect in” quality • Requires coordination of all

stakeholders to build in quality • ISO Quality Management System

– Iterative process • Plan: Define objectives-Systematic

identification of errors throughout process

• Do: Implement changes to address errors

• Check: Measure results• Act: Correct and improve in continuous

feedback loop (in as close to real time as you can get)

*http://www.iso.org/iso/iso_catalogue/management_standards/understand_the_basics.htm

25

Clinical Trial Error Types with Potential to Influence Data Reliability and Subject Safety*

• Design error: Poor protocol design • Procedural error: Failure to follow investigational plan,

failure to conduct proper monitoring• Recording error: Data collection

– Random– Systematic

• Analytical error: Data analysis

A quality system must address each of these*Adapted from Baigent et al., Clinical Trials 2008;5:49-55

*Adapted from Baigent et al., Clinical Trials 2008;5:49-55

26

Effect of Errors • Fraud is rare and usually isolated in scope (i.e.

single clinical investigator)– But when it occurs, it can be dramatic and undermine

public confidence in clinical trials• Errors other than fraud such as poorly designed

and executed protocols, inadequate recordkeeping, or faulty statistical analysis may be more systematic and can render data unreliable– Non-inferiority study designs: “Sloppiness” in clinical

trials may obscure difference between study drug and comparator

27

What is the acceptable error rate?

• No “one size fits all” in clinical trials– Depends on risk; ie, What is the risk of getting

this wrong?• For example, in adverse event reporting, risk of

incorrectly recording a headache is different from missing a subject death

28

FDA Regulations Support a Risk-Based Approach

• Risk management aids in prioritizing our work

• Allows for better use of finite resources! Less time and resources spent on low risk and low

impact areas means more time and resources for meaningful high impact activities

• Within regulatory framework there is flexibilitypermitted in approach and implementation

29

Implementing a Quality Risk Management System

• Identity key areas of risk to data quality/integrity and human subject protections

• Develop an integrated framework– Protocol design, Site selection and training, Data and Safety

Monitoring Plan, Data Management Plan, Quality Assurance Plan, Data Analysis Plan

– Use of information technology in detecting data irregularities• Address human factors in systems

– Hire experienced, qualified staff– Avoid/manage financial conflicts of interest – Decrease number of times data are handled– Clinical trial monitoring: alternative approaches to traditional

industry model using a quality management system approach– Single points of accountability, including appropriate

management of outsourcing

30

Implementing a Quality Risk Management System (2)

• Create systems that limit opportunity for errors– Simplify protocol and outcomes assessed– Standardize systems and formats where possible– Use validated instruments/definitions– Keep amendments to a minimum and check the

CRFs and consent form against each change– Think very carefully about unblinding procedures– Insist on training and then test it– Have a disaster plan

– Do beta-testing/dry-runs

– Monitor and correct errors in real time

31

Quality and Compliance in Clinical Trials: Overview

• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement

• Where are we going?– Quality risk management

• Sponsors and clinical investigators• FDA and other regulatory authorities

32

Data Validation Inspections for New Drug Applications: New Paradigm

• FDA and other regulatory authorities– Use Inspection Risk Model to predict how

inspectional findings from a few sites translates across entire application• Model contains data across all sites in

application, based on risk scores violations observed at a single or a few sites can be extrapolated

• Include inclusion of random sites • Develop a learning algorithm so risk attributes

and weights will evolve over time

33

Site Selection Tool incorporates factors in a multi-attribute decision analysis algorithm

Risk-based Site Selection Tool Algorithm Illustration: Note that Actual Model Differs in Details

Attributes Risk Functions Hierarchical Weighting Schema1 2 3

# Risk Attribute1 Enrollment2 Efficacy Outcome3 Protocol Violations4 Application Type

5 Non-Serious Adverse Events

6 Time Since Last Inspection

7 OAI History8 VAI History9 Protocol Violations

10 Site Type

1% 5% 25% 50% 75% 95% 99%Highest!Risk O O

Very!High!Risk

High!Risk O O

Medium!Risk

Low!Risk O O

Very!Low!Risk

Lowest!Risk O

Enrollment

Efficacy Outcome

PI Complaints

Financial Disclosures

75%

25%

50%

50%

! Column Weights = 100%

30%

20%

Efficacy Drivers

Site Quality

! Column Weights = 100%

Subset of entire risk tree

Site Percentile

Risk Score4

0 0.25 0.5 0.75 1

site a

site b

site c

site e

site f

site g

site h

site i

site j

site d

Risk Attribute Total ContributionPatient Population 30%

Historic Violations 20%

Safety Data Reporting 20%

Site Location 10%

Complexity of Study Design 5%

Complaints 5%

Sample Inspection

Target

34

Data Validation Inspections for New Drug Applications: New Paradigm (cont.)

• Conduct inspections of sponsor’s quality systems– Select products in early phase of development, mid phase, and

at NDA submission; link with clinical investigator inspections– Require submission of sponsor’s quality risk management

system for new drug applications, including monitoring plan• Conduct “real time” inspections of phase 3 clinical

studies– Feedback to site to correct problems in real time

• Involve DSI reviewers earlier in drug development process; i.e. End of Phase 2– Evaluate protocol for data quality and human subject protection

issues• Partner with other regulatory authorities

35

EMEA/FDA Collaboration

• EMEA/FDA(CDER)– Develop a joint GCP program to forecast, coordinate, schedule

and conduct joint and/or collaborative GCP inspections– Establish and implement procedures to share information with

EMEA related to applications for approval of new drugs• Challenges

– Consider legal limitations with information sharing• Potential benefits

– Optimize use of limited GCP inspection resources – Enhance consistency in regulatory approaches – May decrease burden on Sponsors/New Drug Applicants– Extension of partnership to other regulatory authorities

35

36

EMEA Inspections by Region (1997-2008)

21 21

8 7 9 10

21

1 3

10 9

3 2

2

4

1

13

3

19

8

41

5

1

3

0

10

20

30

40

50

60

2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 1997

Rest of the world

North America

EU/EEA/AFTA

Courtesy Dr. Fergus Sweeney and Ana Rodriguez, EMEA

37

“…Economics has been too isolated and too stuck on the view that markets are efficient and self-regulating. It has derailed our thinking…”

“... Rules that are imposed from the outside or unilaterally dictated by powerful insiders have less legitimacy and are more likely to be violated. Likewise, monitoring and enforcement work better when conducted by insiders than by outsiders. These principles are in stark contrast to the common view that monitoring and sanctions are the responsibility of the state andshould be conducted by public employees.”

38

Take Home Messages1. FDA has an important role in ensuring high

quality data • Enforcement efforts should be transparent, timely

and appropriate to the observed violations2. Inspections alone cannot ensure the quality

of clinical research and high quality data 3. Quality of clinical trial enterprise must be built

into the process all at phases of clinical development and involving all stakeholders • Prioritize and manage risks for maximum impact

4. At stake is public confidence and participation in the drug development and ultimately the availability of safe and effective drug products