Page 1

1. Tables, Figures, and Listings Associated with Measures of Central Tendency – Focus on Vital

Sign, Electrocardiogram, and Laboratory Measurements in Phase 2-4 Clinical Trials and

Integrated Submission Documents

Version 1.0Created xx XXXX 201x

Contributors: To Be Inserted

Page 2

2. Table of ContentsSection Page1. Tables, Figures, and Listings Associated with Measures

of Central Tendency – Focus on Vital Sign, Electrocardiogram, and Laboratory Measurements in Phase 2-4 Clinical Trials and Integrated Submission Documents..........................................................................................................................1

2. Table of Contents....................................................................................................................2

3. Revision History......................................................................................................................4

4. Purpose....................................................................................................................................5

5. Background..............................................................................................................................6

6. General Considerations...........................................................................................................76.1. Visual Displays...................................................................................................................76.2. P-values and Confidence Intervals.....................................................................................76.3. Conservativeness................................................................................................................76.4. Planned versus Unplanned Measurements.........................................................................86.5. Measurements at a Discontinuation Visit...........................................................................86.6. Post Study Drug Measurements.........................................................................................86.7. Screening Measurements versus Special Topics................................................................96.8. Transformations of Data.....................................................................................................96.9. Units...................................................................................................................................96.10. Above and Below Quantifiable Limits...............................................................................96.11. Number of Therapy Groups................................................................................................96.12. Multi-phase Clinical Trials.................................................................................................96.13. Homogeneity of Studies in Integrated Analyses..............................................................106.14. ECG Correction Factors...................................................................................................10

7. TFLs for Individual Studies...................................................................................................117.1. Multiple Measurements Over Time..................................................................................117.2. Single Pre- and Post-Treatment Measurements...............................................................127.3. Discussion.........................................................................................................................12

8. TFLs for Integrated Summaries.............................................................................................148.1. Multiple Measurements Over Time – Timing Varies Across

Studies..............................................................................................................................148.2. Multiple Measurements Over Time – Timing Consistent Across

Studies..............................................................................................................................148.3. Single Pre- and Post-Treatment Measurements...............................................................148.4. Discussion.........................................................................................................................14

Page 3

9. Example SAP Language........................................................................................................159.1. Box Plot of Observed Values Over Time.........................................................................159.2. Box Plot of Change Values Over Time............................................................................159.3. Change to Last, Minimum, and Maximum Table............................................................159.4. Scatterplot.........................................................................................................................16

10. References.............................................................................................................................17

11. Contributions.........................................................................................................................18

Page 4

3. Revision HistoryVersion 1.0 was reviewed and finalized xx XXXX 201x.

Page 5

4. PurposeThe purpose of this white paper is to provide advice on displaying, summarizing, and/or analyzing measures of central tendency, with a focus on vital sign, electrocardiogram (ECG), and laboratory measurements in Phase 2-4 clinical trials and integrated submission documents. The intent is to begin the process of developing industry standards with respect to analysis and reporting for measurements that are common across clinical trials and across therapeutic areas. This white paper provides recommended Tables, Figures, and Listings (TFLs) intended for measures of central tendency for a common set of safety measurements. It is the first of five white papers that are planned for development as part of an FDA/PhUSE Working Group effort. The four additional white papers include:

Recommended Tables, Figures, and Listings Associated with Outliers or Shifts from Normal to Abnormal – With a Focus on Vitals, ECGs, and Labs in Phase 2-4 Clinical Trials and Integrated Submission Documents

Recommended Tables, Figures, and Listings Associated with Adverse Events and Deaths – With a Focus on Phase 2-4 Clinical Trials and Integrated Submission Documents

Recommended Tables, Figures, and Listings Associated with Demographics, Concomitant Medications, and Disposition – With a Focus on Phase 2-4 Clinical Trials and Integrated Submission Documents

Recommended Tables, Figures, and Listings Associated with Hepatotoxicity – With a Focus on Phase 2-4 Clinical Trials and Integrated Submission Documents

This advice can be used when developing the analysis plan for individual clinical trials, integrated summary documents, or other documents in which measures of central tendency are of interest. Although the focus of this white paper pertains to specific safety measurements (vital signs, ECGs, and laboratory measurements), some of the content may apply to other measurements (e.g., different safety measurements and efficacy assessments).

Development of standard Tables, Figures, and Listings (TFLs) and associated analyses will lead to improved standardization from collection through data storage. (You need to know how you want to analyze and display results before finalizing how to collect and store data.) The development of standard TFLs will also lead to improved product lifecycle by ensuring reviewers receive the desired analyses for the consistent and efficient evaluation of patient safety and drug effectiveness. Although having standard TFLs is an ultimate goal, this white paper reflects recommendations only and should not be interpreted as “required” by any regulatory agency.

Page 6

5. Background Industry standards have evolved over time for data collection (CDASH), observed data (SDTM), and analysis datasets (ADaM). There is now recognition that the next step would be to develop standard TFLs for common measurements across clinical trials and across therapeutic areas. Some could argue that perhaps the industry should have started with creating standard TFLs prior to creating standards for collection and data storage (consistent with end-in-mind philosophy), however, having industry standards for data collection and analysis datasets provides a good basis for creating standard TFLs.

The beginning of the effort leading to this white paper came from the FDA computational statistics group that led to the creation of a FDA/PhUSE Working Group. [Insert background on this.]

There are several existing documents that contain suggested TFLs for common measurements. However, many of the documents are now relatively outdated, and generally lack sufficient detail to be used as support for the entire standardization effort. Nevertheless, these documents were used as a starting point in the development of this white paper. The documents include:

ICH E3: Structure and Content of Clinical Study Reports Guideline for Industry: Structure and Content of Clinical Study Reports Guidance for Industry: Premarketing Risk Assessment Guidance for Industry: Drug-Induced Liver Injury Reviewer Guidance. Conducting a Clinical Safety Review of a New Product Application

and. Preparing a Report on the Review ICH M4E: Common Technical Document for the Registration of Pharmaceuticals for

Human Use - Efficacy E14: The Clinical Evaluation of QT/QTc Interval Prolongation and Proarrhythmic

Potential For Non-Antiarrhythmic Drugs Guidance for Industry: E14 Clinical Evaluation of QT/QTc. Interval Prolongation and

Proarrhythmic Potential for Non-Antiarrhythmic Drugs ICH E2F: Development Safety Update Report

We consider the reviewer guidance a key document. As discussed in the reviewer guidance, there is generally an expectation that analyses of central tendency are conducted for vital signs, ECGs, and laboratory measurements. The guidance recognizes value to both analyses of central tendency and analyses of outliers or shifts from normal to abnormal. We assume both will be conducted for safety signal detection. This guidance covers the central tendency portion, with the expectation that an additional TFL or TFLs will also be created with a focus on outliers or shifts.

Page 7

6. General Considerations

6.1. Visual Displays[Needs further development; Get references and perhaps additional content from Cross-industry Graphics Working Group?] The value of visual displays (i.e., figures) of safety data has been recognized for many years (references?), however, the practice to routinely display safety data visually varies across the industry. The interest has re-surfaced through recent activity (references). Throughout this white paper, visual displays will generally be a preferred option. There has also been advancement in interactive visual capabilities (references). These interactive capabilities are certainly beneficial, but are considered out-of-scope for this version of the white paper.

6.2. P-values and Confidence Intervals There has been ongoing debate on the value or lack of value for the inclusion of p-values and/or confidence intervals (or other measures of spread) in safety assessments (Crowe et al 2009, other references?). This white paper does not attempt to resolve this debate. As noted in the reviewer guidance, p-values or confidence intervals can provide some evidence of the strength of the finding, but unless the trials are designed for hypothesis testing (rarely the case), these should be thought of as descriptive. Throughout this white paper, p-values and measures of spread are included in several places. Where these are included, they should not be considered as hypothesis testing (i.e., descriptive only). If a company or compound team decides that these are not helpful as a tool for reviewing the data, they can be excluded from the display.

Some teams may find p-values and/or confidence intervals useful to facilitate focus, but have concerns that lack of “statistical significance” provides unwarranted dismissal of a potential signal. Conversely, there are concerns that due to multiplicity issues, there could be over-interpretation of p-values adding potential concern for too many outcomes. Similarly, there are concerns that the lower- or upper-bound of confidence intervals will be over-interpreted. (A mean change can be as high as xxxx causing undue alarm.) It is important for the users of these TFLs to be educated on these issues. (See xxxxx reference to facilitate such education. Note: is there such a reference?)

If p-values and/or confidence intervals are included, p-values are generally preferred over confidence intervals in tables since it’s easier to review a column of p-values over a column of confidence intervals. However, confidence intervals (or alternative measure of spread) are generally preferred over p-values in figures.

6.3. ConservativenessThe focus of this white paper pertains to clinical trials in which there is comparator data. As such, the concept of “being conservative” is different than when assessing a potential safety signal within an individual subject or a single arm. Taking a conservative approach (i.e., an approach with a high number of subjects reaching a threshold, or having an approach giving the highest change value) with respect to defining outcomes may actually make it more difficult to

Page 8

identify potential safety signals with respect to comparing treatment with a comparator (see Section 7.1.7.3.2 in the reviewer guidance). Thus, some of the outcomes recommended in this white paper may appear less conservative than alternatives, but the intent is to propose methodology that can identify meaningful potential safety signals for a treatment relative to a comparator group.

6.4. Planned versus Unplanned MeasurementsOne topic that tends to be unique to safety is the collection of planned and unplanned measurements. Unplanned safety measurements can arise for various reasons. During a study, the clinical investigator sometimes orders a repeat test or “retest” of a laboratory test especially if he/she has received an abnormal value. The investigator may also request the patient return for a “follow-up visit” due to clinical concerns. In general, retests are repeat tests performed because an initial test result had an abnormal value. The repeat result may either confirm the initial test results, or (less commonly) indicate that a laboratory error occurred in the case of the initial result. Retests are often performed to verify that the action taken by the investigator (e.g., changing the dose of study drug as allowed by the protocol) has the desired effect (e.g., test results have returned to normal). If such retests are conducted until normality has been reached, analyses from baseline to last observation, for example, would be biased toward normality. Thus, we recommend including only planned measurements when creating displays or conducting analyses over time and when assessing change from baseline to endpoint. However, we recommend including planned and unplanned measurements when assessing maximum and minimum changes as these are intended to focus the most extreme change. (Including planned an unplanned measurements will also be recommended for analyses that focus on outliers or shifts, which will be a topic for the 2nd white paper.) Of note, these recommendations can only be implemented if planned and unplanned measurements can be distinguished via data collection.

6.5. Measurements at a Discontinuation Visit[Needs to be completed still] Discuss measurements at a discontinuation visit that isn’t aligned with a normal timing. These would be considered “planned” per protocol, but not consistent with the planned timing.

6.6. Post Study Drug MeasurementsThere is currently no standard approach on how to handle safety assessments post study drug. Some guidances (references?) contain advice on collecting safety measurements post study drug (e.g, 30 days post or, x half-lives). In addition, study designs which keep subjects in a study after deciding to stop study drug are becoming more popular (reference). In these cases, subjects can be off study medication for an extended period of time. Any advice or decisions related to the collection of safety measurements post study drug should not be confused with how to include such data in displays and/or analyses. One approach is to include such assessments in the displays assuming the subject remained on study drug. This would be consistent with the intent-to-treat principle. However, the intent-to-treat principle may not be appropriate for safety (reference?) and could make it more difficult to understand the safety profile of a compound.

Page 9

We recommend that the TFLs in this white paper include measurements while on study drug. Separate TFLs can be created for measurements off study drug. This enables the researcher to distinguish between drug-related potential safety signals versus potential safety signals that could be more related to discontinuing a drug (e.g., return of disease symptoms, introduction of a concomitant medication, and/or discontinuation- or withdrawal-effects of the drug). We assume it is important to distinguish among these.

If a study team considers the intent-to-treat approach necessary for a complete assessment, the summary/analysis could be added but should not be instead of the approach where such data is handled separately.

6.7. Screening Measurements versus Special TopicsThe focus of this white paper pertains to measurements as part of normal safety screening. For many compounds, some measurements are relevant to addressing a-priori special topics of interest. In these cases, it is possible that additional TFLs and/or different TFLs are warranted. Special topics are out-of-scope for this white paper (and for all white papers that are planned for development as part of an FDA/PhUSE Working Group effort, with the exception of hepatotoxicity). In addition, it is possible that additional TFLs are warranted when a potential safety signal is identified using these methods focused on central tendency and/or the methods that focus on outliers or shifts (2nd white paper). Additional TFLs that would be considered “post-hoc” for further investigation are considered out-of-scope.

6.8. Transformations of Data[Needs further development] Topic of debate. One position: OK to not transform if enough patients (approx xxxx), due to Central Limit Theorem. Another position: Test assumptions first, then decide (iterative approach).

We recommend using original scale data as the general recommendation (when the sample size is sufficient). Iterative approach may not make sense for screening purposes ….. Expand……

6.9. Units[Needs to be completed still]

6.10. Above and Below Quantifiable Limits[Needs to be completed still]

6.11. Number of Therapy Groups[Needs to be completed still] Discuss pooling treatment arms versus not. The example TFLs show one treatment arm versus comparator in this version of the white paper. Most TFLs can be easily adapted to include multiple treatment arms.

6.12. Multi-phase Clinical TrialsThe example TFLs show one treatment arm versus comparator within a controlled phase of a study. Discussion around additional phases (e.g., open-label extensions) is considered out-of-

Page 10

scope in this version of the white paper. Many of the TFLs can be adapted to handle multiple phases.

6.13. Homogeneity of Studies in Integrated Analyses[Needs to be completed still] Discuss that studies should be “reasonably” similar. Methods can handle a fair amount of differences though. Adjusting for study is important. Reviewing OR (or similar summary) adjusted for study would be important to ensure not impacted by Simpson’s Paradox. [Get content from Cross-Industry Meta-Analysis Group? There is some discussion in the review guidance]

6.14. ECG Correction Factors[Needs to be completed still – Lilly internal experts?]

Page 11

7. TFLs for Individual Studies

7.1. Multiple Measurements Over Time For safety assessments that have multiple measurements over time (typically the case for vital signs, sometimes the case for ECGs and laboratory measurements), a box plot of the observed values and a box plot of change from baseline over time are recommended. See Figures 7.1 and 7.2. In each of the plots, lines indicating the reference limits (absolute limits for Figure 7.1, delta limits for Figure 7.2) can be added to ease the review of the plots. In cases where limits vary across age and gender, the lowest of the high limits and the highest of the low limits can be used. We recommend large clinical trial population based reference limits for this purpose as opposed to conventional reference limits. The 2nd planned white paper (Recommended Tables, Figures, and Listings Associated with Outliers or Shifts from Normal to Abnormal) will contain the rationale. However, until large clinical trial population based reference limits become more broadly available, conventional reference limits can be used.

In addition, a table that displays mean changes to last, minimum, and maximum observation is recommended. See Table 7.1. In particular, we recommend the following five summaries/analyses:

Change from baseline (last non-missing observation in the baseline period) to last observation (last non-missing observation in the treatment period); includes all subjects who have both a baseline and post-baseline observation; ANCOVA containing terms for treatment and the continuous covariate of baseline measurement

Change from baseline (last non-missing observation in the baseline period) to Visit x (where x is the last visit in the treatment period); includes all subjects who have both a baseline and Visit x result (i.e., Completers Population); ANCOVA containing terms for treatment and the continuous covariate of baseline measurement

Change from baseline (last non-missing observation in the baseline period) to Visit x (where x is the last visit in the treatment period); includes all subjects who have both a baseline and post-baseline observation; a maximum likelihood-based mixed-effects repeated measures analysis using all the longitudinal observations at each post-baseline visit [Vital signs and ECG measurements only]

Change from the minimum value during the baseline period to the minimum value during the treatment period including all subjects who have both a baseline and post-baseline observation; includes all subjects who have both a baseline and post-baseline observation; ANCOVA containing terms for treatment and the continuous covariate of baseline measurement

Change from the maximum value during the baseline period to the maximum value during the treatment period including all subjects who have both a baseline and post-baseline observation; includes all subjects who have both a baseline and post-baseline observation; ANCOVA containing terms for treatment and the continuous covariate of baseline measurement

Page 12

[To be inserted]

Figure 7.1. Box Plot – xxx Over Time (Weeks Since Randomized)

[To be inserted]

Figure 7.2. Box Plot – Change in xxx Over Time (Weeks Since Randomized)

Table 7.1. Mean Change to Last, Minimum, and Maximum Values

[To be inserted]

7.2. Single Pre- and Post-Treatment Measurements For safety assessments that have single pre- and post-treatment measurements planned, a scatterplot (including planned measurements only) is recommended (see Figure 7.3). Alternatively, Figures 7.1 and 7.2 can be considered. In these cases, Table 7.1 with just the change from baseline to last observation is recommended.

[Insert Figure 7.3]

Figure 7.3. Scatterplot

7.3. DiscussionSection 6.1 provides rationale for preferring visual displays. There are certainly multiple visual displays that can be used for central tendency. Another visual display for showing trends over time that was considered is one that displays means and standard error bars (See Figure 7.4). We recommend box plots instead since they have the advantage of easily showing additional summaries of interest beyond the mean for safety (median, min, max, quartiles). It also shows the impact of outliers on the central tendency. The disadvantage of box plots is that they have limited readability if there are multiple treatment arms and/or many time points such that having the data fit on one page becomes difficult. (Of course, this could be a limitation of Figure 7.4. as well.) A particular box plot may also have limited readability when there’s an extreme outlier which then squishes the box portion. Various techniques can be considered to handle this situation. Slashes can be used on the y-axis to provide separation (See Figure 7.5), a transformation (e.g., log transformation) can be used for just the measurements in which this problem occurs, or the outlier is not shown (or “clipped”) from the display but included in the summary statistics. The “slash-method” is generally preferred by medical colleagues, however may be more difficult to implement routinely.

Consideration was given on whether we really need 2 figures and a table to assess central tendency as the general recommendation, especially since marked outliers is typically of greatest

Page 13

interest (See Section 7.1.7.3.1 of the reviewer guidance). Having the ability to visually view potential time trends over time is generally desirable by medical colleagues. However, for screening purposes, including an assessment of maximum and minimum changes is desirable as well, since potentially meaningful changes can occur at different times for different patients (Fraser 2001?). Changes to minimum and maximum values provide an average of the range of changes, and the differences in these averages between treatment and control is considered useful for potential safety signal detection purposes. Changes to last observation or last visit do not appear to have as much value for potential safety signal detection purposes, but may be considered expected for a more thorough assessment.

Once it is decided to include an assessment of minimum and maximum changes, there is debate on how to define baseline when more than one pre-treatment measure is collected per protocol. We recommend taking the minimum value across the baseline period for change to minimum, and taking the maximum value across the baseline period for change to maximum. This approach is not currently common across the industry, but is recommended as a means to continuously improve analytical approaches for detecting more meaningful changes. This is preferred over the last measurement during the baseline period since this minimizes the effect of normal variation and generally reflects more clinically meaningful changes of interest. For example, assume there are two pre-treatment assessments where the first assessment is in the high range and the second assessment is the normal range. Also assume the subject has a value in the high range during treatment. If the last pre-treatment value is used as baseline, the subject contributes a high change toward the central tendency measure. This is inconsistent with what medical colleagues generally feel is appropriate. Another method to minimize the effect of variation is to take the average of pre-treatment measurements. This is a common approach in thorough QTc studies when multiple ECG assessments are taken at the same visit (true? reference?) We recommend the minimum and maximum value during the pre-treatment period over the average since it minimizes the effect of normal variation to a greater degree.

Page 14

8. TFLs for Integrated Summaries

8.1. Multiple Measurements Over Time – Timing Varies Across Studies

[Needs to be completed still – probably after Section 7 is more complete] Mean Changes to Last, Minimum, and Maximum Observations. Recommend ANCOVA with study in the model.

8.2. Multiple Measurements Over Time – Timing Consistent Across Studies

[Needs to be completed still – probably after Section 7 is more complete] When the timing of visits is consistent across studies, Figures 7.1 and 7.2 can be considered. However, there may not be many situations in which these would actually be recommended due to concerns about showing summary statistics in which data are simply pooled without accounting for study.

8.3. Single Pre- and Post-Treatment Measurements[Needs to be completed still – probably after Section 7 is more complete]

8.4. Discussion[Needs to be completed still – probably after Section 7 is more complete] Do not recommend box plots when some time points have some studies while other time points have other studies. Forest plot when need to understand by-study differences.

Page 15

9. Example SAP Language[To be completed after finalizing Sections 7 and 8]

9.1. Box Plot of Observed Values Over Time[To be completed]

9.2. Box Plot of Change Values Over Time [To be completed]

9.3. Change to Last, Minimum, and Maximum TableThe following is example SAP language that can be used as a starting point when Table 7.1 is planned:

Change from baseline to last observation for laboratory tests, vital signs characteristics (systolic BP, diastolic BP, pulse, weight, BMI, temperature), and ECGs (PR, QRS, uncorrected QT, corrected QT [QTc], and heart rate) will be summarized for subjects who have both baseline and at least one post-baseline result (except where noted otherwise). The following QT correction formula will be used: QTc=QT/RR0.413. Baseline will be the last non-missing observation in the baseline period. The last non-missing observation in the treatment period will be analyzed. Original-scale data will be analyzed. Unplanned measurements will be excluded.

[Add completer analysis and MMRM-based endpoint analysis]

Change from the minimum value during the baseline period to the minimum value during the treatment period for laboratory tests will be summarized for subjects who have both baseline and at least one post-baseline result. Baseline will be the minimum of non-missing observations in the baseline period. The minimum value in the treatment period will be analyzed. Similarly, change from the maximum value during the baseline period to the maximum value during the treatment period for laboratory tests will be summarized for subjects who have both baseline and at least one post-baseline result. Baseline will be the maximum of non-missing observations in the baseline period. The maximum value in the treatment period will be analyzed. Original-scale data will be analyzed. Planned and unplanned (repeat, unscheduled visits) measurements will be included.

For individual studies, treatment differences in mean change will be evaluated using an analysis of covariance (ANCOVA) model containing terms for treatment and the continuous covariate of baseline measurement. For the Summaries of Clinical Safety, treatment differences in mean change for laboratory tests will be evaluated using an analysis of covariance (ANCOVA) model containing terms for treatment, study, and the continuous covariate of baseline measurement. Type 3 sums of squares will be used.

For individual studies, treatment differences in mean change from baseline to selected post-baseline visits will be assessed using a maximum likelihood-based mixed-effects repeated measures analysis using all the longitudinal observations at each post-baseline visit. The model

Page 16

will include the fixed categorical effects of treatment, visit, and treatment-by-visit interaction and the continuous covariate of baseline result, where subject is treated as a random effect. The Kenward-Roger method will be used to determine denominator degrees of freedom. SAS (V8.2) PROC MIXED will be used to perform the analysis. Type 3 sums of squares will be used. The covariance structure to model the within-subject errors will be unstructured. If the unstructured covariance structure leads to lack of convergence, Akaike’s Information Criteria will be used to select the best fitting covariance structure.

9.4. Scatterplot[To be completed]

Page 17

10. ReferencesCrowe BJ, Xia A, Berlin JA, Watson DJ, Shi H, Lin SL, et. al. (2009). Recommendations for safety planning, data collection, evaluation and reporting during drug, biologic and vaccine development: a report of the safety planning, evaluation, and reporting team. Clinical Trials 2009; 6: 430-440.

Fraser C. Biological Variation: From principles to practice.

Page 18

11. Contributions [We can discuss how we want to track this. This might help us remember who to go to for follow-up questions/discussion. We can put all contributions here for now and decide later who goes on the title page (or maybe we don’t have a list on the title page).]

In addition to the guidance documents listed in the Introduction, the following internal documents were used as a starting point:

Reference Limit Discussion Document (Eli Lilly & Company)xxxx (Cross-industry Graphics Working Group)xxxx (Cross-industry Meta-analysis Working Group?)

Table 10.1. Summary of Contributions

Contributor Company Summary of ContributionMary Nilsson, MS Eli Lilly & Company Wrote first draft for most sectionsWei Wang, MSc Eli Lilly & Company Created Figures x.x, x.x., and x.x. Key contributor to first

set of proposed TFLs.Brenda Crowe, PhD Eli Lilly & CompanyDamon Disch, PhD Eli Lilly & Company Discussed transformations as part of first draftCraig Mallinkrodt, PhD Eli Lilly & Company Discussed potential application of MMRM in this settingCharles Beasley, MD Eli Lilly & Company Primary contributor to proposing the use of large clinical

trial populated based reference limits

Page 19

Note: Putting figures and tables at the end for now. Eventually they will be inserted within the document. I do not have all figures and tables available right now.

Page 20

Page 21