STATISTICS IN MEDICINE, VOL. 12.451-455 (1993)

DATA MONITORING COMMITTEES FOR SOUTHWEST ONCOLOGY GROUP CLINICAL TRIALS

STEPHANIE GREEN AND JOHN CROWLEY Fred Hutchinson Cancer Research Center. Southwest Oncology Group, I124 Columbia Strcvt , MP-557, Seattle,

Washington 98104-2092, U.S.A.

SUMMARY A monitoring committee policy was established by the Southwest Oncology Group in 1985 to address documented problems in study conduct. Committees consisting of the study co-ordinators, disease commit- tee chair, study statistician, Group chair, Group statistician, an uninvolved Southwest Oncology Group clinician, an NCI representative, and a representative from each involved group for intergroup studies have been appointed for each phase Ill study. Business is conducted largely through the mail and through conference calls. Since then, problems identified in older studies have been largely resolved. Appropriate and timely decisions are being made at modest additional expense to the Group.

The Southwest Oncology Group is one of the multicentre cancer clinical trials groups doing research under co-operative agreements with the National Cancer Institute. Prior to 1985, Southwest Oncology Group trials were conducted without formal stopping guidelines or data monitoring committees. Results of each ongoing trial were reported frequently, both at the semi-annual Group meetings and at national oncology meetings. An examination of 14 trials conducted under these circumstances demonstrated a variety of problems.' Declining accrual occurred in five, two were inappropriately closed early, two had no set accrual goals, and three studies were reported early as positive, but final results were not as convincing. In contrast, a group with a monitoring policy experienced minimal problems in the conduct of its trials. In this paper we examine the development and successful implementation of a study monitoring policy in the Southwest Oncology Group.

A change in statistical centres in late 1984 presented the opportunity to evaluate Group policies and procedures. As part of this evaluation, members of the statistical centre and the operations office led discussions on the consequences of repeated testing and reporting of results. Presenta- tions on the issues emphasized the number of incorrect conclusions expected when studies are analysed frequently. Examples of misleading early reports and multiple conflicting reports of Southwest Oncology Group trials were given. The suggestion that such reporting could be detrimental generated considerable debate, with objections centring on ethical issues. Certain clinicians felt that it was their individual responsibility to judge whether interim evidence warranted continued participation in trials. They were not convinced that monitoring commit- tees would have sufficient first-hand information to make decisions in the best interests of patients. There was also concern that interest in participation would decline if results were not provided, with a corresponding decrease in patient accrual and meeting attendance.

0277-671 5 /93/05045l4l5$07.50 0 1993 by John Wiley & Sons, Ltd.

452 S. GREEN AND J. CROWLEY

A sufficient number of investigators were convinced that treatment decisions based on early impressions of the data are often wrong. With strong support from the Group chair, the Board of Governors of the Southwest Oncology Group voted to establish monitoring committee policy in July 1985. Monitoring committee responsibilities were to include (1) evaluation of accumulating survival, relapse, response, and toxicity data for convincing evidence of benefit or of no benefit; (2) recommendations of design changes; and (3) decisions when to stop accrual and when to report results. Survival, relapse, and response data were to remain confidential within the monitoring committees. It was considered important for proper patient management, however, to continue unblinded reporting of toxicity data to the general membership.

The model of independent monitoring committees used in other NIH institutes is not feasible in the cancer cooperative groups. There are 152 open, phase 111 trials in the groups; the Southwest Oncology Group is currently coordinating or participating in 44 of these. The cost alone of flying ten independent monitoring committee members per trial’ to meeting places every six months would be prohibitive. In addition, so many clinical cancer researchers are involved in at least one of the cooperative groups that it would be difficult to identify a sufficient number of cancer experts without substantial conflicts to serve on so many committees.

If they are objective, the people most knowledgeable about a trial should also have the best judgement concerning its conduct; thus the study statistician, study coordinators, and disease committee chair were identified as key members of Southwest Oncology Group monitoring committees. Objectivity was enhanced by inclusion of a Southwest Oncology Group investigator not directly involved in the trial and a representative of the National Cancer Institute. As the Group officials responsible for proper conduct of all group trials, the Group chair and Group statistician were named ex oflcio members of all monitoring committees.

It was anticipated that major issues would arise infrequently during the conduct of trials. Thus the decision was made to minimize cost and inconvenience by doing most committee business by mail or conference calls. Meetings were to be held only when needed, and scheduled at Group meetings (which most members attend) or American Society of Clinical Oncology meetings (which intergroup members also attend).

The policy, as amended in January 1986 and July 1991, is as follows:

“BACKGROUND

Interim evaluations of comparative trials are necessary in order to monitor for early extreme therapeutic results, as well as for excess toxicity and practical difficulties. Routine reporting of some of these evaluations, those concerning side effects of treat- ment and study conduct, is necessary to apprise participating physicians of safety issues and to improve study quality. Routine reporting of interim therapeutic results, however, is not only unnecessary but also detrimental to study quality.

One source of difficulty lies in what constitutes an ‘extreme therapeutic result.’ Investi- gators might wish to consider a result extreme and close a study whenever the p-value of a test statistic falls below .05. There are at least two major problems with this approach. The first problem has to do with the long term nature of most clinical trials. For instance, early separation of time to relapse curves is not convincing evidence of long term remission or survival benefit for the superior arm. If a study is closed on the basis of early differences, the ability to make long term evaluations will be compromised due to the limited sample size.

DATA MONITORING COMMITTEES FOR ONCOLOGY CLINICAL TRIALS 453

The second problem is that the probability of a false positive result is greatly increased by allowing repeated testing at the -05 level. For example, assume a two arm study accrues 120 patients over three years. If each treatment has the same true survival distribution, the probability of falsely concluding there is a difference in survival when the log-rank test is performed at the end of four years is .05. If an additional test at two years is allowed, the probability of concluding there is a difference at that time is also .05. However, the experiments which reject at two years will not generally be the same as those that reject at four. In this example, if the true median survival in both groups is one year, the probability of falsely concluding there is a difference at either two or four years is nearly .1. By testing repeatedly and reporting whenever p < .05, the probability of obtaining a false positive can increase to over .25. A related problem is that of false negative results. If studies are stopped early due to no observed differences, the probability of missing a true difference can be increased.

These problems are addressed by requiring very small p-values to close a study early (e.g., p < 405). Using this approach, the ability to compare long term results is given up much less frequently. Furthermore, stringent closing requirements at interim analyses allow final analyses to be performed at nearly the usual -05 level while maintaining an overall 5% chance for false positive results for the entire study.

A second source of difficulty with interim reporting is that a study can be stopped informally as well as formally. Presentation of early results with p-values indicated at the .05 or -01 level (or with no p-values but widely separated curves or identical curves shown) is likely to result in reduced accrual, with many physicians treating according to the current best (or least toxic) regimen rather than randomizing their patients. Further- more, decisions to close a study are rarely as straightforward as examining a single test statistic; many factors must be carefully considered by knowledgeable participants before a decision to terminate is made.

POLICY

A system of monitoring committees has been implemented for Phase 111 studies. For each study, the committee will consist of the Study Coordinator(s), the Group Chair, Group Statistician, the Statistician on the study, the Disease Committee Chair, Disci- pline Committee Chair (if applicable), a member of the Southwest Oncology Group not involved in the study to be appointed by the Group Chair, an NCI representative, and one representative of each involved Group for Intergroup studies. An evaluation will be prepared by the Study Coordinator and Statistician and distributed to committee members before the semi-annual meetings, along with recommendations on whether to close the study and whether to report the results. Monitoring committee members will indicate to the Statistician and Study Coordinator any disagreements. If necessary, a meeting of the monitoring committee will be scheduled at the semi-annual meeting. If no decision can be reached, the issue will be resolved by the Group Chair. Only monitoring committee members will have access to survival, relapse and response data, although toxicity and accrual information will still be presented to the general member- ship. The exception to this will be confidential discussions of early results with investiga- tors planning follow-up studies.”

454 S. GREEN AND J. CROWLEY

Phase I1 studies are also covered in the policy. Response information is not presented on open phase I1 studies. Formal monitoring committees are not considered necessary. The study co-ordinator and statistician serve in this role unless disagreements arise; in this case, decisions are made by the disease committee chair.

None of the anticipated decline in accrual or meeting attendance occurred after implementa- tion of the policy. In fact, the Southwest Oncology Group has experienced an unprecedented increase in registrations, from about 4000 to more than 8000 per year. Meeting attendance has also doubled in the same period.

Thirteen randomized studies managed by the Southwest Oncology Group have been opened and completed since the policy was established. (Five others had inadequate accrual from the beginning to support the study and were closed for this reason.) Only ten formal meetings had to be convened at Group meetings for these 13 studies; resolution of disputes by the Group chair was not required for any.

All 13 had specific accrual goals and early stopping and reporting guidelines. Three were appropriately stopped early, five completed accrual ( f lo%), and three accrued up to 29 per cent additional patients in order to better address important subsets. One trial overaccrued by 21 per cent owing to unexpectedly rapid accrual. The other study underaccrued by 1 1 per cent owing to a very high ineligibility rate. Accrual patterns were fairly stable after initial increases. Four of the ten studies not stopped early were reported before the time of the planned final analysis owing to definitive results at interim analyses after accrual was complete. No study has had misleading early results reported.

The trials stopped or reported early provide examples of how Southwest Oncology Group monitoring committees function. Study 8412 is a study of carboplatin plus cyclophosphamide versus cisplatin plus cyclophosphamide in stage I11 and IV ovarian cancer. The study was designed to demonstrate equivalent therapeutic benefit on the two arms. Monitoring committee activity for this trial included approval of a change of primary endpoints from pathologic CR (for which we were not getting complete information) to survival, and a decision to stop accrual and report the trial early. At the time of the first formal interim analysis (at approximately one-quarter of the anticipated number of deaths), a 30 per cent improvement in survival due to the cisplatin arm was ruled out at the specified level for early stopping; in fact, at this point the carboplatin arm appeared superior with respect to survival. The decision to stop the trial was not clear-cut, however. The apparent lack of survival benefit due to cisplatin and the clear superiority of carboplatin with respect to most of the severe cisplatin toxicities had to be weighed against an increase in thrombocytopenia on the carboplatin arm, inconclusive results with respect to response and time to failure, and no long-term survival information. Discussion included observations from investigators treating patients on the trial that they were relieved when their patients were randomized to carboplatin. (Carboplatin was about to be approved by the FDA but had not yet been, so patients had to be registered on study for access to the drug.) Discussion also included thoughtful comments from the NCI representative concerning the risk of an equivocal trial after further follow-up, if the study were closed early. After a formal meeting and two rounds of letters it was decided that results were sufficiently convincing to stop accrual and to report the trial.

Study 8812 provides another example of a decision by the monitoring committee to stop a trial early. This study in limited small-cell lung cancer was designed primarily to detect an improve- ment in survival of responders due to the addition of interferon to brain radiotherapy. Formal interim analyses were to be performed after approximately 400 and 600 responders had been accrued. A secondary goal was to detect a decrease in the number of severe infections in patients receiving induction therapy due to the addition of GM-CSF (granulocyte-macrophage colony

DATA MONITORING COMMITTEES FOR ONCOLOGY CLINICAL TRIALS 455

stimulating factor, used to stimulate the regeneration of white blood cells killed during chemo- therapy). The early stopping guideline for this endpoint called for an interim analysis after 160 patients had been evaluated for infection; the final analysis for this endpoint was to be done after 350 patients. The monitoring committee recommended early closure of the GM-CSF randomiz- ation after 160 patients had been evaluated (1 76 registered). The reason was an unanticipated and striking increase in the number of patients with life-threatening thrombocytopenia on the GM-CSF arm. There was also an increase in the number of severe infections, although this may have been due in part to misdiagnosis of radiation pneumonitis. Accrual to the no CSF arm and randomization to maintenance were continued for another six months. At this time the monitor- ing committee closed the rest of the trial with only 125 on the maintenance randomization owing to severe compliance problems. One-half of all patients on interferon were refusing therapy before relapse despite only moderate toxicity.

Study 8642 completed accrual, but was not scheduled for reporting until after one and a half additional years of follow-up. This study was designed to compare treatment with no adjuvant therapy versus gamma interferon in patients with resected malignant melanoma. At the time of the first interim analysis after accrual was complete, the observed direction of the difference favoured the observation arm. The monitoring committee concluded that the results definitively ruled out a moderate benefit due to gamma interferon and should be provided to the NCI and another co-operative group which was designing a new adjuvant melanoma trial that included a gamma interferon arm. A letter was published in the Journal ofthe National Cancer Institute announcing the decision; the other group changed the design based on our results. The observed difference has decreased since then, but there remains no evidence of benefit from adjuvant gamma interferon in malignant melanoma.

In conclusion, the monitoring committee policy established by the Southwest Oncology Group seems to work. The problems identified concerning study conduct in older Group studies have largely been resolved. Monitoring committee members appear to take their responsibilities seriously and to be making appropriate and timely decisions. Further, this is being accomplished at minimal additional expense to the Group or to NCI.

REFERENCES 1. Green, S., Fleming, T. and OFallon, J. 'Policies for study monitoring and interim reporting of results',

Journal OJ Clinical Oncology, 9, 1477-1484 (1987). 2. Hawkins, B. 'Data monitoring committees for multicenter clinical trials sponsored by the National

Institutes of Health: 1. Roles and membership of data monitoring committees for trials sponsored by the National Eye Institute', Controlled Clinical Trials. 12, 424437 (1991).