Setting the bar in phase II trials: The use of historical data for determining go/no go decision for definitive phase III testing

Purpose: Phase II trials aim to determine whether a cancer treatment is sufficiently promising to justify phase III study. Whether an agent is declared promising in a phase II trial depends on prespecified null and alternative rates of an outcome of interest such as tumor response. In some cases, the null must be determined with reference to historical data. We sought to determine the proportion of phase II trials that require historical data to establish the null and to determine how these historical estimates were derived. Experimental Design: We conducted a systematic review of phase II trials published in the Journal of Clinical Oncology or Cancer in the 3 years to June 2005. Data were extracted following a prespecified protocol. Results: We retrieved 251 papers, of which 117 were found to be ineligible; 70 of 134 included trials (52%) were defined as requiring historical data for design. Nearly half (32, 46%) of these papers did not cite the source of the historical data used, and just 9 (13%) clearly gave a single historical estimate as the rationale for the null. Trials that failed to cite prior data appropriately were significantly more likely to declare an agent to be active (82% versus 33%; P = 0.005). No study incorporated statistical methods to account for either sampling error or possible differences in case mix between the phase II sample and the historical cohort. Conclusions: Many phase II trials require historical data to determine null response rates. Simple guidelines may improve design and reporting of such trials.