Regional differences in multinational clinical trials: anticipating chance variation

Background Multinational clinical trials efficiently pool resources and provide treatment comparisons across diverse populations. However, they may be difficult to interpret when there are larger than expected differences in region-specific treatment effects, such as a positive study that includes regions favoring the control arm. Purpose This article investigates the extent of chance variation that can be expected in regional treatment effects from multinational studies. It advocates studying this expected variation during the design stage, hence limiting the potential for surprises and misinterpretations at the end of the study. Methods The theory of order statistics was used to quantify chance variation between regions, assuming a homogeneous treatment effect. The expected values of the smallest and largest treatment difference were used to calculate the expected range of regional effects. This range was supplemented by the probability of observing at least one regional effect favoring the control arm. Results Chance variation led to a wide range of expected regional effects. For a study with five regions and 80% power, the expected regional treatment effects ranged from no difference to double the true difference, while the probability of observing a region favoring the control was approximately 50%. With 10 regions this probability exceeded 85% and the expected range of regional effects extended to values substantially favoring the control. Increasing the power of the study to 90% or more offered little protection against wide variation in regional effects. Limitations The proposed approach does not replace heterogeneity testing at the end of the study, but provides useful supporting information for interpreting such tests. Conclusions Large differences between regions should be anticipated in multinational studies. The expected range of treatment effects should be assessed during study design in order to inform stakeholders and calibrate expectations. Clinical Trials 2010; 7: 147-156. http://ctj.sagepub.com.