Design-Based Approaches to Causal Replication Studies

Recent interest in promoting replication efforts assumes that there is well-established methodological guidance for designing and implementing these studies. However, no such consensus exists in the methodology literature. This article addresses these challenges by describing design-based approaches for planning systematic replication studies. Our general approach is derived from the Causal Replication Framework (CRF), which formalizes the assumptions under which replication success can be expected. The assumptions may be understood broadly as replication design requirements and individual study design requirements. Replication failure occurs when one or more CRF assumptions are violated. In design-based approaches to replication, CRF assumptions are systematically tested to evaluate the replicability of effects, as well as to identify sources of effect variation when replication failure is observed. The paper describes research designs for replication and demonstrates how multiple designs may be combined in systematic replication efforts, as well as how diagnostic measures may be used to assess the extent to which CRF assumptions are met in field settings.

Automated summary

Recent interest in promoting replication efforts assumes well-established methodological guidance, however, no consensus exists in the methodology literature. This article addresses these challenges by describing design-based approaches for planning systematic replication studies, derived from the Causal Replication Framework (CRF). Testing CRF assumptions systematically evaluates replicability of effects and identifies sources of effect variation in replication failure.