Benefits of realist evaluation for rapidly changing health service delivery

Realist evaluation is a methodology that addresses the questions: 'what works, for whom, in which circumstances, and how?'. In this approach, programme theories are developed and tested against available evidence. However, when complex interventions are implemented in rapidly changing environments, there are many unpredictable forces that determine the programme's scope and architecture, as well as resultant outcome. These forces can be theorised, in real time, and included in realist evaluation outputs for current and future optimisation of programmes. Reflecting on a realist evaluation of first-contact physiotherapy in primary care (the FRONTIER Study), five important considerations are described for improving the quality of realist evaluation outputs when studying rapidly changing health service delivery. These are: (1) ensuring that initial programme theories are developed through creative thinking sessions, empirical and non-empirical literature, and stakeholder consultation; (2) testing the causal impact of formal and informal (eg, emergent) components of service delivery models; (3) contrasting initial programme theories with rival theory statements; (4) envisioning broad system impacts beyond the immediate implementation setting; and (5) incorporating rapidly evolving service developments and context changes into the theory testing process in real-time (eg, Additional Role Reimbursement Scheme, COVID-19). Through the reflections presented, the aim is to clarify the benefit of realist evaluation to assess emerging models of care and rapidly changing health service delivery.

Automated summary

Realist evaluation is a methodology that assesses the effectiveness of complex interventions by developing and validating program theories. In rapidly changing environments, unpredictable forces need to be incorporated into the evaluation process to optimize program implementation. When studying rapidly changing health service delivery, initial program theories can be developed through creative thinking, empirical literature, and stakeholder consultation, and the causal impact of different components can be tested to improve the quality of evaluation outputs.