Evaluation capacity building in response to the agricultural research impact agenda: Emerging insights from Ireland, Catalonia (Spain), New Zealand, and Uruguay

Performance-based funding and calls for public-funded science to demonstrate societal impact are encouraging public research organisations to evaluate impact, the so-called impact agenda. This paper explores evaluation methods of four fully or partially public-funded agricultural research organisations and how they are building evaluative capacity to respond to the impact agenda. Drawing on cross-organisational comparison of the readiness of each organisation to implement evaluation, the implications for improving evaluative capacity building (ECB) are discussed. This study extends the current literature on ECB, as very little has focussed on research organisations in general, and particularly agricultural research. Driven by the impact agenda, the organisations are beginning to emphasise summative evaluation. Organisational leaders valuing the demonstration of impact and commitment to building evaluation capacity are important precursors to other aspects of organisational readiness to implement evaluation. However, organisational emphasis remains on using evaluation for accountability and to improve efficiency and allocation of funding. The organisations have yet to systematically embed evaluation processes and capabilities for learning at programme and organisation-levels. There is, therefore, an opportunity to develop organisation and programme-level evaluation processes that inform each other and the pathways to impact from science. To realise this opportunity, organisations could strengthen internal and external networks of evaluation practitioners and academics to bridge the gap between the theory and practice of monitoring and evaluation for learning (MEL) and to begin to reshape organisational culture by using evaluation methods that are grounded in co-production and integrated scientific and societal values.

Automated summary

This paper explores the evaluation methods and capacity building of four public-funded agricultural research organisations in response to the impact agenda. The study highlights the need for improving evaluative capacity building, with an emphasis on summative evaluation. The organizations have yet to fully integrate evaluation processes and capabilities for learning, but there is an opportunity to develop organization and program-level evaluation processes to inform the pathways to impact from science. The paper also suggests strengthening networks of evaluation practitioners and academics to bridge the gap between theory and practice in monitoring and evaluation.