Evaluation of regional climate services: Learning from seasonal-scale examples across the Americas

A growing body of scientific work is focused on the evaluation of climate services. This literature has helped to characterize the relative success of individual services; it has also helped weigh the pros and cons of particular evaluation methods in context. This literature has been less useful in providing a snapshot of the extent to which evaluation has become a regular practice among climate service providers, nor what that regular practice might look like. This study addresses this gap by reviewing the practices of evaluation associated with 19 seasonal climate information services offered regionally across the Americas. All but one of the climate services included in the study have undergone some form of evaluation, though evaluation practices vary from case to case. The paper traces this variation to the different evaluation goals that climate services providers articulate, as well as the challenges those providers face in implementing their preferred evaluation plans. Interpreting the results of this analysis, this paper argues that regional climate service providers should consider a three-phase approach to evaluation, beginning with (1) rigorous efforts to understand who accesses their information; developing (2) a nuanced understanding of how that information is used; and culminating with (3) a characterization of the utility and value of the information in context. This study also identifies the role that long-term partnerships might play in helping regional climate service providers access the capacity needed to advance evaluation with respect to these goals, and concludes by discussing the implications of this work for those organizations seeking to build climate service capacity at regional scales.