A global comparative analysis of impact evaluation methods in estimating the effectiveness of protected areas

Impact evaluation aims to estimate the effect of an intervention on intended, and perhaps unintended, outcomes compared to the outcomes of no intervention or different intervention. Traditional impact evaluation methods used in environmental sciences tend to compare protected and control areas that differ in several characteristics, thereby hampering the attribution of causality such as lower rates of deforestation occurring as consequence of protection. To overcome this problem, counterfactual methods have been developed to improve impact evaluation in environmental sciences, including studies that aim to measure the effects of protected areas in avoiding deforestation. The goal of counterfactual methods is achieved by identification of carefully selected and comparable control areas. Here, we report on a systematic review to evaluate whether estimates about the effectiveness of protected area differ between traditional and counterfactual impact evaluation methods. We found that estimates from traditional methods of avoided deforestation due to the establishment of protected areas were generally higher than those from counterfactual methods. However, estimates based on traditional linear models and multivariate ordinations were similar to those obtained by counterfactual methods. Although rarely used, linear methods and ordinations appear promising as parts of the impact evaluation toolbox, although their limitations need to be better understood.