Using counterfactuals to evaluate the cost-effectiveness of controlling biological invasions

Prioritizing limited conservation funds for controlling biological invasions requires accurate estimates of the effectiveness of interventions to remove invasive species and their cost-effectiveness (cost per unit area or individual). Despite billions of dollars spent controlling biological invasions worldwide, it is unclear whether those efforts are effective, and cost-effective. The paucity of evidence results from the difficulty in measuring the effect of invasive species removal: a researcher must estimate the difference in outcomes (e.g. invasive species cover) between where the removal program intervened and what might have been observed if the program had not intervened. In the program evaluation literature, this is called a counterfactual analysis, which formally compares what actually happened and what would have happened in the absence of an intervention When program implementation is not randomized, estimating counterfactual outcomes is especially difficult. We show how a thorough understanding of program implementation, combined with a matching empirical design can improve the way counterfactual outcomes are estimated in nonexperimental contexts. As a practical demonstration, we estimated the cost-effectiveness of South Africa's Working for Water program, arguably the world's most ambitious invasive species control program, in removing invasive alien trees from different land use types, across a large area in the Cape Floristic Region. We estimated that the proportion of the treatment area covered by invasive trees would have been 49% higher (5.5% instead of 2.7% of the grid cells occupied) had the program not intervened. Our estimates of cost per hectare to remove invasive species, however, are three to five times higher than the predictions made when the program was initiated. Had there been no control (counterfactual), invasive trees would have spread on untransformed land, but not on land parcels containing plantations or land transformed by agriculture or human settlements. This implies that the program might have prevented a larger area from being invaded if it had focused all of its clearing effort on untransformed land. Our results show that, with appropriate empirical designs, it is possible to better evaluate the impacts of invasive species removal and therefore to learn from past experiences.