Intensity Analysis and the Figure of Merit's components for assessment of a Cellular Automata - Markov simulation model

Some popular metrics to evaluate land change simulation models are misleading. Therefore, land change scientists have called for the development of methods to evaluate various aspects of modelling applications. This article answers the call by giving novel methods to compare three types of land change: 1) reference change during the calibration time interval, 2) simulation change during the validation time interval, and 3) reference change during the validation time interval. We compare these changes by using Intensity Analysis' three levels and the Figure of Merit's four components: Misses, Hits, Wrong Hits and False Alarms. We illustrate the concepts by applying a Cellular Automata - Markov land change model to a case study in northeast Hungary. We used reference maps of five land categories to calibrate the model during 2000-2006, then to validate the simulation during 2006-2012. Intensity Analysis' time interval level shows that the simulation change and the reference change decelerated from 2000-2006 to 2006-2012. Intensity Analysis' category level shows that the simulation losses were less than what a pure Markov chain would have dictated. Intensity Analysis' transition level shows that the model's Markov algorithm simulated correctly that the gain of Forest targeted Agriculture and Wetland. The Figure of Merit's components reveals more allocation error than quantity error. Our collection of metrics show that more error derived from the Cellular Automata algorithm than from the Markov algorithm. We recommend that scientists use Intensity Analysis and the Figure of Merit's components to reveal various fundamental aspects of modelling applications.