Multi-metric evaluation of the models WARM, CropSyst, and WOFOST for rice

WARM (Water Accounting Rice Model) simulates paddy rice (Oryza sativa L), based on temperature-driven development and radiation-driven crop growth. It also simulates: biomass partitioning, floodwater effect on temperature, spikelet sterility, floodwater and chemicals management, and soil hydrology. Biomass estimates from WARM were evaluated and compared with the ones from two generic crop models (CropSyst, WOFOST). The test-area was the Po Valley (Italy). Data collected at six sites from 1989 to 2004 from rice crops grown under flooded and non-limiting conditions were split into a calibration (to estimate some model parameters) and a validation set. For model evaluation, a fuzzy-logic based multiple-metrics indicator (MQI) was used: 0 (best) <= MQI <= 1 (worst). WARM estimates compared well with the actual data (mean MQI = 0.037 against 0.167 and 0.173 with CropSyst and WOFOST, respectively). On an average, the three models performed similarly for individual validation metrics such as modelling efficiency (EF>0.90) and correlation coefficient (R>0.98). WARM performed best in a weighed measure of the Akaike Information Criterion: (worst) 0 < W-k < 1 (best), considering estimation accuracy and number of parameters required to achieve it (mean w(k) = 0.983 against 0.007 and similar to 0.000 with CropSyst and WOFOST, respectively). WARM results were sensitive to 30% of the model parameters (ratio being lower with both CropSyst, <10%, and WOFOST, <20%), but appeared the easiest model to use because of the lowest number of crop parameters required (10 against 15 and 34 with CropSyst and WOFOST, respectively). This study provides a concrete example of the possibilities offered using a range of assessment metrics to evaluate model estimates, predictive capabilities, and complexity. (C) 2009 Elsevier B.V. All rights reserved.