4.7 Article

On how data are partitioned in model development and evaluation: Confronting the elephant in the room to enhance model generalization*

相关参考文献

注意:仅列出部分参考文献,下载原文获取全部文献信息。
Article Environmental Sciences

Symptoms of Performance Degradation During Multi-Annual Drought: A Large-Sample, Multi-Model Study

Luca Trotter et al.

Summary: Hydrologic models are important tools for understanding climate change impacts, but they perform poorly in transitory climate conditions. This study examines the performance of five rainfall-runoff models in Australian catchments during a 13-year drought. The research finds that the models' performance degrades extensively during and after the drought, primarily due to overestimation of flow. The study also highlights the models' deficiencies in representing flow delay, moisture deficits, and removal before reaching the stream.

WATER RESOURCES RESEARCH (2023)

Article Computer Science, Interdisciplinary Applications

TALKS: A systematic framework for resolving model-data discrepancies

Maria P. Vilas et al.

Summary: Models and data are crucial for decision-making in environmental systems, offering diverse and complementary information. The TALKS framework is introduced to address discrepancies between models and data, highlighting the importance of identifying and improving issues in both. Through three case studies, it demonstrates how models can be used to enhance the quality of data and integrate models and data more effectively.

ENVIRONMENTAL MODELLING & SOFTWARE (2023)

Article Environmental Sciences

A Robust Strategy to Account for Data Sampling Variability in the Development of Hydrological Models

Feifei Zheng et al.

Summary: It is common to divide available data into calibration and evaluation sets for hydrological model calibration. Failure to consider data sampling variability (DSV) in the calibration process can weaken the model's ability to generalize. This paper proposes a strategy called stochastic evolutionary parameter optimization (SEPO) that explicitly accounts for DSV. It uses random sub-samples to drive the optimization algorithm towards the distribution of model parameters, without requiring derivative information.

WATER RESOURCES RESEARCH (2023)

Review Computer Science, Interdisciplinary Applications

Beyond engineering: A review of reservoir management through the lens of wickedness, competing objectives and uncertainty

Wenyan Wu et al.

Summary: Traditionally, reservoir management has focused on optimizing the operation and control of engineering infrastructure systems. However, the impacts of reservoirs on society and the environment, as well as the uncertainties associated with societal values, risk appetite, and politics, have been often overlooked. This paper addresses these issues by reviewing reservoir management through the lens of wickedness, competing objectives, and uncertainty, highlighting the challenges and research efforts needed to ensure these systems best serve society and the environment.

ENVIRONMENTAL MODELLING & SOFTWARE (2023)

Article Computer Science, Interdisciplinary Applications

Exploding the myths: An introduction to artificial neural networks for prediction and forecasting

Holger R. Maier et al.

Summary: Artificial Neural Networks (ANNs), also known as models for deep learning, are widely used for predicting environmental variables. However, there is a lack of understanding about their inner workings, leading to misconceptions that applying ANNs simply involves inputting data into black-box software. This approach results in less useful models. To address these issues, this introductory paper debunks common myths about ANNs and presents state-of-the-art approaches for developing ANNs that can be confidently applied in practice.

ENVIRONMENTAL MODELLING & SOFTWARE (2023)

Article Computer Science, Interdisciplinary Applications

Fit-for-purpose environmental modeling: Targeting the intersection of usability, reliability and feasibility

Serena H. Hamilton et al.

Summary: This article proposes a practical framework that considers fitness-for-purpose modeling as the intersection of usefulness, reliability, and feasibility requirements. By applying this framework, environmental models that effectively bridge science and management can be designed and developed.

ENVIRONMENTAL MODELLING & SOFTWARE (2022)

Article Environmental Sciences

Time to Update the Split-Sample Approach in Hydrological Model Calibration

Hongren Shen et al.

Summary: This study empirically assesses how different data splitting methods influence post-validation model testing period performance in hydrological modeling. The findings suggest that calibrating to older data and then validating models on newer data produces inferior model testing period performance, while calibrating to the full available data and skipping model validation is the most robust split-sample decision. The experimental findings remain consistent across different factors and strongly support revising the traditional split-sample approach in hydrological modeling.

WATER RESOURCES RESEARCH (2022)

Article Environmental Sciences

Achieving Robust and Transferable Performance for Conservation-Based Models of Dynamical Physical Systems

Feifei Zheng et al.

Summary: In this paper, a novel strategy is proposed to improve the performance of physics-based models of dynamical systems by using continuous simulation and deterministic data allocation. The strategy addresses the challenge of ensuring distributional similarity in partitioning data into independent subsets. The results of testing on rainfall-runoff models demonstrate that the proposed strategy consistently outperforms the traditional approach, especially under conditions of larger runoff skewness.

WATER RESOURCES RESEARCH (2022)

Article Geosciences, Multidisciplinary

The Great Lakes Runoff Intercomparison Project Phase 4: the Great Lakes (GRIP-GL)

Juliane Mai et al.

Summary: This study conducted a model intercomparison to compare different model setups in simulating outputs in the Great Lakes region. The results showed that the machine-learning-based model performed the best in simulating streamflow, while the locally calibrated models and regionally calibrated models showed varying performances in different areas. The study also compared additional model outputs, such as evapotranspiration, soil moisture, and snow water equivalent, against gridded reference datasets.

HYDROLOGY AND EARTH SYSTEM SCIENCES (2022)

Article Geosciences, Multidisciplinary

Hydrological concept formation inside long short-term memory (LSTM) networks

Thomas Lees et al.

Summary: This article investigates the interpretability of using Neural Networks (LSTM) for hydrological model predictions. By studying the internal states of LSTMs, especially the cell-state vector, the research explores what information the LSTM learns about the hydrological system and compares it with known hydrological concepts. Experimental results show that LSTMs can replicate known hydrological processes, act as a method for estimating intermediate water storage, and the learned representations are interpretable by scientists.

HYDROLOGY AND EARTH SYSTEM SCIENCES (2022)

Article Computer Science, Interdisciplinary Applications

The proper care and feeding of CAMELS: How limited training data affects streamflow prediction

Martin Gauch et al.

Summary: Accurate streamflow prediction relies on historical meteorological records and streamflow measurements, but data scarcity is a common issue. This study evaluates tree and LSTM models' sensitivity to limited training data, finding LSTMs outperforming with more data. The study highlights the importance of optimizing predictions by adjusting training data limitations.

ENVIRONMENTAL MODELLING & SOFTWARE (2021)

Article Computer Science, Interdisciplinary Applications

Sensitivity analysis: A discipline coming of age

Andrea Saltelli et al.

Summary: Sensitivity analysis as a key component of scientific development and policy support has been rapidly evolving over the past three decades. Researchers and practitioners from various disciplines have contributed to the development of sensitivity analysis, fostering a community culture through conferences. The field is now maturing into an independent science with emerging applications in new areas such as data science and machine learning.

ENVIRONMENTAL MODELLING & SOFTWARE (2021)

Article Computer Science, Interdisciplinary Applications

Deep learning, explained: Fundamentals, explainability, and bridgeability to process-based modelling

Saman Razavi

Summary: This paper discusses the application of artificial intelligence (AI) and deep learning (DL) in earth and environmental sciences, addressing key issues that DL may face in practice, aiming to contribute to the future of AI-assisted earth and environmental sciences.

ENVIRONMENTAL MODELLING & SOFTWARE (2021)

Article Computer Science, Interdisciplinary Applications

Identifying critical climate conditions for use in scenario-neutral climate impact assessments

S. Culley et al.

Summary: The use of scenario-neutral climate impact assessments is increasing to evaluate water resource system responses to climate changes. These assessments aim to identify system sensitivity to different climate conditions and stressors, with results showing that the choice of climate conditions included can significantly impact the assessment outcome.

ENVIRONMENTAL MODELLING & SOFTWARE (2021)

Article Computer Science, Interdisciplinary Applications

The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support

Saman Razavi et al.

Summary: Sensitivity analysis is becoming an essential part of mathematical modeling, with untapped potential benefits for both mechanistic and data-driven modeling as well as decision making. This perspective paper revisits the current status of SA and outlines research challenges in various areas, emphasizing the need for structuring and standardizing SA as a discipline, tapping into its potential for systems modeling, addressing computational burdens, progressing SA in the context of machine learning, clarifying its relationship with uncertainty quantification, and evolving its use in decision making. An outlook for the future of SA is provided to better serve science and society.

ENVIRONMENTAL MODELLING & SOFTWARE (2021)

Article Computer Science, Interdisciplinary Applications

A modelling framework and R-package for evaluating system performance under hydroclimate variability and change

Bree Bennett et al.

Summary: The article introduces a unified five-step framework and supporting R-package, foreSIGHT, for conducting climate impact assessments in a scenario-neutral manner. The software enables the analysis of system performance, comparing current performance with alternative management or design options, and includes a case study demonstration.

ENVIRONMENTAL MODELLING & SOFTWARE (2021)

Article Environmental Sciences

Legacy, Rather Than Adequacy, Drives the Selection of Hydrological Models

N. Addor et al.

WATER RESOURCES RESEARCH (2019)

Article Computer Science, Interdisciplinary Applications

Uncertainty analysis of a semi-distributed hydrologic model based on a Gaussian Process emulator

Jing Yang et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2018)

Article Engineering, Environmental

Statistics for sample splitting for the calibration and validation of hydrological models

Dedi Liu et al.

STOCHASTIC ENVIRONMENTAL RESEARCH AND RISK ASSESSMENT (2018)

Article Engineering, Civil

The hazards of split-sample validation in hydrological model calibration

Richard Arsenault et al.

JOURNAL OF HYDROLOGY (2018)

Article Geosciences, Multidisciplinary

Rainfall-runoff modelling using Long Short-Term Memory (LSTM) networks

Frederik Kratzert et al.

HYDROLOGY AND EARTH SYSTEM SCIENCES (2018)

Article Computer Science, Interdisciplinary Applications

Improved validation framework and R-package for artificial neural network models

Greer B. Humphrey et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2017)

Review Computer Science, Interdisciplinary Applications

A review of current calibration and validation practices in land-change modeling

Jasper van Vliet et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2016)

Review Computer Science, Interdisciplinary Applications

Sensitivity analysis of environmental models: A systematic review with practical workflow

Francesca Pianosi et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2016)

Editorial Material Biochemical Research Methods

POINTS OF SIGNIFICANCE Model selection and overfitting

Jake Lever et al.

NATURE METHODS (2016)

Article Engineering, Civil

Gauging the Ungauged Basin: Relative Value of Soft and Hard Data

J. Seibert et al.

JOURNAL OF HYDROLOGIC ENGINEERING (2015)

Article Environmental Sciences

Bayesian model averaging to explore the worth of data for soil-plant model selection and prediction

Thomas Woehling et al.

WATER RESOURCES RESEARCH (2015)

Article Geosciences, Multidisciplinary

Large-sample hydrology: a need to balance depth with breadth

H. V. Gupta et al.

HYDROLOGY AND EARTH SYSTEM SCIENCES (2014)

Article Computer Science, Interdisciplinary Applications

Characterising performance of environmental models

Neil D. Bennett et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2013)

Article Water Resources

A decade of Predictions in Ungauged Basins (PUB)a review

M. Hrachowitz et al.

HYDROLOGICAL SCIENCES JOURNAL-JOURNAL DES SCIENCES HYDROLOGIQUES (2013)

Article Water Resources

Calibration of hydrological models on hydrologically unusual events

Shailesh Kumar Singh et al.

ADVANCES IN WATER RESOURCES (2012)

Article Geosciences, Multidisciplinary

The transferability of hydrological models under nonstationary climatic conditions

C. Z. Li et al.

HYDROLOGY AND EARTH SYSTEM SCIENCES (2012)

Review Computer Science, Interdisciplinary Applications

Methods used for the development of neural networks for the prediction of water resource variables in river systems: Current status and future directions

Holger R. Maier et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2010)

Article Computer Science, Artificial Intelligence

Data splitting for artificial neural networks using SOM-based stratified sampling

R. J. May et al.

NEURAL NETWORKS (2010)

Article Water Resources

Multi-model ensemble hydrologic prediction using Bayesian model averaging

Qingyun Duan et al.

ADVANCES IN WATER RESOURCES (2007)

Article Computer Science, Interdisciplinary Applications

Ten iterative steps in development and evaluation of environmental models

A. J. Jakeman et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2006)

Article Environmental Sciences

Optimal division of data for neural network models in water resources applications

GJ Bowden et al.

WATER RESOURCES RESEARCH (2002)