期刊
NPJ COMPUTATIONAL MATERIALS
卷 8, 期 1, 页码 -出版社
NATURE PORTFOLIO
DOI: 10.1038/s41524-022-00794-8
关键词
-
资金
- National Science Foundation [1545481, 1636950, 1636910, 1931298]
- University of Wisconsin Harvey D. Spangler Professorship
- Direct For Computer & Info Scie & Enginr
- Div Of Information & Intelligent Systems [1636910] Funding Source: National Science Foundation
- Direct For Computer & Info Scie & Enginr
- Office of Advanced Cyberinfrastructure (OAC) [1636950] Funding Source: National Science Foundation
- Direct For Education and Human Resources
- Division Of Graduate Education [1545481] Funding Source: National Science Foundation
- Office of Advanced Cyberinfrastructure (OAC)
- Direct For Computer & Info Scie & Enginr [1931298] Funding Source: National Science Foundation
This paper presents a calibration method to improve the accuracy of uncertainty estimation in machine learning models. It demonstrates that the direct bootstrap ensemble standard deviation is not an accurate estimate of uncertainty, but can be improved through calibration.
Obtaining accurate estimates of machine learning model uncertainties on newly predicted data is essential for understanding the accuracy of the model and whether its predictions can be trusted. A common approach to such uncertainty quantification is to estimate the variance from an ensemble of models, which are often generated by the generally applicable bootstrap method. In this work, we demonstrate that the direct bootstrap ensemble standard deviation is not an accurate estimate of uncertainty but that it can be simply calibrated to dramatically improve its accuracy. We demonstrate the effectiveness of this calibration method for both synthetic data and numerous physical datasets from the field of Materials Science and Engineering. The approach is motivated by applications in physical and biological science but is quite general and should be applicable for uncertainty quantification in a wide range of machine learning regression models.
作者
我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。
推荐
暂无数据