Journal
STAT
Volume 10, Issue 1, Pages -Publisher
WILEY
DOI: 10.1002/sta4.413
Keywords
bootstrap; prediction; resampling methods; statistical learning
Categories
Funding
- NSF [1915-842]
Ask authors/readers for more resources
This study focuses on estimating the generalization error of a CV-tuned predictive model and proposes the use of an honest leave-one-out cross-validation framework for an unbiased estimator. Demonstrations with kernel SVM and kernel logistic regression show competitive performance even against the state-of-the-art .632+ estimator.
Many machine learning models have tuning parameters to be determined by the training data, and cross-validation (CV) is perhaps the most commonly used method for selecting tuning parameters. This work concerns the problem of estimating the generalization error of a CV-tuned predictive model. We propose to use an honest leave-one-out cross-validation framework to produce a nearly unbiased estimator of the post-tuning generalization error. By using the kernel support vector machine and the kernel logistic regression as examples, we demonstrate that the honest leave-one-out cross-validation has very competitive performance even when competing with the state-of-the-art .632+ estimator.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available