Journal
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS
Volume 32, Issue 3, Pages 1241-1253Publisher
IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2020.2982936
Keywords
Optimization; Search problems; Bayes methods; Predictive models; Measurement; Machine learning; Sampling methods; Coverage-based sample design; hyperparameter optimization; Poisson disk sampling (PDS); predictive modeling; sequential optimization
Ask authors/readers for more resources
Sampling effective solutions from large search spaces is a recurring idea in machine learning, and sequential optimization has become a popular approach. By proposing a coverage-based sample design method and developing algorithms for effective sample synthesis, we show that our approach consistently outperforms existing exploratory sampling methods in sample mining and hyperparameter optimization for supervised learning.
Sampling one or more effective solutions from large search spaces is a recurring idea in machine learning (ML), and sequential optimization has become a popular solution. Typical examples include data summarization, sample mining for predictive modeling, and hyperparameter optimization. Existing solutions attempt to adaptively trade off between global exploration and local exploitation, in which the initial exploratory sample is critical to their success. While discrepancy-based samples have become the de facto approach for exploration, results from computer graphics suggest that coverage-based designs, e.g., Poisson disk sampling, can be a superior alternative. In order to successfully adopt coverage-based sample designs to ML applications, which were originally developed for 2-D image analysis, we propose fundamental advances by constructing a parameterized family of designs with provably improved coverage characteristics and developing algorithms for effective sample synthesis. Using experiments in sample mining and hyperparameter optimization for supervised learning, we show that our approach consistently outperforms the existing exploratory sampling methods in both blind exploration and sequential search with Bayesian optimization.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available