4.7 Article

Population-Based Hyperparameter Tuning With Multitask Collaboration

出版社

IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
DOI: 10.1109/TNNLS.2021.3130896

关键词

Task analysis; Tuning; Collaboration; Statistics; Sociology; Optimization; Deep learning; Deep neural network; hyperparameter (HP) tuning; multitask learning

向作者/读者索取更多资源

In this work, a population-based hyperparameter tuning framework with multitask collaboration is proposed. The framework utilizes parallel and sequential phases to enhance generalization abilities and incorporates meta-information from existing tasks to assist the initialization of new tasks. Experimental results show significant improvements in neural network generalization abilities and better performance in multitask metalearning.
Population-based optimization methods are widely used for hyperparameter (HP) tuning for a given specific task. In this work, we propose the population-based hyperparameter tuning with multitask collaboration (PHTMC), which is a general multitask collaborative framework with parallel and sequential phases for population-based HP tuning methods. In the parallel HP tuning phase, a shared population for all tasks is kept and the intertask relatedness is considered to both yield a better generalization ability and avoid data bias to a single task. In the sequential HP tuning phase, a surrogate model is built for each new-added task so that the metainformation from the existing tasks can be extracted and used to help the initialization for the new task. Experimental results show significant improvements in generalization abilities yielded by neural networks trained using the PHTMC and better performances achieved by multitask metalearning. Moreover, a visualization of the solution distribution and the autoencoder's reconstruction of both the PHTMC and a single-task population-based HP tuning method is compared to analyze the property with the multitask collaboration.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据