4.7 Article

RL-DARTS: Differentiable neural architecture search via reinforcement-learning-based meta-optimizer

期刊

KNOWLEDGE-BASED SYSTEMS
卷 234, 期 -, 页码 -

出版社

ELSEVIER
DOI: 10.1016/j.knosys.2021.107585

关键词

Neural architecture search; Reinforcement learning; Meta optimizer; Deep learning; Bi-level optimization

向作者/读者索取更多资源

This study introduces a novel differentiable search method based on reinforcement learning to optimize the architecture-parameter optimization problem in neural architecture search, aiming to improve computation efficiency, network precision, and robustness. By utilizing a double-loop algorithm to address the optimization problem in the searched super-network, the method alternates between optimizing the super-network and the meta-optimizer, leading to faster and more robust convergence.
Differentiable search approaches have attracted extensive attention recently due to their advantages in effectively finding novel neural architectures. However, these methods suffer from shortcomings on heavy computation consumption and low robustness in some cases. In this work, we propose a novel differentiable search method based on reinforcement learning, to further improve the computation efficiency, network precision, and robustness in the neural architecture search area. Our method constructs a reinforcement learning-based meta-optimizer to solve the architecture-parameter optimization problem, which is superior in properties of adaptability and robustness to fixed optimizers. This learnable meta-optimizer can alter its model parameters along with the search process to adapt the optimization procedure, making it possible to find out better structures and parameters with less time. Specifically, we formulate a double-loop algorithm to address the optimization problem in the searched super-network. Through switching between the external and internal loops, our method alternately optimizes the super-network and the meta-optimizer, which converges to the optimal location more rapidly and robustly. (C) 2021 Elsevier B.V. All rights reserved.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.7
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据