3.8 Proceedings Paper

Shapley-NAS: Discovering Operation Contribution for Neural Architecture Search

Publisher

IEEE COMPUTER SOC
DOI: 10.1109/CVPR52688.2022.01159

Keywords

-

Funding

  1. National Key Research and Development Program of China [2017YFA0700802]
  2. National Natural Science Foundation of China [62125603, U1813218]
  3. Beijing Academy of Artificial Intelligence (BAAI)
  4. Institute for Guo Qiang, Tsinghua University

Ask authors/readers for more resources

In this paper, a Shapley value based method, Shapley-NAS, is proposed to evaluate the operation contribution for neural architecture search. By evaluating the direct influence of operations on validation accuracy, the optimal architectures with significant contributions to the tasks are derived through optimizing the supernet weights and updating the architecture parameters.
In this paper, we propose a Shapley value based method to evaluate operation contribution (Shapley-NAS) for neural architecture search. Differentiable architecture search (DARTS) acquires the optimal architectures by optimizing the architecture parameters with gradient descent, which significantly reduces the search cost. However, the magnitude of architecture parameters updated by gradient descent fails to reveal the actual operation importance to the task performance and therefore harms the effectiveness of obtained architectures. By contrast, we propose to evaluate the direct influence of operations on validation accuracy. To deal with the complex relationships between supernet components, we leverage Shapley value to quantify their marginal contributions by considering all possible combinations. Specifically, we iteratively optimize the supernet weights and update the architecture parameters by evaluating operation contributions via Shapley value, so that the optimal architectures are derived by selecting the operations that contribute significantly to the tasks. Since the exact computation of Shapley value is NP-hard, the Monte-Carlo sampling based algorithm with early truncation is employed for efficient approximation, and the momentum update mechanism is adopted to alleviate fluctuation of the sampling process. Extensive experiments on various datasets and various search spaces show that our Shapley-NAS outperforms the state-of-the-art methods by a considerable margin with light search cost. The code is available at https://github.com/Euphoria16/Shapley-NAS.git.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

3.8
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

No Data Available
No Data Available