Joint operation and attention block search for lightweight image restoration

Recently, block-based design methods have shown effectiveness in image restoration tasks, which are usually designed in a handcrafted manner and have computation and memory consumption challenges in practice. In this paper, we propose a joint operation and attention block search algorithm for im-age restoration, which focuses on searching for optimal combinations of operation blocks and atten-tion blocks. Specifically, we first construct two search spaces: operation block search space and atten-tion block search space. The former is used to explore the suitable operation of each layer and aims to construct a lightweight and effective operation search module (OSM). The latter is applied to dis-cover the optimal connection of various attention mechanisms and aims to enhance the feature expres-sion. The searched structure is called the attention search module (ASM). Then we combine OSM and ASM to construct a joint search module (JSM), which serves as the basic module to build the final net-work. Moreover, we propose a cross-scale fusion module (CSFM) to effectively integrate multiple hier-archical features from JSMs, which helps to mine feature corrections of intermediate layers. Extensive experiments on image super-resolution, gray image denoising, and JPEG image deblocking tasks demon-strate that our proposed network can achieve competitive performance. The source code is available on https://github.com/it-hao/JSNet .(c) 2022 Elsevier Ltd. All rights reserved.

Automated summary

This paper proposes a joint operation and attention block search algorithm for image restoration tasks. The algorithm searches for optimal combinations of operation blocks and attention blocks to construct a lightweight and effective operation search module and attention search module. The modules are combined to build the final network, and a cross-scale fusion module is introduced to integrate hierarchical features and improve feature expression.