AutoRank: Automated Rank Selection for Effective Neural Network Customization

Tensor decomposition is a promising approach for low-power and real-time application of neural networks on resource-constrained embedded devices. This paper proposes AutoRank, an end-to-end framework for customizing neural network decomposition using cross-layer rank-selection. For many-layer networks, determining the optimal decomposition ranks is a cumbersome task. To overcome this challenge, we establish a state-action-reward system that effectively absorbs inference accuracy and platform specifications into the rank-selection policy. Our proposed framework brings platform characteristics and performance in the customization loop to enable direct incorporation of hardware cost, e.g., runtime and memory footprint. By means of this hardware-awareness, AutoRank customization engine delivers high accuracy decomposed deep neural networks with low execution cost. Our framework minimizes the engineering cost associated with rank selection by providing an automated API for AutoRank that is compatible with popular deep learning libraries and can be readily used by developers.

Automated summary

Tensor decomposition is a promising method for implementing low-power and real-time neural network applications on resource-constrained embedded devices. The proposed AutoRank framework allows customization of neural network decomposition through cross-layer rank selection, incorporating both inference accuracy and platform specifications while minimizing engineering costs. This framework is hardware-aware and delivers high accuracy decomposed deep neural networks with low execution costs, with an automated API for compatibility with popular deep learning libraries.