A Novel Parallel Algorithm for Sparse Tensor Matrix Chain Multiplication via TCU-Acceleration

Analysis of multi-dimensional data, especially tensor decomposition, which extracts latent information, is becoming considerably popular. Although multi-dimensional sparse data is typically processed on multi-core processors, developing highly optimized GPU-based Sparse Tensor Matrix Chain Multiplication (SpTMCM) is challenging. The purpose of this paper is to investigate a novel approach named SpTMCM and to explore the discovery of SpTMCM coupled with the emerging computing core, Tensor Core Unit (TCU). In contrast to prior work, the proposed novel approach enables a uniform storage format and optimization approach for SpTMCM. We design a hybrid tensor format based on multi-dimensional tiling that divides the tensor depending on the tile threshold to address the inefficient memory accesses caused by the irregular nonzero distribution of the sparse tensor. Further, we develop a TCU-based tensor parallel algorithm with our novel approach to increase the memory bandwidth. Compared to stateof-the-art works, our method achieves 1.16 similar to 24.12x speedup for SpMTTKRP and 5.07 similar to 7.15x speedup for SpTTMChain across NVIDIA A100 GPU on a range of real-world sparse tensors.

Automated summary

This paper presents a novel approach called SpTMCM and investigates its coupling with the Tensor Core Unit (TCU). The proposed approach offers a uniform storage format and optimization method for SpTMCM, addressing the inefficient memory accesses caused by irregular distribution of sparse tensors. A TCU-based tensor parallel algorithm is developed to improve memory bandwidth. Experimental results show significant speedups compared to state-of-the-art methods for SpMTTKRP and SpTTMChain on real-world sparse tensors using NVIDIA A100 GPU.