Efficient Alternating Least Squares Algorithms for Low Multilinear Rank Approximation of Tensors

The low multilinear rank approximation, also known as the truncated Tucker decomposition, has been extensively utilized in many applications that involve higher-order tensors. Popular methods for low multilinear rank approximation usually rely directly on matrix SVD, therefore often suffer from the notorious intermediate data explosion issue and are not easy to parallelize, especially when the input tensor is large. In this paper, we propose a new class of truncated HOSVD algorithms based on alternating least squares (ALS) for efficiently computing the low multilinear rank approximation of tensors. The proposed ALS-based approaches are able to eliminate the redundant computations of the singular vectors of intermediate matrices and are therefore free of data explosion. Also, the new methods are more flexible with adjustable convergence tolerance and are intrinsically parallelizable on high-performance computers. Theoretical analysis reveals that the ALS iteration in the proposed algorithms is q-linear convergent with a relatively wide convergence region. Numerical experiments with large-scale tensors from both synthetic and real-world applications demonstrate that ALS-based methods can substantially reduce the total cost of the original ones and are highly scalable for parallel computing.

Automated summary

A new class of truncated HOSVD algorithms based on alternating least squares (ALS) is proposed in this paper for efficiently computing the low multilinear rank approximation of tensors. These ALS-based approaches eliminate the redundant computations of the singular vectors of intermediate matrices, thus avoiding data explosion issue and providing adjustable convergence tolerance, as well as intrinsic parallelizability on high-performance computers. Theoretical analysis shows that the ALS iteration in the proposed algorithms is q-linear convergent with wide convergence region, and numerical experiments demonstrate that ALS-based methods can substantially reduce the total cost of the original ones and are highly scalable for parallel computing.