Journal
SIAM JOURNAL ON MATRIX ANALYSIS AND APPLICATIONS
Volume 30, Issue 2, Pages 844-881Publisher
SIAM PUBLICATIONS
DOI: 10.1137/07070471X
Keywords
CU R matrix decomposition; random sampling algorithms; data analysis; approximate least squares
Categories
Ask authors/readers for more resources
Many data analysis applications deal with large matrices and involve approximating the matrix using a small number of components. Typically, these components are linear combinations of the rows and columns of the matrix, and are thus difficult to interpret in terms of the original features of the input data. In this paper, we propose and study matrix approximations that are explicitly expressed in terms of a small number of columns and/or rows of the data matrix, and thereby more amenable to interpretation in terms of the original data. Our main algorithmic results are two randomized algorithms which take as input an m x n matrix A and a rank parameter k. In our first algorithm, C is chosen, and we let A' = CC+ A, where C+ is the Moore-Penrose generalized inverse of C. In our second algorithm C, U, R are chosen, and we let A' = CUR. (C and R are matrices that consist of actual columns and rows, respectively, of A, and U is a generalized inverse of their intersection.) For each algorithm, we show that with probability at least 1 - delta , parallel to A - A'parallel to F <= (1 + epsilon) parallel to A - A(k)parallel to(F), where A(k) is the best rank-k approximation provided by truncating the SVD of A, and where parallel to X parallel to(F) is the Frobenius norm of the matrix X. The number of columns of C and rows of R is a low-degree polynomial in k, 1/epsilon, and log(1/delta). Both the Numerical Linear Algebra community and the Theoretical Computer Science community have studied variants of these matrix decompositions over the last ten years. However, our two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist. Both of our algorithms are simple and they take time of the order needed to approximately compute the top k singular vectors of A. The technical crux of our analysis is a novel, intuitive sampling method we introduce in this paper called subspace sampling. In subspace sampling, the sampling probabilities depend on the Euclidean norms of the rows of the top singular vectors. This allows us to obtain provable relative-error guarantees by deconvoluting subspace information and size-of-A information in the input matrix. This technique is likely to be useful for other matrix approximation and data analysis problems.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available