Journal
SIAM JOURNAL ON SCIENTIFIC COMPUTING
Volume 38, Issue 6, Pages C748-C781Publisher
SIAM PUBLICATIONS
DOI: 10.1137/140993478
Keywords
parallel processing; linear algebra; matrix multiplication; libraries
Categories
Funding
- NSF [OCI-0850750, CCF-0917167, ACI-1148125/1340293, CCF-1320112]
- Microsoft
- Intel
- Sandia Fellowship
- Institute of Computational Engineering and Sciences
- Division of Computing and Communication Foundations
- Direct For Computer & Info Scie & Enginr [1320112] Funding Source: National Science Foundation
Ask authors/readers for more resources
We expose a systematic approach for developing distributed-memory parallel matrix-matrix multiplication algorithms. The journey starts with a description of how matrices are distributed to meshes of nodes (e.g., MPI processes), relates these distributions to scalable parallel implementation of matrix-vector multiplication and rank-1 update, continues on to reveal a family of matrix-matrix multiplication algorithms that view the nodes as a two-dimensional (2D) mesh, and finishes with extending these 2D algorithms to so-called three-dimensional (3D) algorithms that view the nodes as a 3D mesh. A cost analysis shows that the 3D algorithms can attain the (order of magnitude) lower bound for the cost of communication. The paper introduces a taxonomy for the resulting family of algorithms and explains how all algorithms have merit depending on parameters such as the sizes of the matrices and architecture parameters. The techniques described in this paper are at the heart of the Elemental distributed-memory linear algebra library. Performance results from implementation within and with this library are given on a representative distributed-memory architecture, the IBM Blue Gene/P supercomputer.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available