Skip to content

Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions

Nathan Halko, Per‐Gunnar Martinsson, Joel A. Tropp

SIAM Review · 2011 · 4,401 citationsOpen access

No retraction on record in OpenAlex.Check Crossref now

Cite this paper

Halko, N., Martinsson, P., & Tropp, J. A. (2011). Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 53(2), 217–288. https://doi.org/10.1137/090771806

In text: (Halko et al., 2011)

Ask a question about this research

One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.

Abstract

Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets. This paper presents a modular framework for constructing randomized algorithms that compute partial matrix decompositions. These methods use random sampling to identify a subspace that captures most of the action of a matrix. The input matrix is then compressed—either explicitly or implicitly—to this subspace, and the reduced matrix is manipulated deterministically to obtain the desired low-rank factorization. In many cases, this approach beats its classical competitors in terms of accuracy, robustness, and/or speed. These claims are supported by extensive numerical experiments and a detailed error analysis. The specific benefits of randomized techniques depend on the computational environment. Consider the model problem of finding the k dominant components of the singular value decomposition of an $m \times n$ matrix. (i) For a dense input matrix, randomized algorithms require $\bigO(mn \log(k))$ floating-point operations (flops) in contrast to $ \bigO(mnk)$ for classical algorithms. (ii) For a sparse input matrix, the flop count matches classical Krylov subspace methods, but the randomized approach is more robust and can easily be reorganized to exploit multiprocessor architectures. (iii) For a matrix that is too large to fit in fast memory, the randomized techniques require only a constant number of passes over the data, as opposed to $\bigO(k)$ passes for classical algorithms. In fact, it is sometimes possible to perform matrix approximation with a single pass over the data.

  1. LINCS: A linear constraint solver for molecular simulations1997
  2. GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems1986
  3. Methods of conjugate gradients for solving linear systems1952
  4. A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs1998
  5. A Singular Value Thresholding Algorithm for Matrix Completion2010

Metadata from OpenAlex (CC0). Citations are generated from the published record.