Skip to content

Distance Metric Learning for Large Margin Nearest Neighbor Classification

Kilian Q. Weinberger, Lawrence K. Saul

Journal of Machine Learning Research · 2009 · 4,652 citations

No retraction on record in OpenAlex.Check Crossref now

Cite this paper

Weinberger, K. Q., & Saul, L. K. (2009). Distance metric learning for large margin nearest neighbor classification. Journal of Machine Learning Research, 10(9), 207–244. https://doi.org/10.5555/1577069.1577078

In text: (Weinberger & Saul, 2009)

Ask a question about this research

One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.

Abstract

The accuracy of k-nearest neighbor (kNN) classification depends significantly on the metric used to compute distances between different examples. In this paper, we show how to learn a Mahalanobis distance metric for kNN classification from labeled examples. The Mahalanobis metric can equivalently be viewed as a global linear transformation of the input space that precedes kNN classification using Euclidean distances. In our approach, the metric is trained with the goal that the k-nearest neighbors always belong to the same class while examples from different classes are separated by a large margin. As in support vector machines (SVMs), the margin criterion leads to a convex optimization based on the hinge loss. Unlike learning in SVMs, however, our approach requires no modification or extension for problems in multiway (as opposed to binary) classification. In our framework, the Mahalanobis distance metric is obtained as the solution to a semidefinite program. On several data sets of varying size and difficulty, we find that metrics trained in this way lead to significant improvements in kNN classification. Sometimes these results can be further improved by clustering the training examples and learning an individual metric within each cluster. We show how to learn and combine these local metrics in a globally integrated manner.

  1. Regularization and Variable Selection Via the Elastic Net2005
  2. Reducing the Dimensionality of Data with Neural Networks2006
  3. Nonlinear Dimensionality Reduction by Locally Linear Embedding2000
  4. Representation Learning: A Review and New Perspectives2013
  5. Eigenfaces for Recognition1991

Metadata from OpenAlex (CC0). Citations are generated from the published record.