Skip to content

Evaluating collaborative filtering recommender systems

Jonathan L. Herlocker, Joseph A. Konstan, Loren Terveen, John Riedl

ACM Transactions on Information Systems · 2004 · 5,975 citations

No retraction on record in OpenAlex.Check Crossref now

Cite this paper

Herlocker, J. L., Konstan, J. A., Terveen, L., & Riedl, J. (2004). Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22(1), 5–53. https://doi.org/10.1145/963770.963772

In text: (Herlocker et al., 2004)

Ask a question about this research

One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.

Abstract

Recommender systems have been evaluated in many, often incomparable, ways. In this article, we review the key decisions in evaluating collaborative filtering recommender systems: the user tasks being evaluated, the types of analysis and datasets being used, the ways in which prediction quality is measured, the evaluation of prediction attributes other than quality, and the user-based evaluation of the system as a whole. In addition to reviewing the evaluation strategies used by prior researchers, we present empirical results from the analysis of various accuracy metrics on one content domain where all the tested metrics collapsed roughly into three equivalence classes. Metrics within each equivalency class were strongly correlated, while metrics from different equivalency classes were uncorrelated.

  1. Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions2005
  2. Authoritative sources in a hyperlinked environment1999
  3. An algorithm for suffix stripping1980
  4. A vector space model for automatic indexing1975
  5. Content-based image retrieval at the end of the early years2000

Metadata from OpenAlex (CC0). Citations are generated from the published record.