Skip to content

A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection

Ron Kohavi

1995 · 10,883 citationsOpen access

No retraction on record in OpenAlex.Check Crossref now

Cite this paper

Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. 2, 1137–1143. https://doi.org/10.5281/zenodo.19712698

In text: (Kohavi, 1995)

Ask a question about this research

One question, free, no account. Anchorcite finds the papers, checks each against Crossref and links every sentence of the answer to the passage it comes from.

Abstract

We review accuracy estimation methods and compare the two most common methods: crossvalidation and bootstrap. Recent experimental results on arti cial data and theoretical results in restricted settings have shown that for selecting a good classi er from a set of classiers (model selection), ten-fold cross-validation may be better than the more expensive leaveone-out cross-validation. We report on a largescale experiment| over half a million runs of C4. 5 and a Naive-Bayes algorithm| to estimate the e ects of di erent parameters on these algorithms on real-world datasets. For crossvalidation, we vary the number of folds and whether the folds are strati ed or not; for bootstrap, we vary the number of bootstrap samples. Our results indicate that for real-word datasets similar to ours, the best method to use for model selection is ten-fold strati ed cross validation, even if computation power allows using more folds.

  1. Scikit-learn: Machine Learning in Python2012
  2. LIBSVM2011
  3. SMOTE: Synthetic Minority Over-sampling Technique2002
  4. The WEKA data mining software2009
  5. The Elements of Statistical Learning: Data Mining, Inference, and Prediction2010

Metadata from OpenAlex (CC0). Citations are generated from the published record.