Measuring the Accuracy of Diagnostic Systems

Abstract

Diagnostic systems of several kinds are used to distinguish between two classes of events, essentially "signals" and "noise". For them, analysis in terms of the "relative operating characteristic" of signal detection theory provides a precise and valid measure of diagnostic accuracy. It is the only measure available that is uninfluenced by decision biases and prior probabilities, and it places the performances of diverse systems on a common, easily interpreted scale. Representative values of this measure are reported here for systems in medical imaging, materials testing, weather forecasting, information retrieval, polygraph lie detection, and aptitude testing. Though the measure itself is sound, the values obtained from tests of diagnostic systems often require qualification because the test data on which they are based are of unsure quality. A common set of problems in testing is faced in all fields. How well these problems are handled, or can be handled in a given field, determines the degree of confidence that can be placed in a measured value of accuracy. Some fields fare much better than others.

Cite this paper

Related papers

1. The Measurement of Observer Agreement for Categorical Data1977
2. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research2016
3. Intraclass correlations: Uses in assessing rater reliability.1979
4. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies2007
5. Meta-analysis of Observational Studies in Epidemiology A Proposal for Reporting2000