ROC Analysis (Receiver Operating Characteristic)
Receiver Operating Characteristic Analysis · Also known as: ROC curve analysis, AUC analysis, sensitivity-specificity analysis, diagnostic accuracy analysis
ROC analysis evaluates how well a continuous or ordinal test variable discriminates between two binary outcome classes. By plotting the true positive rate (sensitivity) against the false positive rate (1 − specificity) across all decision thresholds, it produces a curve whose area under the curve (AUC) quantifies overall discriminative power, ranging from 0.5 (chance) to 1.0 (perfect discrimination).
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use ROC analysis when you need to assess or compare the discriminative accuracy of a classifier or diagnostic test on a binary outcome, especially when the optimal decision threshold is not fixed in advance. It is the standard evaluation tool in clinical diagnostics, biomarker studies, machine learning classification, and credit-scoring validation. Do not use ROC analysis when the outcome is not binary or the predictor is purely categorical with no natural ordering; and be cautious interpreting AUC in heavily class-imbalanced datasets, where precision-recall curves may be more informative.
Strengths & limitations
- Threshold-independent: summarizes performance across all operating points, not just one arbitrary cutoff.
- AUC has a direct probabilistic interpretation (probability that a positive case scores higher than a negative case).
- Allows statistically rigorous comparison of multiple diagnostic tests on the same sample.
- Widely accepted and interpretable by clinicians, policy-makers, and engineers alike.
- Robust to changes in disease prevalence (unlike predictive values).
- AUC can be misleading in strongly imbalanced datasets where even a poor classifier achieves high AUC.
- Summarizing a whole curve with a single number loses information about performance at specific thresholds.
- Choice of optimal cutoff requires explicit cost assumptions that are often implicit or arbitrary.
- Assumes the predictor score is consistent across all subjects; calibration (probability accuracy) is a separate concern not captured by AUC.
Frequently asked
What is a 'good' AUC?
A common heuristic is: 0.5 = chance, 0.5–0.7 = poor, 0.7–0.8 = acceptable, 0.8–0.9 = good, >0.9 = excellent. These are rough guides. In life-critical screening, even 0.85 may be insufficient; in exploratory research, 0.70 may be noteworthy. Always report the confidence interval and interpret in domain context.
How is AUC related to the Mann-Whitney U statistic?
The AUC equals the probability that a randomly selected positive case scores higher than a randomly selected negative case. This is algebraically identical to the Mann-Whitney U statistic normalized by the product of group sizes, so the two approaches yield the same result and share the same non-parametric foundations.
Should I use ROC or precision-recall curves?
ROC curves are appropriate when both classes matter roughly equally. When positive cases are rare (e.g. disease prevalence < 5%), precision-recall curves are more sensitive to differences in model performance, because ROC AUC can remain high even when the model misses most true positives.
How do I compare two ROC curves statistically?
For paired data (same subjects tested by two methods), use the DeLong non-parametric method or the Hanley-McNeil approach, both of which account for the correlation between curves. For independent samples, bootstrapping is the standard approach.
Does a high AUC mean the test is well-calibrated?
No. AUC measures discrimination — the ability to rank positive cases above negative ones — not calibration, which is whether predicted probabilities match observed event rates. A test can have excellent AUC but be systematically overconfident or underconfident. Report calibration metrics (e.g. Hosmer-Lemeshow, calibration plots) separately.
Sources
- Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29–36. DOI: 10.1148/radiology.143.1.7063747 ↗
- Zweig, M. H., & Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4), 561–577. DOI: 10.1093/clinchem/39.4.561 ↗
How to cite this page
ScholarGate. (2026, June 3). Receiver Operating Characteristic Analysis. ScholarGate. https://scholargate.app/en/statistics/roc-analysis
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Discriminant AnalysisStatistics↔ compare
- Effect size analysisStatistics↔ compare
- Kendall's tauStatistics↔ compare
- Sensitivity and SpecificityResearch Statistics↔ compare