Holsti's Method
Also known as: Holsti reliability, Holsti's coefficient of reliability, Holsti C.R., Holsti Güvenirlik Katsayısı
Holsti's method is a percent-agreement reliability index for content analysis, popularized by Ole Holsti's 1969 textbook and derived from Osgood's earlier formula. For two coders it is twice the number of coding decisions on which they agree divided by the total number of decisions each made — a simple, intuitive measure of how often coders reach the same judgment.
Key highlights
- Extremely simple to compute and explain, making it useful for teaching and quick pilot checks.
- Handles the case where two coders code slightly different numbers of decisions through its 2M/(N1+N2) form.
- Provides an interpretable raw-agreement figure that complements chance-corrected coefficients.
- Widely reported historically, aiding comparison with older content-analysis literature.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use Holsti's method as a transparent, easy-to-communicate descriptive figure of raw coder agreement, and when comparing to older studies that reported it. It is reasonable as a first glance during codebook piloting and for variables where categories are roughly balanced so that chance agreement is modest. It is not adequate as the sole reliability statistic for publication, because it ignores chance and therefore overstates reliability whenever one category is prevalent. In any rigorous content analysis, report Holsti's percent agreement only in addition to a chance-corrected coefficient; for multiple coders, ordinal data, or missing values, rely on Krippendorff's alpha instead.
Strengths & limitations
- Extremely simple to compute and explain, making it useful for teaching and quick pilot checks.
- Handles the case where two coders code slightly different numbers of decisions through its 2M/(N1+N2) form.
- Provides an interpretable raw-agreement figure that complements chance-corrected coefficients.
- Widely reported historically, aiding comparison with older content-analysis literature.
- Makes no correction for chance agreement, so it systematically overstates reliability when one category dominates.
- Cannot distinguish skilled coding from agreement that arises mechanically from a skewed distribution.
- Restricted in its standard form to nominal agreement counts; it ignores how far apart disagreements are on ordered scales.
- Methodologists discourage its use as a standalone reliability claim, limiting its acceptability for publication.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
Is Holsti's method the same as percent agreement?
Essentially yes. When two coders code the same units, Holsti's coefficient of reliability equals the proportion of units on which they agree — ordinary percent agreement. Its 2M/(N1+N2) form generalizes this slightly to allow the two coders to have made different numbers of coding decisions. Like percent agreement, it does not correct for chance.
Why do reviewers ask for kappa or alpha when I report Holsti's coefficient?
Because Holsti's index does not discount agreement expected by chance. If one category is very common, coders agree often just by both choosing it, inflating the index without indicating genuine reliability. Reviewers want a chance-corrected coefficient — Scott's pi, Cohen's kappa, or Krippendorff's alpha — to show the agreement exceeds what the category distribution alone would produce. Report Holsti's percent agreement as a supplement, not a substitute.
Can Holsti's method handle more than two coders?
Yes, by averaging percent agreement across all coder pairs, giving a mean observed agreement for the team. But this still ignores chance and masks which specific coder pairs disagree. For multiple coders the recommended choice is Krippendorff's alpha, which corrects for chance, accommodates any number of coders and measurement levels, and handles missing data.
Sources
- 1.Holsti, O. R. (1969). Content Analysis for the Social Sciences and Humanities. Reading, MA: Addison-Wesley.ISBN 9780201029406
- 2.Krippendorff, K. (2004). Content Analysis: An Introduction to Its Methodology (2nd ed.). Thousand Oaks, CA: Sage.ISBN 9780761915454
- 3.Scott, W. A. (1955). Reliability of content analysis: The case of nominal scale coding. Public Opinion Quarterly, 19(3), 321–325.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 22). Holsti's Method. ScholarGate. https://scholargate.app/communication/holsti-reliability