Ordinal Content Validity Assessment
Also known as: ordinal CVI, Likert-scale content validity, ordinal expert rating validity, graded content validity
Ordinal content validity replaces the traditional binary (yes/no) expert relevance judgment with a graded, Likert-type rating scale, allowing richer expert opinion to be captured when evaluating whether scale items adequately represent the intended construct domain.
Read the full method
Sign in with a free account to read this section.
Method map
The neighbourhood of related methods — select a node to explore.
When to use it
Use ordinal content validity when you are developing or validating a scale, questionnaire, or test and want expert judgment on whether items represent the target construct. It is preferred over binary rating when experts are likely to perceive items on a continuum of fit rather than as simply in or out, and when you have a moderate panel (5–15 experts) whose nuanced ratings can be captured meaningfully. Do not use it as a substitute for construct validity evidence from factor analysis or convergent/discriminant validity studies — content validity establishes domain coverage, not the internal structure of scores. Also avoid ordinal CVI when the construct domain boundaries are unclear or when expert consensus on relevance is unlikely to be achieved, as low CVIs may reflect domain ambiguity rather than item quality.
Strengths & limitations
- Captures graded expert opinion rather than forcing a binary decision, preserving nuance in validity assessment.
- Produces item-level (I-CVI) and scale-level (S-CVI) indices that are quantitative, reportable, and comparable across studies.
- Straightforward to administer through online surveys and to compute, making it accessible for scale development researchers.
- Well-integrated with established CVI thresholds and guidelines, easing peer review acceptance.
- Compatible with a modified Delphi approach where experts are given summarized group ratings and invited to revise judgments in subsequent rounds.
- Dichotomizing the ordinal ratings for CVI computation discards some of the ordinal information the design was intended to preserve.
- CVI is sensitive to panel size: with very small panels (fewer than 5), even one dissenting expert dramatically lowers I-CVI values, and with very large panels, achieving high agreement becomes difficult.
- Content validity evidence alone is insufficient to support scale validity; it must be supplemented with construct, criterion, and/or convergent validity evidence.
- Expert selection bias can inflate CVI if panel members share similar theoretical orientations that may not represent the full construct domain.
Frequently asked
What threshold should I use for I-CVI?
For panels of 6 or more experts, an I-CVI of 0.78 or higher is the standard benchmark. For smaller panels the threshold should be higher: for 5 experts, 0.80 is recommended. These cut-offs correspond to acceptable agreement beyond chance.
Should I report S-CVI/Ave or S-CVI/UA?
Report both when possible. S-CVI/Ave (mean of item CVIs, threshold ≥ 0.90) is more lenient and widely used. S-CVI/UA (proportion of items with I-CVI = 1.00, threshold ≥ 0.80) is more stringent. Polit and Beck (2007) recommend S-CVI/Ave as the primary index and S-CVI/UA as a supplementary one.
Does high content validity mean my scale is valid?
No. Content validity is necessary but not sufficient. A high CVI shows that experts agree the items cover the construct domain, but you still need construct validity evidence from factor analysis, and ideally convergent and discriminant validity evidence from correlations with related and unrelated measures.
How is ordinal content validity different from the original Lawshe CVR?
Lawshe's Content Validity Ratio asks experts to classify each item as essential, useful, or not necessary — a 3-category ordinal judgment — and computes the proportion rating it essential minus chance expectation. The ordinal CVI approach uses a 4-point relevance scale and computes proportions above a threshold without correcting for chance, making it simpler to apply and interpret but also more susceptible to liberal inflation.
Can ordinal content validity be used in a Delphi process?
Yes. The ordinal rating format is well-suited to Delphi consensus studies: in round one, experts rate items independently; the summarized ratings are fed back; and in subsequent rounds, experts can revise their ratings in light of group consensus, iterating until stability or agreement is achieved.
Sources
- Wynd, C. A., Schmidt, B., & Schaefer, M. A. (2003). Two quantitative approaches for estimating content validity. Western Journal of Nursing Research, 25(5), 508–518. DOI: 10.1177/0193945903252998 ↗
- Polit, D. F., & Beck, C. T. (2007). The content validity index: Are you sure you know what's being reported? Critique and recommendations. Research in Nursing & Health, 30(4), 459–467. DOI: 10.1002/nur.20147 ↗
How to cite this page
ScholarGate. (2026, June 3). Ordinal Content Validity Assessment. ScholarGate. https://scholargate.app/en/psychometrics/ordinal-content-validity
Which method?
Set this method beside its closest kin and read them side by side — the library lays the books on the table; the choice is yours.
- Content Validity RatioPsychometrics↔ compare
- EFAStatistics↔ compare
- Item AnalysisPsychometrics↔ compare
- Scale developmentPsychometrics↔ compare