Paired Comparison Method
Also known as: Method of Paired Comparisons, Pairwise Comparison Task, Pair-Comparison Ranking, Pairwise Judgment Elicitation
The paired comparison method is a systematic elicitation technique in which informants are shown every possible pair of items from a set and asked, for each pair, which member better fits a single criterion — which is sweeter, more dangerous, more prestigious, or more similar to a reference. Because every item is judged against every other item, the procedure forces fine, transitive discriminations that a one-shot ranking would blur. Aggregating the pairwise verdicts across informants yields a dominance or proximity matrix from which a stable rank order or an interval scale can be recovered.
Key highlights
- Decomposes a hard global ranking into easy two-item judgments, improving informant accuracy and reliability.
- Provides built-in consistency checks through transitivity and circular-triad counts.
- Supports interval-level scaling via Thurstone Case V, not just an ordinal rank.
- Works well with low-literacy informants because each trial is a simple binary choice.
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Use paired comparison when you need a precise rank order or an interval scale of a modest number of items on a single, well-defined criterion, and when respondent accuracy matters more than speed. It is ideal for scaling preferences, perceived severity, prestige, or similarity in cultural domain work, and for building consensus scales where transitivity can be checked. It assumes the criterion is unidimensional and that the item set is small enough that all pairs are feasible — the number of comparisons grows quadratically, so beyond roughly fifteen items the task becomes burdensome. It is poorly suited to large item sets, to multidimensional or vague criteria, or when you only need a coarse grouping rather than a fine ordering.
Strengths & limitations
- Decomposes a hard global ranking into easy two-item judgments, improving informant accuracy and reliability.
- Provides built-in consistency checks through transitivity and circular-triad counts.
- Supports interval-level scaling via Thurstone Case V, not just an ordinal rank.
- Works well with low-literacy informants because each trial is a simple binary choice.
- The number of comparisons grows as k(k−1)/2, making large item sets impractical and fatiguing.
- Assumes the comparison criterion is unidimensional; results are uninterpretable if items are judged on shifting bases.
- Thurstone scaling rests on normal-distribution assumptions that may not hold for all judgment processes.
- Intransitive or near-tie judgments require ad hoc handling that can affect the recovered scale.
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
How many items can a paired-comparison study handle?
Because every pair must be judged, the number of trials is k(k−1)/2: ten items mean 45 comparisons, fifteen items mean 105, and twenty items mean 190. In practice the item set is kept to roughly a dozen or fewer per informant so that fatigue does not degrade the later judgments; larger sets are handled with balanced incomplete designs that show each informant only a subset of pairs.
What is Thurstone Case V and why use it?
Case V is the simplest form of Thurstone's law of comparative judgment, assuming equal and uncorrelated discriminal variances for all items. Under it the scale distance between two items equals the inverse normal transform of the proportion of times one beats the other, so win proportions become interval distances. It is preferred over raw win counts when you need a genuine measurement scale rather than just an ordering.
What do intransitive judgments tell me?
Circular triads — where A beats B, B beats C, but C beats A — signal either that the criterion is not truly unidimensional, that informants are guessing on near-ties, or genuine heterogeneity in the group. Counting circular triads gives a coefficient of consistency; a high rate of intransitivity is a warning that a single scale may not faithfully represent the data.
Sources
- 1.Weller, S. C., & Romney, A. K. (1988). Systematic Data Collection. Qualitative Research Methods Series 10. Newbury Park, CA: Sage.ISBN 9780803930742
- 2.Bernard, H. R. (2017). Research Methods in Anthropology: Qualitative and Quantitative Approaches (6th ed.). Lanham, MD: Rowman & Littlefield.ISBN 9780759112421
You have read it. What now?
Cite this page
ScholarGate. (2026, June 22). Paired Comparison Method. ScholarGate. https://scholargate.app/anthropology/paired-comparison-method