SIBTEST
SIBTEST (Simultaneous Item Bias Test) is a non-parametric method for detecting differential item functioning (DIF) and differential test functioning (DTF) developed by Shealy and Stout (1993). Unlike parametric approaches, SIBTEST does not assume a particular item response model and directly tests whether groups differ in their probability of correct responses at equal levels of overall ability.
Key highlights
- Non-parametric: does not assume IRT or specific item response model, avoiding model misfit bias
- Simple interpretation: directly tests whether item probabilities differ across groups at equal ability levels
- Detects both item and test bias: identifies both individual items (DIF) and cumulative test bias (DTF)
- Accounts for ability matching: eliminates confounding between group ability differences and bias
- Robust: performs well across different distributions and item types (dichotomous and polytomous)
Intuition
This section is available to Pro members. Upgrade to Pro
How it works
This section is available to Pro members. Upgrade to Pro
When to use it
Apply SIBTEST when investigating whether test items function differently for different demographic groups (ethnicity, gender, language background), ensuring fair assessment across groups. Ideal when you have multiple items and groups, expect some items may be biased, and want a straightforward, assumption-light approach. Works well for large samples (100+ per group) with diverse ability distributions.
Strengths & limitations
- Non-parametric: does not assume IRT or specific item response model, avoiding model misfit bias
- Simple interpretation: directly tests whether item probabilities differ across groups at equal ability levels
- Detects both item and test bias: identifies both individual items (DIF) and cumulative test bias (DTF)
- Accounts for ability matching: eliminates confounding between group ability differences and bias
- Robust: performs well across different distributions and item types (dichotomous and polytomous)
- Sample size dependent: requires substantial samples (100+ per group) to have adequate power for individual item testing
- Ability matching constraints: requires sufficient overlap in ability distributions across groups; works poorly when groups differ dramatically in mean ability
- Score band choices: results can be sensitive to how ability bands are constructed (deciles vs. quintiles, etc.)
- Incomplete detection: may miss bias when groups have very different ability distributions across items
Common pitfalls
This section is available to Pro members. Upgrade to Pro
Applications
This section is available to Pro members. Upgrade to Pro
Frequently asked
What is the difference between DIF and DTF?
DIF (differential item functioning) tests whether individual items are biased. DTF (differential test functioning) examines whether bias accumulates across the whole test. An item can show DIF but not meaningfully contribute to DTF if the bias is small.
What sample sizes do I need for SIBTEST?
Minimum 100 examinees per group is recommended; larger samples (200+) provide better power. With very small samples, you may detect only large biases. Always conduct power analysis specific to your context.
How do I choose the number of ability bands?
Common practice is to use deciles (10 bands) or quintiles (5 bands). Sensitivity analyses with different numbers of bands help identify robust bias signals. Fewer bands increase stability; more bands increase detail.
Can SIBTEST handle polytomous (partial credit) items?
Yes. The method was extended by Chang, Mazzeo, and Roussos (1996) to handle polytomous items where respondents earn partial credit. This version compares expected scores rather than proportions correct.
What should I do if I find significant DIF?
Investigate the source: Is it truly unfair bias or a real group difference in knowledge? Combine SIBTEST with qualitative review (expert judgment of item content). Consider flagging items for review, revising them, or removing them from scoring if bias is confirmed.
Sources
- 1.Shealy, R., & Stout, W. F. (1993). A model-based standardization approach that separates true bias/DIF from group differences and detects test bias/DTF. Psychometrika, 58(2), 159-194.
- 2.Chang, H. H., Mazzeo, J., & Roussos, L. (1996). Detecting DIF for polytomously scored items: An adaptation of the SIBTEST procedure. Journal of Educational Measurement, 33(3), 333-353.
- 3.Stout, W. F. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52(4), 589-617.
You have read it. What now?
Cite this page
ScholarGate. (2026, June 3). SIBTEST. ScholarGate. https://scholargate.app/psychometrics/sibtest