Examining the Effects of Differential Item (Functioning and Differential) Test Functioning on Selection Decisions: When Are Statistically Significant Effects Practically Important?
Document Type
Article
Publication Date
6-2004
Keywords
item response theory, differential test functioning, effect size measures, mean raw scores, 4/5th rule, successive cut scores, personality assessment, professional licensure, college admissions
Digital Object Identifier (DOI)
https://doi.org/10.1037/0021-9010.89.3.497
Abstract
Item response theory differential test functioning (DTP) methods are often used to address issues in personnel selection, but the results are frequently difficult to interpret because statistically significant findings may have little practical importance. In this article, the authors proposed 2 effect size measures for DTP. One related DTP to mean raw score differences across groups: the other related DTP to the 4/5th rule for adverse impact at successive cut scores. The effects of DTP were examined in the context of personality assessment, professional licensure, and college admissions. Overall, the result indicated that although many items exhibited bias in analyses of the large samples, the net magnitudes of effect on potential selection decisions were nugatory.
Was this content written or created while at USF?
Yes
Citation / Publisher Attribution
Journal of Applied Psychology, v. 89, issue 3, p. 497-508
Scholar Commons Citation
Stark, Stephen; Chernyshenko, Oleksandr S.; and Drasgow, Fritz, "Examining the Effects of Differential Item (Functioning and Differential) Test Functioning on Selection Decisions: When Are Statistically Significant Effects Practically Important?" (2004). Psychology Faculty Publications. 1971.
https://digitalcommons.usf.edu/psy_facpub/1971
