Title

Examining the Effects of Differential Item (Functioning and Differential) Test Functioning on Selection Decisions: When Are Statistically Significant Effects Practically Important?

Document Type

Article

Publication Date

6-2004

Keywords

item response theory, differential test functioning, effect size measures, mean raw scores, 4/5th rule, successive cut scores, personality assessment, professional licensure, college admissions

Digital Object Identifier (DOI)

https://doi.org/10.1037/0021-9010.89.3.497

Abstract

Item response theory differential test functioning (DTP) methods are often used to address issues in personnel selection, but the results are frequently difficult to interpret because statistically significant findings may have little practical importance. In this article, the authors proposed 2 effect size measures for DTP. One related DTP to mean raw score differences across groups: the other related DTP to the 4/5th rule for adverse impact at successive cut scores. The effects of DTP were examined in the context of personality assessment, professional licensure, and college admissions. Overall, the result indicated that although many items exhibited bias in analyses of the large samples, the net magnitudes of effect on potential selection decisions were nugatory.

Was this content written or created while at USF?

Yes

Citation / Publisher Attribution

Journal of Applied Psychology, v. 89, issue 3, p. 497-508

Share

COinS