Graduation Year
2013
Document Type
Dissertation
Degree
Ph.D.
Degree Granting Department
Educational Measurement and Research
Major Professor
Jeffrey D. Kromrey, Ph.D.
Co-Major Professor
Yi-Hsin Chen, Ph.D.
Committee Member
Robert F. Dedrick, Ph.D.
Committee Member
Gladis Kersaint, Ph.D.
Keywords
Cognitive Diagnostic Assessment, Linear Logistit Test Model, Misspecification of the Q-matrix, Simulation
Abstract
A simulation study was conducted to explore the performance of the linear logistic test model (LLTM) when the relationships between items and cognitive components were misspecified. Factors manipulated included percent of misspecification (0%, 1%, 5%, 10%, and 15%), form of misspecification (under-specification, balanced misspecification, and over-specification), sample size (20, 40, 80, 160, 320, 640, and 1280), Q-matrix density (60% and 46%), number of items (20, 40, and 60 items), and skewness of person ability distribution (-0.5, 0, and 0.5). Statistical bias, root mean squared error, confidence interval coverage, confidence interval width, and pairwise cognitive components correlations were computed. The impact of the design factors were interpreted for cognitive components, item difficulty, and person ability parameter estimates.
The simulation provided rich results and selected key conclusions include (a) SAS works superbly when estimating LLTM using a marginal maximum likelihood approach for cognitive components and an empirical Bayes estimation for person ability, (b) parameter estimates are sensitive to misspecification, (c) under-specification is preferred to over-specification of the Q-matrix, (d) when properly specified the cognitive components parameter estimates often have tolerable amounts of root mean squared error when the sample size is greater than 80, (e) LLTM is robust to the density of Q-matrix specification, (f) the LLTM works well when the number of items is 40 or greater, and (g) LLTM is robust to a slight skewness of the person ability distribution. In sum, the LLTM is capable of identifying conceptual knowledge when the Q-matrix is properly specified, which is a rich area for applied empirical research.
Scholar Commons Citation
Macdonald, George T., "The Performance of the Linear Logistic Test Model When the Q-Matrix is Misspecified: A Simulation Study" (2013). USF Tampa Graduate Theses and Dissertations.
https://digitalcommons.usf.edu/etd/5065
