This paper introduces a rubric for assessing QR in student papers and analyzes the inter-rater reliability of the instrument based on a reading session involving 11 participants. Despite the disciplinary diversity of the group (which included a faculty member from the arts and literature, two staff members, and representatives from five natural and social science departments), the rubric produced reliable measures of QR use and proficiency in a sample of student papers. Readers agreed on the relevance and extent of QR in 75.0 and 81.9 percent of cases respectively (corresponding to Cohen’s κ= 0.611 and 0.693). A four-category measure of quality produced slightly less agreement (66.7 percent, κ = 0.532). Collapsing the index into a 3-point scale raises the inter-rater agreement to 77.8 percent (κ = 0.653). The substantial agreement attained by this rubric suggests that it is possible to construct a reliable instrument for the assessment of QR in student arguments.
Grawe, Nathan D., Neil S. Lutsky, and Christopher J. Tassava. "A Rubric for Assessing Quantitative Reasoning in Written Arguments." Numeracy 3, Iss. 1 (2010): Article 3. DOI: http://dx.doi.org/10.5038/1936-46184.108.40.206
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License