Assessing Reliability and Validity of Peer and Self-Graded Assessments
Faculty Mentor(s)
Dr. Ruth Reynolds
Project Type
EdD Colloquium - ONU
Scholarship Domain(s)
Scholarship of Discovery
Presentation Type
Presentation
Abstract
The purpose of this study was to determine the usefulness of peer and self-grades as proxies for the teacher’s grades by evaluating the reliability and validity of evidence-based grading practices in a high school setting. The results of this study were intended to reveal grading strategies, if any, that were useful for teachers in order to bridge the gap between theoretical models of learning and explicit pedagogical techniques. This quantitative correlational study involved one participating teacher from a public suburban high school who administered a written assessment to 77 student participants in an Advanced Placement science course. The students self-graded and peer-graded the assessments using a publicly-released grading rubric from College Board. The researcher employed descriptive and inferential statistics to probe the relationships between peer-grades, self-grades, and teacher grades. The teacher’s grades were highly, positively, and statistically significantly correlated with averaged peer-grades (r(75) = 0.75, p < .001) and self-grades (r(74 )= .66, p < .001). Peer-graders demonstrated good inter-rater reliability (Cronbach’s α = 0.85). Peer-grading accuracy did not statistically significantly improve as students continued to peer-grade. No statistically significant difference in self-grading accuracy was detected when students peer-graded before self-grading. Peer-grades were more acceptable measures of student performance compared with self-grades and can be used as substitutes for teachers’ grades in the limited methodological and demographical context of this study. A larger and more diverse sample of student and teacher participants is recommended for future studies.
Cohort XX
Permission Type
This work is licensed under a Creative Commons Attribution 4.0 License.
Assessing Reliability and Validity of Peer and Self-Graded Assessments
Wisner Auditorium
The purpose of this study was to determine the usefulness of peer and self-grades as proxies for the teacher’s grades by evaluating the reliability and validity of evidence-based grading practices in a high school setting. The results of this study were intended to reveal grading strategies, if any, that were useful for teachers in order to bridge the gap between theoretical models of learning and explicit pedagogical techniques. This quantitative correlational study involved one participating teacher from a public suburban high school who administered a written assessment to 77 student participants in an Advanced Placement science course. The students self-graded and peer-graded the assessments using a publicly-released grading rubric from College Board. The researcher employed descriptive and inferential statistics to probe the relationships between peer-grades, self-grades, and teacher grades. The teacher’s grades were highly, positively, and statistically significantly correlated with averaged peer-grades (r(75) = 0.75, p < .001) and self-grades (r(74 )= .66, p < .001). Peer-graders demonstrated good inter-rater reliability (Cronbach’s α = 0.85). Peer-grading accuracy did not statistically significantly improve as students continued to peer-grade. No statistically significant difference in self-grading accuracy was detected when students peer-graded before self-grading. Peer-grades were more acceptable measures of student performance compared with self-grades and can be used as substitutes for teachers’ grades in the limited methodological and demographical context of this study. A larger and more diverse sample of student and teacher participants is recommended for future studies.
Cohort XX