Introduction
Introduction Statistics Contact Development Disclaimer Help
ERIC ED422397: A Comparison of Two Scoring Strategies for Performan...
by ERIC
Thumbnail
Download
Web page
The aim of this study was to explore a method of
improving the objectivity, reliability, and efficiency of
scoring performance assessments that involve constructed
written responses. Millman (1997) has suggested an
alternative to using model responses at each score
category. The proposed strategy, hypothesized to increase
scorer reliability and cost effectiveness, would model
answers judged to be halfway between the score
categories. This paper reports on a small study designed
to compare a scoring method using model responses at each
category to a variation of Millman's suggested
alternative. Existing student responses to a fifth grade
reading prompt from a large school district's assessment
program were used. Twenty volunteers (graduate students)
served as raters, and 200 responses to the same prompt
were divided into 5 groups of 40 responses. Two raters
from each scoring group scored the same 40 papers,
allowing the comparison of 2 scores for each response
under each scoring condition. No differences were
detected between the scoring methods. This may be due to
the difficulty of obtaining agreement on borderline
responses to be used in training, or it may represent the
absence of a consensus on borderline anchor papers. In
conclusion, it is stated that no evidence is found to
differentiate levels of rater agreement between using
judgments of dominance and judgments of proximity.
Appendixes present two study scoring rubrics. (Contains
one table and nine references.) (SLD)
Date Published: 2015-12-26 21:25:51
Identifier: ERIC_ED422397
Item Size: 9916253
Language: english
Media Type: texts
# Topics
ERIC Archive; Comparative Analysis; C...
# Collections
ericarchive
additional_collections
# Uploaded by
@chris85
# Similar Items
View similar items
PHAROS
You are viewing proxied material from tilde.pink. The copyright of proxied material belongs to its original authors. Any comments or complaints in relation to proxied material should be directed to the original authors of the content concerned. Please see the disclaimer for more details.