When proportion consensus scoring works

Kimberly A. Barchard

doi:10.1016/j.paid.2013.01.017

Return

When proportion consensus scoring works

2013-07-01

PRE

Kimberly A. Barchard *

Emily D. Anderson

DOI：10.1016/j.paid.2013.01.017

Original

Original request for help

Save

Abstract

En 中文

Most objectively scored tests use items with easily identifiable correct answers. When such veridical scoring keys cannot be constructed, researchers sometimes use proportion consensus scoring (PCS) to identify the best answers. To determine if PCS identifies the best answers, we scored a test using both PCS and veridical scoring. Among 353 undergraduates, regular PCS, two-stage PCS, and expert PCS all had high correlations for easy items, but no PCS methods had high correlations for difficult items. Thus, PCS cannot reliability identify the best answers to individual items. However. PCS worked well for total scores. For easy items, total scores had correlations above .99 for all PCS methods. For difficult items, expert and two-stage PCS had correlations of .92 and .82 for the 60-item test. Thus, expert and two-stage PCS can be justified (even for difficult items) if the scoring key is based upon people who truly possess some degree of expertise and if scores are summed over many items. (C) 2013 Elsevier Ltd. All rights reserved.

Keywords:

Proportion consensus scoring

Scoring key

Item difficulty

Emotional intelligence

Leadership

AI Summary

Key information extracted from the uploaded paper, including a brief overview, abstract, background, key highlights, visual analysis, and future outlook.