Semantics and Scale: The Impact of Anchor Polarity on Performance Rating Accuracy

Abstract

The present study examined the impact of rating scale anchor labels and their polarity on rater accuracy in the context of performance measurement. Participants (n = 128) viewed a video depicting three individuals (ratees) working together as a team on a decision-making and problem solving activity. Participants evaluated each ratee in each of the following performance dimensions: decision making, verbal communication, and collaboration. The performance-rating instrument used was a five-point anchored scale and participants were randomly assigned to one of three conditions in which the scale anchor labels were different: a unipolar scale condition, a bipolar scale condition, and a no anchor label scale condition. Ratings provided by participants were compared to ratings provided by trained expert raters, and analysis of variance enabled examination of participant accuracy. Findings demonstrated significantly higher ratings in the bipolar scale condition in several ratee/ performance dimension combinations, thus partially supporting the hypotheses. The implication of these findings for the practitioner community involved in workforce/employee performance measurement is substantial, myriad organizations may be using ambiguous or subjective rating scales that yield inaccurate employee performance data, affecting the extent to which organizational stakeholders have valid and reliable information to sustain overall business performance.

Keywords: Performance, Accuracy, Rating, Scale, Anchor, Polarity

View PDF