The Role of Calibration Committees in Subjective Performance Evaluation Systems

We provide the first empirical evidence of the role that calibration committees play in subjective performance evaluation systems. Using proprietary data from a large multinational organization, we begin by showing that calibration committees adjust ratings sparingly (i.e., 25 percent adjustment rate), but when they do, downward adjustments are significantly more frequent and of greater magnitude than upward adjustments. Calibration committees tend to downward (upward) adjust ratings of supervisors who give higher (lower) than average initial ratings. Taken together, calibration committees improve the consistency of ratings across supervisors and mitigate leniency bias, but exacerbate centrality bias. We also show that calibration committees facilitate the appropriate allocation of decision rights by deferring rating decisions to supervisors who possess a relatively greater information advantage. That is, calibration committees are less likely to adjust the rating of a subordinate who is further removed from committee members in the organizational hierarchy. Finally, we show that calibration committees promote supervisor learning about organizational performance rating expectations through calibration adjustments. This study contributes to the literature on performance evaluation by providing new insights regarding the organizational dynamics of subjective performance evaluation systems when decision rights span hierarchical levels of the organization