When do experts listen to other experts? : the role of negative information in expert evaluations for novel projects

The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects resulting in 761 proposal-evaluation pairs and over $300,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores and measured how exposures to higher and lower scores affect the focal evaluator’s propensity to change the initial score. We found causal evidence of a negativity bias, where evaluators lower their scores by more points after seeing scores more critical than their own than raise them after seeing more favorable scores. Qualitative coding of the evaluators’ justifications for score changes reveal that exposures to lower scores were associated greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores with increased emphasis on non-evaluation criteria, such as confidence in one’s judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure than maximizing success