The Imprecise Science of Evaluating Scholarly Performance

In a growing number of countries, government-appointed assessment panels develop ranks on the basis of the quality of scholarly outputs to apportion budgets in recognition of evaluated performance and to justify public funds for future R&D activities. When business and management journals are being grouped in broad quality categories, a recent study has noted that this procedure was placing the same journals in essentially the same categories. Drawing on journal quality categorizations by several German- and English-speaking business departments and academic associations, the author performs nonparametric tests and correlations to analyze whether this claim can be substantiated. In particular, he examines the ability of broad quality categorizations to add value to governmental, administrative, and academic decision making by withstanding the criticism traditionally levied at research quality assessments.