Measuring What Matters : Principles for a Balanced Data Suite That Prioritizes Problem-Solving and Learning

Responding effectively and with professional integrity to the many challenges of public administration requires recognizing that access to more and better quantitative data is necessary but insufficient. Overreliance on quantitative data comes with its own risks, of which public sector managers should be keenly aware. This paper focuses on four such risks. The first is that attaining easy-to-measure targets becomes a false standard of broader success. The second is that measurement becomes conflated with what management is and does. The third is that measurement inhibits a deeper understanding of the key policy problems and their constituent parts. The fourth is that political pressure to manipulate key indicators can lead, if undetected, to falsification and unwarranted claims or, if exposed, to jeopardizing the perceived integrity of many related (and otherwise worthy) measurement efforts. Left unattended, the cumulative concern is that these risks will inhibit rather than promote the core problem-solving and implementation capabilities of public sector organizations, an issue of high importance everywhere but especially in developing countries. The paper offers four cross-cutting principles for building an approach to the use of quantitative data - a “balanced data suite”—that strengthens problem-solving and learning in public administration: (1) identify and manage the organizational capacity and power relations that shape data management; (2) focus quantitative measures of success on those aspects which are close to the problem; (3) embrace a role for qualitative data, especially for those aspects that require in-depth, context-specific knowledge; and (4) protect space for judgment, discretion, and deliberation in those (many) decision-making domains that inherently cannot be quantified