The measurement and comparison of health system responsiveness

Measuring the performance of health systems has become a key tool in aiding decision makers to describe, analyze, compare and ultimately improve the delivery and outcomes achieved by a system. The World Health Organization’s (WHO) framework for assessing performance includes three intrinsic goals of health systems, namely health improvement, fairness in financial contribution and responsiveness to user preferences. Broadly speaking health system responsiveness can be defined as the way in which individuals are treated and the environment in which they are treated, encompassing the notion of patient experience. Perhaps the most ambitious attempt to implement a cross-country comparative instrument aimed at measuring health system performance is the World Health Survey (WHS). The modules on responsiveness and health ask respondents to rate their experiences using a 5-point categorical scale (e.g. very good to very bad). A common problem with self-reported data is that individuals, when faced with the instrument, are likely to interpret the meaning of the response categories in a way that systematically differs across populations or population sub-groups. In such cases the ordinal response categories will not be cross-population comparable since they will not imply the same underlying level of the construct. Recently the method of anchoring vignettes has been promoted as a means for controlling for systematic differences across socioeconomic groups in preferences, expectations and norms when responding to survey questions. This paper applies the method of anchoring vignettes to adjust survey reports of responsiveness for reporting heterogeneity. We present preliminary results for a selected number of domains and countries to illustrate the approach and find systematic reporting by income and education, but not by age and gender. Further analysis will extend the method to a larger set of countries and domains to investigate more fully the application of the approach for international comparative analysis of health system performance.