Ranking Individuals Versus Groups

Ranking groups (schools, regions, counties) according to the average score of their constituent parts - say, ranking schools by the academic achievements of students - is a common yardstick in evaluation and a cornerstone of any planning process. In this paper we show that under certain circumstances the ranking of groups, unlike the ranking of individuals, can be affected by the examiner. The fact that a ranking reversal is possible does not necessarily imply that the test itself is deficient, it merely reveals that the groups are non-homogeneous and therefore, the ranking of groups is meaningless. An investigation of the conditions under which such manipulation can occur leads us to suggest new statistical indicators as warning signals that can help one recognize such situations if and when they arise. It turns out that these indicators are related to the indicators for finding whether one distribution dominates another