Comparing the Similarities and Differences of PISA 2003 and TIMSS

This paper makes an in-depth comparison of the PISA (OECD) and TIMSS (IEA) mathematics assessments conducted in 2003. First, a comparison of survey methodologies is presented, followed by an examination of the mathematics frameworks in the two studies. The methodologies and the frameworks in the two studies form the basis for providing explanations for the observed differences in PISA and TIMSS results. At the country level, it appears that Western countries perform relatively better in PISA as compared to their performance in TIMSS. In contrast, Asian and Eastern European countries tend to do better in TIMSS than in PISA. This paper goes beyond making mere conjectures about the observed differences in results between PISA and TIMSS. The paper provides supporting evidence through the use of regression analyses to explain the differences. The analyses showed that performance differences at the country level can be attributed to the content balance of the two tests, as well as the sampling definitions – age-based and grade-based – in PISA and TIMSS respectively. Apart from mathematics achievement, the paper also compares results from the two studies on measures of self-confidence in mathematics. Gender differences are also examined in the light of contrasting results from the two studies. Overall, the paper provides a comprehensive comparison between PISA and TIMSS, and, in doing so, it throws some light on the interpretation of results of large-scale surveys more generally.