Comparison of methods in the analysis of dependent ordered catagorical data

Rating scales for outcome variables produce categorical data which are often ordered and measurements from rating scales are not standardized. The purpose of this study is to apply commonly used and novel methods for paired ordered categorical data to two data sets with different properties and to compare the results and the conditions for use of these models. The two applications consist of a data set of inter-rater reliability and a data set from a follow-up evaluation of patients. Standard measures of agreement and measures of association are used. Various loglinear models for paired categorical data using properties of quasi-independence and quasi-symmetry as well as logit models with a marginal modelling approach are used. A nonparametric method for ranking and analyzing paired ordered categorical data is also used. We show that a deeper insight when it comes to disagreement and change patterns may be reached using the nonparametric method and illustrate some problems with standard measures as well as parametric loglinear and logit models. In addition, the merits of the nonparametric method are illustrated.