A Bayesian Hierarchical Approach to Multirater Correlated ROC Analysis

In a common ROC study design, several readers are asked to rate diagnostics of the same cases processed under different modalities. We describe a Bayesian hierarchical model that facilitates the analysis of this study design by explicitly modeling the three sources of variation inherent to it. In so doing, we achieve substantial reductions in the posterior uncertainty associated with estimates of the differences in areas under the estimated ROC curves and corresponding reductions in the mean squared error (MSE) of these estimates. Based on simulation studies, both the widths of confidence intervals and MSE of estimates of differences in the area under the curves appear to be reduced by a factor that often exceeds two. Thus, our methodology has important implications for increasing the power of analyses based on ROC data collected from an available study population.