The Myth of the Double-Blind Review? Author Identification Using Only Citations

Prior studies have questioned the degree of anonymity of thedouble-blind review process for scholarly research articles. Forexample, one study based on a survey of reviewers concluded that authorsoften could be identified by reviewers using a combination of theauthor's reference list and the referee's personal background knowledge.For the KDD Cup 2003 competition's "Open Task" we examined howwell various automatic matching techniques could identify authors withinthe competition's very large archive of research papers. This paperdescribes the issues surrounding author identification, how these issuesmotivated our study, and the results we obtained. The best method, basedon discriminative self-citations, identified authors correctly 40-45% ofthe time. One main motivation for double-blind review is to eliminatebias in favor of well-known authors. However, identification accuracyfor authors with substantial publication history is even better (60%accuracy for the top-10% most prolific authors, 85% for authors with 100or more prior papers).