Rating Analyst Degrees of Freedom

Credit rating agencies are widely regarded as partially responsible for the collapse of financial markets in 2008. Congress set out to reform the ratings industry with Dodd-Frank but recent scholarship suggests that these reforms have failed and rating agencies continue to make objective errors in issuing ratings. This article seeks to explain why. This article provides a new way to understand the biased and inaccurate nature of credit ratings. The existing accounts focus on factors external to the credit ratings process, such as licensing or financial incentives. By contrast, this article suggests problems in the ratings industry are best understood as internal to the credit ratings process. To make this case, this article relies on the descriptive framework scholars have developed to explain the replication crisis in behavioral psychology. The framework suggests that mistakes in experimental research are the product of two factors, which can lead researchers to underestimate the existence of false positives: (1) researcher degrees of freedom (i.e., freedom in data selection and analysis) and (2) a researcher's implicit bias towards finding statistically significant results. Rating agencies use statistical models to rate debt and rating analysts face challenging design decisions in developing these models. In managing these decisions, this article contends that the factors that lead behavioral psychologists to underestimate the existence of false positives may lead rating analysts to build models that are biased towards predicting fewer defaults. Reframing problems in the ratings industry in this manner helps us understand why rating agencies continue to make mistakes and why the reforms in Dodd-Frank, as presently applied, fail to constrain rating agency error. This article concludes by suggesting two reforms within the current framework of Dodd-Frank that may result in less biased and more accurate credit ratings: liberalization of the NRSRO application process and making rating agencies compete to preserve their NRSRO status. This article explains how the SEC can effectuate these reforms and addresses some implementation problems with creating a market for regulatory status