Robust Data-Driven Guarantees in Auctions

Analysis of welfare in auctions comes traditionally via one of two approaches: precise but fragile inference of the exact details of a setting from data or robust but coarse theoretical price of anarchy bounds that hold in any setting. As markets get more and more dynamic and bidders become more and more sophisticated, the weaknesses of each approach are magnified.In this paper, we provide tools for analyzing and estimating the empirical price of anarchy of an auction. The empirical price of anarchy is the worst case efficiency loss of any auction that could have produced the data, relative to the optimal. Our techniques are based on inferring simple properties of auctions: primarily the expected revenue and the expected payments and allocation probabilities from possible bids. These quantities alone allow us to empirically estimate the revenue covering parameter of an auction which allows us to re-purpose the theoretical machinery of Hartline et al. [2014] for empirical purposes. Moreover, we show that under general conditions the revenue covering parameter estimated from the data approaches the true parameter with the error decreasing at the rate proportional to the square root of the number of auctions and at most polynomially in the number of agents. While we focus on the setting of position auctions, and particularly the generalized second price auction, our techniques are applicable far more generally. Finally, we apply our techniques to a selection of advertising auctions on Microsoft's Bing and find empirical results that are a significant improvement over the theoretical worst-case bounds