We study the Markov perfect equilibria (MPEs) of an infinite horizon game in which pairs of players connected in a network are randomly matched to bargain. Players who reach agreement are removed from the network without replacement. We establish the existence of MPEs and show that MPE payoffs...