A population of agents plays a stochastic dynamic game wherein there is an underlying state process with a Markovian dynamics that also affects their costs. A learning mechanism is proposed which takes into account intertemporal effects and incorporates an explicit process of expectation...