The problem of choosing an optimal toolkit day after day, when the distribution of values of different toolkits is uncertain and can only be observed by carrying different toolkits, is a multi-armed bandit problem with non-independent arms. Accordingly, except for very simple specifications,...

This paper continues our study of heuristics employed to choose dynamically tools to put in a toolkit, where the value of any tool can be discovered only by choosing it. This is a multi-armed bandit problem with “arms” that are not independent, hence it is a problem for which the optimal...