This paper presents a novel framework for studying partially observable Markov decision processes (POMDPs) with finite state, action, observation sets and discounted rewards. The new framework is solely based on future-reward vectors associated with future policies, which is more parsimonious...

A new methodology is presented to solve an important model of dynamic decision-making with a continuous unknown parameter (or state). The methodology centers on the concepts of “continuation-value function” (which gives the expected value-to-go as a function of the parameter under a feasible...

Problems concerning dynamic learning and decision-making are difficult to solve, especially analytically. We study an infinite-horizon discrete-time model with a constant unknown state that has two possible values and strive to derive an analytical solution. As a special partially observable...