In this paper discounted and average Markov decision processes with finite state space and countable action set (semi-infinite MDP for short) are discussed. Without ordinary continuity and compactness conditions, for discounted semi-infinite MDP we have shown that by exploiting the results on...