In this paper we discuss the development of a dynamic agent-based model which simulates how agents search and explore in nonstationary environments and ultimately develop habitual, context-dependent, activity–travel patterns. Conceptually, the creation of a choice set is context dependent. Individuals are assumed to have aspiration levels associated with location attributes that, in combination with evaluation results, determine whether the agent will start exploring or persist in habitual behavior. An awareness level of each location determines whether or not it is included in the awareness set in the next time step. An activation level of each location determines whether or not it is qualified as a habitual choice, and an evaluation (utility) function allows individuals to evaluate each location given current beliefs. By implementing choices, agents may observe the differences between actual experience and expectation, which may give rise to negative or positive emotions that influence the awareness of locations and the evaluation, and hence trigger choice change. Principles of reinforcement and Bayesian belief learning are used to simulate the dynamics. The result of these behavior mechanisms is the evolution of choice sets and choice patterns, reflecting emergent behavior in relation to nonstationary environments. We report the results of a case study, implemented in an agent-based microsimulation system, of dynamic decision making of avoiding higher uncertainty in location choice, distinguishing habitual, exploitation, and exploration modes of choice behavior. Simulations indicate that solutions generated by the model are sensitive to rational and emotional considerations in decision making in well-interpretable ways. The suggested approach is scalable in the sense that it is applicable to study areas of large size (eg, region wide). <br> <b>Keywords:</b> habit formation, emotional value, location choice, learning
