Morphing Consumer Dynamics : Bandits Meet HMM

Websites that match the information needs of visitors are efficient, but information needs may change as visitors browse a website. When consumers visit online stores they are exposed to information that may help them move along the purchase funnel, or make them bounce and leave the website. The position in the funnel is not directly observed, so learning from the outcome of a visit is a major challenge for multi-armed bandit models used in website design and online advertising. We develop a novel dynamic program to explicitly model the trade-off that the firm faces between nudging a visitor to later states along the funnel, and maximizing immediate expected reward given the current-state purchase probabilities. We use a hidden Markov model to dynamically assess the consumers stages based on clickstream, and couple it with a website-morphing multi-armed bandit model to learn the effectiveness of each design-stage combination using a dynamic allocation index. We provide a proof-of-concept based on adapting the full-time MBA website of a major European university. In this application, a team designed morphs based on categorization and construal-level theories, which provides us with substantive support for the choice of morph content and language tailored for early and late stages of the funnel. We find that matching concrete and abstract morphs to dynamic states using our algorithm outperforms current methods based on non-dynamic policies and HMM policies that are agnostic to the trade-off between bouncing and nudging