Emotion AI Meets Strategic Users

When organizations adopt artificial intelligence (AI) to recognize individuals’ negative emotions and accordingly match limited resources to them, strategic users are incentivized to game the system by escalating emotional intensities. The economic value of AI may be undermined by gaming behavior, algorithmic noise in emotion detection, and the spillover effect of negative emotions. We develop a game-theoretical model to identify the conditions under which adopting emotion AI is valuable to the organization and design the associated optimal allocation policies. We find that adopting emotion AI is always valuable if the spillover effect of negative emotions is negligible compared to resource misallocation loss, regardless of algorithmic noise and gaming behavior. We also quantify the welfare impacts of emotion AI on individuals, the organization, and society. Notably, we show that a stronger AI is not always socially desirable and regulation on data-driven allocation is needed. Finally, we characterize a broad set of conditions under which adopting emotion AI is more profitable for the organization than hiring employees to recognize emotions and allocate resources. Our work provides implications for designing, adopting, and regulating emotion AI