Predicting Strategic Decisions Based on Emotional Signals

Predicting human strategic decisions is fundamental to the design of intelligent human-interacting systems. While the connection between emotion and strategic action has been well-established in the past, in this work, we introduce the following question: Can emotional signals, automatically captured and interpreted from short video and audio recordings of a user, serve as good predictors of strategic action selection in an economic context, modeled as a game? In order to initiate the research on this question, we perform a user study where emotional signals are elicited from users using short video clips of emotional content. These signals are automatically analyzed by standard off-the-shelf computational means and are in turn used as potential predictors of strategic decisions in two classic, well-studied one-shot economical games: the ultimatum game and the trust game. By employing supervised machine learning techniques, we demonstrate the potential predictive power of these emotional signals and show that relying on the interpreted emotions can bring about more accurate predictions than standard baseline approaches.