Towards Personalised Gaming via Facial Expression Recognition


  • Paris Mavromoustakos Blom University of Amsterdam
  • Sander Bakkes University of Amsterdam
  • Chek Tan University of Technology Sydney
  • Shimon Whiteson University of Amsterdam
  • Diederik Roijers University of Amsterdam
  • Roberto Valenti University of Amsterdam
  • Theo Gevers University of Amsterdam



Affective Computing, Facial Expression Analysis, Procedural Content Generation, Adaptive Content, Personalised Gaming


In this paper we propose an approach for personalising the space in which a game is played (i.e., levels) dependent on classifications of the user's facial expression  — to the end of tailoring the affective game experience to the individual user. Our approach is aimed at online game personalisation, i.e., the game experience is personalised during actual play of the game. A key insight of this paper is that game personalisation techniques can leverage novel computer vision-based techniques to unobtrusively infer player experiences automatically based on facial expression analysis. Specifically, to the end of tailoring the affective game experience to the individual user, in this paper we (1) leverage the proven InSight facial expression recognition SDK as a model of the user's affective state InSight, and (2) employ this model for guiding the online game personalisation process. User studies that validate the game personalisation approach in the actual video game Infinite Mario Bros. reveal that it provides an effective basis for converging to an appropriate affective state for the individual human player.




How to Cite

Mavromoustakos Blom, P., Bakkes, S., Tan, C., Whiteson, S., Roijers, D., Valenti, R., & Gevers, T. (2021). Towards Personalised Gaming via Facial Expression Recognition. Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, 10(1), 30-36.