Proceedings chapter
Open access

Your Body Reveals Your Impressions about Others: A Study on Multimodal Impression Detection

Presented at Cambridge, UK
PublisherCambridge : IEEE
Publication date2019

Formed impressions are crucial for human-human interaction (e.g. a job interview) and an interaction with a virtual agent/robot, since they can impact people's perceptions and willingness to be involved in the interaction. There are studies on how facial features (e.g. skin color, face shape), acoustic signals and non-verbal behaviors (e.g. gestures, postures) create/leave certain impressions. However there is little research focusing on how our bodies disclose our already formed impression of someone. Forming an impression leads to emotions and behaviors which can be measured. In this paper, we investigate recognition of evoked impression of warmth and competence from the non- verbal behaviors expressed by the person forming the impression. We conducted an experiment in which participants were watching impression stimuli. We measured participant's facial expressions, eye movements and physiological reactions (electrocardiography and galvanic skin response). To recognize impressions, we tested 2 multivariate regression models with the aforementioned mul- timodal recordings. Our best results demonstrate the possibility to detect impressions along warmth and competence dimensions with a concordance correlation coefficient of 0.838 and 0.864. Facial expressions and eye movements are more reliable for impression detection compared with physiological signals. Finally, the higher the Berkeley emotion expressivity scores the partic- ipants have, the more accurately the impressions are detected.

  • Impression detection
  • Machine learning
  • Multi- modal
  • LSTM
  • Swiss National Science Foundation - 2000221E-164326
Citation (ISO format)
WANG, Chen, PUN, Thierry, CHANEL, Guillaume. Your Body Reveals Your Impressions about Others: A Study on Multimodal Impression Detection. In: 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW). Cambridge, UK. Cambridge : IEEE, 2019. doi: 10.1109/ACIIW.2019.8925052
Main files (1)
Proceedings chapter (Accepted version)

Technical informations

Creation10/03/2019 5:38:00 PM
First validation10/03/2019 5:38:00 PM
Update time03/15/2023 6:07:21 PM
Status update03/15/2023 6:07:21 PM
Last indexation01/17/2024 6:25:36 AM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack