Scientific article

Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection

Published inIEEE transactions on affective computing, vol. 7, no. 1, p. 17-28
Publication date2016

Emotions are time varying affective phenomena that are elicited as a result of stimuli. Videos and movies in particular are made to elicit emotions in their audiences. Detecting the viewers' emotions instantaneously can be used to find the emotional traces of videos. In this paper, we present our approach in instantaneously detecting the emotions of video viewers' emotions from electroencephalogram (EEG) signals and facial expressions. A set of emotion inducing videos were shown to participants while their facial expressions and physiological responses were recorded. The expressed valence (negative to positive emotions) in the videos of participants' faces were annotated by five annotators. The stimuli videos were also continuously annotated on valence and arousal dimensions. Long-short-term-memory recurrent neural networks (LSTM-RNN) and continuous conditional random fields (CCRF) were utilized in detecting emotions automatically and continuously. We found the results from facial expressions to be superior to the results from EEG signals. We analyzed the effect of the contamination of facial muscle activities on EEG signals and found that most of the emotionally valuable content in EEG features are as a result of this contamination. However, our statistical analysis showed that EEG signals still carry complementary information in presence of facial expressions.

  • Videos
  • Electroencephalography
  • Feature extraction
  • Motion pictures
  • Databases
  • Tagging
  • Recurrent neural networks
Research group
Citation (ISO format)
SOLEYMANI, Mohammad et al. Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection. In: IEEE transactions on affective computing, 2016, vol. 7, n° 1, p. 17–28. doi: 10.1109/TAFFC.2015.2436926
Main files (1)
Article (Published version)
ISSN of the journal1949-3045

Technical informations

Creation10/07/2016 4:43:00 PM
First validation10/07/2016 4:43:00 PM
Update time03/15/2023 12:50:58 AM
Status update03/15/2023 12:50:58 AM
Last indexation08/30/2023 12:06:06 PM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack