UNIGE document Scientific Article
previous document  unige:88345  next document
add to browser collection

Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection

Asghari-Esfeden, Sadjad
Fu, Yun
Pantic, Maja
Published in IEEE transactions on affective computing. 2016, vol. 7, no. 1, p. 17-28
Abstract Emotions are time varying affective phenomena that are elicited as a result of stimuli. Videos and movies in particular are made to elicit emotions in their audiences. Detecting the viewers' emotions instantaneously can be used to find the emotional traces of videos. In this paper, we present our approach in instantaneously detecting the emotions of video viewers' emotions from electroencephalogram (EEG) signals and facial expressions. A set of emotion inducing videos were shown to participants while their facial expressions and physiological responses were recorded. The expressed valence (negative to positive emotions) in the videos of participants' faces were annotated by five annotators. The stimuli videos were also continuously annotated on valence and arousal dimensions. Long-short-term-memory recurrent neural networks (LSTM-RNN) and continuous conditional random fields (CCRF) were utilized in detecting emotions automatically and continuously. We found the results from facial expressions to be superior to the results from EEG signals. We analyzed the effect of the contamination of facial muscle activities on EEG signals and found that most of the emotionally valuable content in EEG features are as a result of this contamination. However, our statistical analysis showed that EEG signals still carry complementary information in presence of facial expressions.
Keywords VideosElectroencephalographyFeature extractionMotion picturesDatabasesTaggingRecurrent neural networks
Full text
Article (Published version) (1.5 MB) - document accessible for UNIGE members only Limited access to UNIGE
Research group Affective sciences
(ISO format)
SOLEYMANI, Mohammad et al. Analysis of EEG Signals and Facial Expressions for Continuous Emotion Detection. In: IEEE transactions on affective computing, 2016, vol. 7, n° 1, p. 17-28. doi: 10.1109/TAFFC.2015.2436926 https://archive-ouverte.unige.ch/unige:88345

506 hits

1 download


Deposited on : 2016-10-19

Export document
Format :
Citation style :