Computational Understanding of the Interaction between Facial Expressions and Electroencephalogram (EEG) Signals for Emotion Recognition

Diplôme Thèse Computational Understanding of the Interaction between Facial Expressions and Electroencephalogram (EEG) Signals for Emotion Recognition eng 2021-02-09 2022-11-30 author Rayatdoost Soheil 0000-0002-8358-3349 852570 director Chopard Bastien 22320 director Rudrauf David 850236 134 10.13097/archive-ouverte/unige:165667 urn:nbn:ch:unige-1656672 Sc. 5701 <p>Machine recognition of emotions is an essential prerequisite for natural and intelligent interactions between humans and computers. Existing research has demonstrated the potential of electroencephalogram (EEG)-based emotion recognition. In addition to information from neural sources, EEG signals also contain a mixture of signals originating from sensorimotor, cognitive and affective activities in addition to artifacts. Facial expressions, eye and head movements generate electrical activities that have higher amplitude than the EEG signals and are the main source of artifacts. Even though artifacts are considered a nuisance to brain-computer interfaces, in affective brain-computer interfaces (aBCI), behavioral artifacts can be valuable for emotion recognition.</p> <p>Our research aims to identify the inter-modality interaction between EEG signals and facial behaviors to improve EEG-based emotion recognition performance. The interaction between these two modalities can be explained by signal interference and joint emotional variations. For the first time, we designed a specific protocol and collected data to isolate the effects of expressions, subjective feelings, and stimuli in these signals. We recorded a precisely synchronized multimodal database with visual, olfactory and mimicry stimuli, including genuine and acted expressions of emotions. This dataset enabled us to design and evaluate the emotion recognition methods in different scenarios and in the presence of between participant variances. We used our novel dataset in addition to the existing datasets (DEAP and MAHNOB) for within- and cross-corpus evaluation.</p> EEG signals Emotion recognition Cross-modality interaction Facial expression Behavioral activities Deep learning Information fusion Cross-corpus evaluation Domain adaptation Representation learning Dewey 128.37 Dewey 025.063 32 Département d'informatique 18 15 Centre interfacultaire en sciences affectives 0D8 false 715 Computer Vision and Multimedia Laboratory - CVML 94d84805-695f-4bf5-9a15-0881293cc4d1 Thesis Public Thesis.pdf 7329404 application/pdf 983deffe-9098-4a2f-92bd-6812bd418b5b Imprimatur Public imprimatur.pdf 271245 application/pdf Vignette Public SI-EEG_Network.png 86675 image/png openAccess Faculté des sciences / Département d'informatique Centres et instituts / Centre interfacultaire en sciences affectives

Title	Published in	Year	Views	Downloads

Computational Understanding of the Interaction between Facial Expressions and Electroencephalogram (EEG) Signals for Emotion Recognition		2022	509	607
Expression-Guided EEG Representation Learning for Emotion Recognition	ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)	2020	25	0

Archive ouverte UNIGE

Rayatdoost, Soheil

account_balance Affiliation entities

Affiliation entities