en
Scientific article
English

A Multimodal Database for Affect Recognition and Implicit Tagging

Published inIEEE transactions on affective computing, vol. 3, no. 1, p. 42-55
Publication date2012
Abstract

MAHNOB-HCI is a multimodal database recorded in response to affective stimuli with the goal of emotion recognition and implicit tagging research. A multimodal setup was arranged for synchronized recording of face videos, audio signals, eye gaze data, and peripheral/central nervous system physiological signals. Twenty-seven participants from both genders and different cultural backgrounds participated in two experiments. In the first experiment, they watched 20 emotional videos and self-reported their felt emotions using arousal, valence, dominance, and predictability as well as emotional keywords. In the second experiment, short videos and images were shown once without any tag and then with correct or incorrect tags. Agreement or disagreement with the displayed tags was assessed by the participants. The recorded videos and bodily responses were segmented and stored in a database. The database is made available to the academic community via a web-based system. The collected data were analyzed and single modality and modality fusion results for both emotion recognition and implicit tagging experiments are reported. These results show the potential uses of the recorded modalities and the significance of the emotion elicitation protocol.

Keywords
  • Emotion recognition
  • Visual databases
  • MAHNOB-HCI
  • Web-based system
  • Affect recognition
  • Affective stimuli
  • Arousal
  • Audio signal recording
  • Dominance
  • Emotion elicitation protocol
  • Emotional keywords
  • Eye gaze data recording
  • Face video recording
  • Implicit tagging
  • Multimodal database
  • Peripheral-central nervous system physiological signals
  • Predictability
  • Valence
  • Cameras
  • Databases
  • Humans
  • Physiology
  • Tagging
  • Videos
  • EEG
  • Affective computing.
  • Eye gaze
  • Facial expressions
  • Pattern classification
  • Physiological signals
Citation (ISO format)
SOLEYMANI, Mohammad et al. A Multimodal Database for Affect Recognition and Implicit Tagging. In: IEEE transactions on affective computing, 2012, vol. 3, n° 1, p. 42–55. doi: 10.1109/T-AFFC.2011.25
Main files (1)
Article (Published version)
accessLevelRestricted
Identifiers
ISSN of the journal1949-3045
564views
1downloads

Technical informations

Creation03/03/2015 4:36:03 PM
First validation03/03/2015 4:36:03 PM
Update time03/14/2023 10:56:36 PM
Status update03/14/2023 10:56:36 PM
Last indexation01/16/2024 5:01:57 PM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack