Affective characterization of movie scenes based on content analysis and physiological changes

Soleymani, Mohammad; Chanel, Guillaume; Kierkels, Joep Johannes Maria; Pun, Thierry

doi:10.1142/S1793351X09000744

Advanced search

Browse by...

Deposit

Highlights

More informations

Title		Affective characterization of movie scenes based on content analysis and physiological changes
Authors		Soleymani, Mohammad Chanel, Guillaume Kierkels, Joep Johannes Maria Pun, Thierry
Published in		International Journal of Semantic Computing. 2009, vol. 3, no. 2, p. 235-254
Abstract		In this paper, we propose an approach for affective characterization of movie scenes based on the emotions that are actually felt by spectators. Such a representation can be used to characterize the emotional content of video clips in application areas such as affective video indexing and retrieval, and neuromarketing studies. A dataset of 64 different scenes from eight movies was shown to eight participants. While watching these scenes, their physiological responses were recorded. The participants were asked to self-assess their felt emotional arousal and valence for each scene. In addition, content-based audio- and video-based features were extracted from the movie scenes in order to characterize each scene. Degrees of arousal and valence were estimated by a linear combination of features from physiological signals, as well as by a linear combination of content-based features. We showed that a significant correlation exists between valence-arousal provided by the spectator's self-assessments, and affective grades obtained automatically from either physiological responses or from audio-video features. By means of an analysis of variance (ANOVA), the variation of different participants' self assessments and different gender groups self assessments for both valence and arousal were shown to be significant (p-values lower than 0.005). These affective characterization results demonstrate the ability of using multimedia features and physiological responses to predict the expected affect of the user in response to the emotional video content.
Keywords		Multimedia indexing and retrieval — Affective personalization and characterization — Emotion recognition and assessment — Affective computing — Physiological signal analysis
Identifiers		DOI: 10.1142/S1793351X09000744
Full text		Article (Published version) (365 Kb) - Limited access to UNIGE Article (Author postprint) (431 Kb) - Free access
Structures		Faculté des sciences / Département d'informatique Centres et instituts / Centre universitaire d'informatique Centres et instituts / Centre interfacultaire des sciences affectives
Research groups		Affective sciences Computer Vision and Multimedia Laboratory Multimodal Interaction Group
Citation (ISO format)		SOLEYMANI, Mohammad et al. Affective characterization of movie scenes based on content analysis and physiological changes. In: International Journal of Semantic Computing, 2009, vol. 3, n° 2, p. 235-254. doi: 10.1142/S1793351X09000744 https://archive-ouverte.unige.ch/unige:47414