The Emo-FilM dataset stands for Emotion research using Films and fMRI in healthy participants. This dataset includes emotion annotations by 44 raters for 14 short films with a combined duration of over 2½ hours and recordings of respiration, heart rate, and functional magnetic resonance imaging (fMRI) from a sample of 30 individuals watching the same films. 50 items were annotated including discrete emotions and emotion components from the domains of appraisal, motivation, motor expression, physiological response, and feeling. The ratings had a mean inter-rater agreement of 0.38. The fMRI data acquired at 3 Tesla is includes high-resolution structural and resting state fMRI for each participant. Physiological recordings included heart rate, respiration, and electrodermal activity. This dataset is designed, but not limited, to studying the dynamic neural processes involved in emotion experience. It has a high temporal resolution of annotations, and includes validations of annotations by the fMRI sample. The Emo-FilM dataset is a treasure trove for researching emotion in response to naturalistic stimulation in a multimodal framework.