How the processing of emotional expression is influenced by perceived gaze remains a debated issue. Discrepancies between previous results may stem from differences in the nature of stimuli and task characteristics. Here we used a highly controlled set of computer-generated animated faces combining dynamic emotional expressions with varying intensity, and gaze shifts either directed at or averted from the observer. We predicted that perceived self-relevance of fearful faces would be higher with averted gaze-signaling a nearby danger; whereas conversely, direct gaze would be more relevant for angry faces-signaling aggressiveness. This interaction pattern was observed behaviorally for emotion intensity ratings, and neurally for functional magnetic resonance imaging activation in amygdala, as well as fusiform and medial prefrontal cortices, but only for mild- and not high-intensity expressions. These results support an involvement of human amygdala in the appraisal of self-relevance and reveal a crucial role of expression intensity in emotion and gaze interactions.