Eye gaze is a central cue for human social interaction, yet it remains debated whether its effects on perception and attention are automatic or shaped by context. Using EEG across three experiments, we examined how dynamic gaze influences early perceptual encoding (N170) and later attentional selection (N2pc). Participants viewed pairs of faces whose eyes shifted either toward them (gaze toward the observer) or further away (gaze away) while the other closed the eyes. The relevance of eye contact varied across tasks. Across Experiments 1 and 2, gaze toward consistently elicited larger N170 amplitudes than gaze away, independent of context, indicating robust early encoding of eye contact. In contrast, N2pc amplitudes were flexibly modulated: strongest when gaze was task-relevant, reduced when participants judged spatial direction. Cross-experiment analyses confirmed a gaze graded decrease in N2pc effects, while N170 remained stable. Finally, in Experiment 3, N170 and N2pc effects were absent when gaze was irrelevant. These results reveal a two-stage process in dynamic gaze perception: an early, context-independent perceptual encoding and a later, task-dependent attentional selection. The findings refine the Fast-Track Modulator Model by demonstrating that gaze processing is perceptually robust but attentionally flexible, indicating that gaze effects are conditional rather than automatic.