A current controversy surrounds the question whether high-level features of a stimulus such as its relevance to the current task may affect early attentional processes. According to one view abruptly appearing stimuli gain priority during an initial feedforward processing stage and therefore capture attention even if they are irrelevant to the task. Alternatively, only stimuli that share a relevant property with the target may capture attention of the observer. Here, we used high-density EEG to test whether task relevance may modulate early feedforward brain activity, or whether it only becomes effective once the physical characteristics of the stimulus have been processed. We manipulated task relevance and visual saliency of distracters presented left or right of an upcoming central target. We found that only the relevance of distracters had an effect on manual reaction times to the target. However, the analysis of electrocortical activity revealed three discrete processing stages during which pure effects of distracter saliency (~80-160ms), followed by an interaction between saliency and relevance (~130-240ms) and finally pure effects of relevance (~230-370ms) were observed. Electrical sources of early saliency effects and later relevance effects were localized in the posterior parietal cortex, predominantly over the right hemisphere. These findings support the view that during the initial feedforward stage only physical (bottom-up) factors determine cortical responses to visual stimuli, while top-down effects interfere at later processing stages.