Facial expressions of emotions have been shown to modulate early ERP components, in particular the N170. The underlying anatomical structure producing these early effects are unclear. In this study, we examined the N170 enhancement for fearful expressions in healthy controls as well as epileptic patients after unilateral left or right amygdala resection. We observed a greater N170 for fearful faces in healthy participants as well as in individuals with left amygdala resections. By contrast, the effect was not observed in patients who had undergone surgery in which the right amygdala had been removed. This result demonstrates that the amygdala produces an early brain response to fearful faces. This early response relies specifically on the right amygdala and occurs at around 170 ms. It is likely that such increases are due to a heightened response of the extrastriate cortex that occurs through rapid amygdalofugal projections to the visual areas.