As a spoken word unfolds over time, it is temporarily consistent with the acoustic forms of multiple words. Previous behavioral research has shown that, in the face of temporary ambiguity about how a word will end, multiple candidate words are briefly activated. Here, we provide neural imaging evidence that lexical candidates only temporarily consistent with the input activate perceptually based semantic representations. An artificial lexicon and novel visual environment were used to target human MT/V5 and an area anterior to it which have been shown to be recruited during the reading of motion words. Participants learned words that referred to novel objects and to motion or color/texture changes that the objects underwent. The lexical items corresponding to the change events were organized into phonologically similar pairs differing only in the final syllable. Upon hearing spoken scene descriptions in a posttraining verification task, participants showed greater activation in the left hemisphere anterior extent of MT/V5 when motion words were heard than when nonmotion words were heard. Importantly, when a nonmotion word was heard, the level of activation in the anterior extent of MT/V5 was modulated by whether there was a phonologically related competitor that was a motion word rather than another nonmotion word. These results provide evidence of activation of a perceptual brain region in response to the semantics of a word while lexical competition is in process and before the word is fully recognized.