Music is often described in terms of emotion. This notion is supported by empirical evidence showing that engaging with music is associated with subjective feelings, and with objectively measurable responses at the behavioural, physiological, and neural level. Some accounts, however, reject the idea that music may directly induce emotions. For example, the 'paradox of negative emotion', whereby music described in negative terms is experienced as enjoyable, suggests that music might move the listener through indirect mechanisms in which the emotional experience elicited by music does not always coincide with the emotional label attributed to it. Here we discuss the role of metaphor as a potential mediator in these mechanisms. Drawing on musicological, philosophical, and neuroscientific literature, we suggest that metaphor acts at key stages along and between physical, biological, cognitive, and contextual processes, and propose a model of music experience in which metaphor mediates between language, emotion, and aesthetic response.