Animal studies have shown that sensory deprivation in one modality can have striking effects on the development of the remaining modalities. Although recent studies of deaf and blind humans have also provided convincing behavioural, electrophysiological and neuroimaging evidence of increased capabilities and altered organization of spared modalities, there is still much debate about the identity of the brain systems that are changed and the mechanisms that mediate these changes. Plastic changes across brain systems and related behaviours vary as a function of the timing and the nature of changes in experience. This specificity must be understood in the context of differences in the maturation rates and timing of the associated critical periods, differences in patterns of transiently existing connections, and differences in molecular factors across brain systems.