An influential model of spatial attention postulates three main attention-orienting mechanisms: disengagement, shifting, and engagement. Early research linked disengagement deficits with superior parietal damage, regardless of hemisphere or presence of spatial neglect. Subsequent studies supported the involvement of more ventral parietal regions, especially in the right hemisphere, and linked spatial neglect to deficient disengagement from ipsilateral cues. However, previous lesion studies faced serious limitations, such as small sample sizes and the lack of brain-injured controls without neglect. Additionally, some studies employed symbolic cues or used long cue-target intervals, which may fail to reveal impaired disengagement. We here used a machine-learning approach to conduct lesion-symptom mapping (LSM) on 89 patients with focal cerebral lesions to the left (LH) or right (RH) cerebral hemisphere. A group of 54 healthy participants served as controls. The paradigm used to uncover disengagement deficits employed non-predictive cues presented in the visual periphery and at short cue-target intervals, targeting exogenous attention. The main factors of interest were group (healthy participants, LH, RH), target position (left, right hemifield) and cue validity (valid, invalid). LSM-analyses were performed on two indices: the validity effect, computed as the absolute difference between reaction times (RTs) following invalid compared to valid cues, and the disengagement deficit, determined by the difference between contralesional and ipsilesional validity effects. While LH patients showed general slowing of RTs to contralesional targets, only RH patients exhibited a disengagement deficit from ipsilesional cues. LSM associated the validity effect with a right lateral frontal cluster, which additionally affected subcortical white matter of the right arcuate fasciculus, the corticothalamic pathway, and the superior longitudinal fasciculus. In contrast, the disengagement deficit was related to damage involving the right temporoparietal junction. Thus, our results support the crucial role of right inferior parietal and posterior temporal regions for attentional disengagement, but also emphasize the importance of lateral frontal regions, for the reorienting of attention.