Previous studies using the dot-perspective task postulated that people automatically take into account others' perspective even when it prevents them from achieving their own goals. This human ability may be of key importance for the ascription of mental states and social interactions. The cognitive and emotional determinants of automatic perspective taking (APT) is still matter of debate. To address this issue, we examined the performance in the Samson et al. APT task in 91 healthy adults who underwent a detailed neuropsychological testing including assessment of their general intelligence (Wechsler Adult Intelligence Scale, WAIS), attention and impulsivity (Conners' Continuous Performance Test-II, CPT-II), alexithymia (Toronto Alexithymia Scale, TAS), and measures of affective empathy and explicit theory of mind (Geneva Social Cognition Scale, GeSoCS, and mini-Social cognition and Emotional Assessment, mini-SEA). Univariate and multiple linear regression models (adjusted for age, gender, and education) were used to explore the association between mean reaction times (respectively, mean number of errors) in the APT task, and the CPT-II parameters, WAIS global score (as well as subscale scores), TAS, and GeSoCS and mini-SEA scores. Only the CPT-II parameters were significantly associated with the mean reaction times. Increased omissions, commissions, and detectability as well as hit reaction time standard error in CPT-II were all related to worse performances both in Self and Other conditions. The mean number of errors was negatively associated with the GeSoCS score. Among the variables studied, only CPT-II parameters had a significant impact on egocentric and altercentric interference. Neither global intelligence nor alexithymia have an effect on dot-perspective task performance. The present findings suggest that people with lower attentional resources and increased impulsivity display worse performances in the APT task and are less responsive to both egocentric and altercentric interference.