Androgen deprivation therapy (ADT) is a standard treatment for prostate cancer patients, routinely used in the palliative or in the curative setting in association with radiotherapy. Among the systemic long-term side effects of ADT, growing data suggest a potentially increased risk of dementia/Alzheimer's disease in prostate cancer patients treated with hormonal manipulation. While pre-clinical data suggest that androgen ablation may have neurotoxic effects due to Aβ accumulation and increased tau phosphorylation in small animal brains, clinical studies have measured the impact of ADT on long-term cognitive function, with conflicting results, and studies on biological changes after ADT are still lacking. The aim of this review is to report on the current evidence on the association between the ADT use and the risk of cognitive impairment in prostate cancer patients. We will focus on the contribution of Alzheimer's disease biomarkers, namely through imaging, to investigate potential ADT-induced brain modifications. The evidence from these preliminary studies shows brain changes in gray matter volume, cortical activation and metabolism associated with ADT, however with a large variability in biomarker selection, ADT duration and cognitive outcome. Importantly, no study investigated yet biomarkers of Alzheimer's disease pathology, namely amyloid and tau. These preliminary data emphasize the need for larger targeted investigations.