To examine the relationship of serum 25-hydroxy vitamin D3 with cognitive functioning in higher age, using an instrument covering multiple cognitive domains in a population-based study. Follow-up study with measurement of vitamin D levels at baseline and assessment of cognitive functioning at year 5 follow-up. A subgroup of 1639 participants of the ongoing epidemiological ESTHER study of the elderly general population in Saarland State, Germany, aged 65+ years at baseline (2000–2002). Observational study. Cognitive functioning was assessed by the COGTEL phone interview developed by Kliegel et al., which was administered 5years after ESTHER baseline. Vitamin D in baseline samples was measured by chemiluminescence methods. Additional information was obtained by standardised questionnaires. In multiple linear regression adjusted for important confounders, women in the lowest sex-specific quintile of vitamin D showed an on average 2.1 (95% confidence interval: 0.4 to 3.9) units lower COGTEL score than women in the highest quintile. A similar, albeit slightly weaker, association was seen in males (difference of 1.7 [−0.4 to 3.8] units). Spline regression suggested non-linearity with a distinct decline in cognitive performance in the lower range of vitamin D levels. Our findings support suggestions that low levels of vitamin D may be associated with reduced cognitive functioning in the elderly.