Objectives To assess the association of different diabetes-related variables, including self- and physician-reported information, as well as biomarkers, with cognitive functioning in the elderly general population in Germany. Design Cross-sectional observational study. Setting and Participants A total of 1,697 subjects with a mean ± standard deviation age of 74 ± 2.8 years were included. These were recruited from among the participants of an ongoing epidemiological study of the elderly general population in Saarland state and had been recruited 5 years earlier on the occasion of a health screening exam by their general practitioners. Measurements Cognitive functioning across six subdomains was assessed using the Cognitive Telephone Screening Instrument. Data on prevalent diabetes at baseline were obtained from the study participants and their general practitioners. Baseline fasting glucose was assessed as part of the screening exam, and baseline HbA1c was determined centrally by standardized methods. Results The association of cognitive functioning with self-reported diabetes (N = 189) was more pronounced than with physician-reported diabetes (N = 280). HbA1c showed a nonlinear association with cognitive functioning, with a peak of cognitive performance in the central quintile of HbA1c. In the case of fasting glucose, lower cognitive functioning was only observed in the highest quintile. The estimates were robust in confounder-adjusted models, but attentuated when excluding subjects with baseline prevalent or follow-up incident diabetes. Conclusions Future studies of diabetes-related biomarkers and cognition should take possible nonlinearity of the relationships into account, as the strength of the associations otherwise might be underestimated.