Certain personality traits, in particular higher neuroticism, have been associated, on one hand, with elevated cortisol levels, and on the other hand, with poorer cognitive performance. At the same time, several studies highlighted the association between high cortisol and poor cognitive functioning. Here, we hypothesized that increased cortisol may be associated with poorer cognition and with certain personality traits (mainly high neuroticism), and that personality might explain the association between cortisol and cognition. A cross-sectional analysis was conducted using data from Colaus/PsyColaus, a population-based study involving residents of Lausanne, Switzerland. Salivary cortisol samples (upon waking, 30 min after waking, at 11 am and at 8 pm) along with cognitive and personality measures were obtained from 643 non-demented participants aged at least 65. Personality traits were assessed using the NEO Five-Factor Inventory (NEO-FFI). We examined the links between the cortisol Area under the Curve (AUC), the Clinical Dementia Rating Sum of Boxes (CDRSOB) and the NEO-FFI scores. No association was found between personality traits and the CDRSOB or the MMSE score, controlling for age, sex, depression, education and BMI. However, the executive functioning domain z-score was negatively associated with agreeableness (p = 0.005; slope = -0.107 [-0.181; -0.033]) and openness (p = 0.029; slope = -0.081 [-0.154; -0.008]) after controlling for age, sex, depression, education and BMI. The CDRSOB score was positively associated with the cortisol AUC after controlling for age, sex, BMI, education and depression, (p = 0.003; slope = 0.686 [0.240; 1.333]). This association remained significant after controlling for personality traits and for the interaction between personality traits and the cortisol AUC (p = 0.006; slope = 0.792 [0.233; 1.352]. High agreeableness and openness might be associated with poorer executive performance in later life. Increased cortisol may be associated with both specific personality traits (high extraversion, low openness) and worse cognitive performance. Increased salivary cortisol does not mediate the relationship between personality traits and cognitive impairment.