The relationship between personality profiles and brain integrity in old age is still a matter of debate. We examined the association between Big Five factor and facet scores and MRI brain volume changes on a 54-month follow-up in 65 elderly controls with 3 neurocognitive assessments (baseline, 18 months, and 54 months), structural brain MRI (baseline and 54 months), brain amyloid PET during follow-up, and APOE genotyping. Personality was assessed with the Neuroticism Extraversion Openness Personality Inventory-Revised. Regression models were used to identify predictors of volume loss including time, age, sex, personality, amyloid load, presence of APOE ε4 allele, and cognitive evolution. Lower agreeableness factor scores (and 4 of its facets) were associated with lower volume loss in the hippocampus, entorhinal cortex, amygdala, mesial temporal lobe, and precuneus bilaterally. Higher openness factor scores (and 2 of its facets) were also associated with lower volume loss in the left hippocampus. Our findings persisted when adjusting for confounders in multivariable models. These data suggest that the combination of low agreeableness and high openness is an independent predictor of better preservation of brain volume in areas vulnerable to neurodegeneration.