Background. Concern about falling is common in older people. Various related psychological constructs as well as poor balance and slow gait have been associated with decreased gray matter (GM) volume in old age. The current study investigates the association between concern about falling and voxel-wise GM volumes. Methods. A total of 281 community-dwelling older people aged 70–90 years underwent structural magnetic resonance imaging. Concern about falling was assessed using Falls Efficacy Scale-International (FES-I). For each participant, voxel-wise GM volumes were generated with voxel-based morphometry and regressed on raw FES-I scores (p < .05 family-wise error corrected on cluster level). Results. FES-I scores were negatively correlated with total brain volume (r = −.212; p ≤ .001), GM volume (r = −.210; p ≤ .001), and white matter volume (r = −.155; p ≤ .001). Voxel-based morphometry analysis revealed significant negative associations between FES-I and GM volumes of (i) left cerebellum and bilateral inferior occipital gyrus (voxels-in-cluster = 2,981; p < .001) and (ii) bilateral superior frontal gyrus and left supplementary motor area (voxels-in-cluster = 1,900; p = .004). Additional adjustment for vision and physical fall risk did not alter these associations. After adjustment for anxiety, only left cerebellum and bilateral inferior occipital gyrus remained negatively associated with FES-I scores (voxels-in-cluster = 2,426; p < .001). Adjustment for neuroticism removed all associations between FES-I and GM volumes. Conclusions. Our study findings show that concern about falling is negatively associated with brain volumes in areas important for emotional control and for motor control, executive functions and visual processing in a large sample of older men and women. Regression analyses suggest that these relationships were primarily accounted for by psychological factors (generalized anxiety and neuroticism) and not by physical fall risk or vision.