It is well established that cerebral white matter lesions (WML), present in the majority of older adults, are associated with cardiovascular and cerebrovascular diseases and also with cognitive decline. However, much less is known about how WML are related to other important individual characteristics and about the generality vs. brain region-specificity of WML. In a longitudinal study of 112 community-dwelling adults (age 50–71 years at study entry), we used a machine learning approach to evaluate the relative strength of 52 variables in association with WML burden. Variables included socio-demographic, lifestyle, and health indices—as well as multiple cognitive abilities (modeled as latent constructs using factor analysis)—repeatedly measured at three- to six-year intervals. Greater chronological age, symptoms of cardiovascular disease, and processing speed declines were most strongly linked to elevated WML burden (accounting for ∼49% of variability in WML). Whereas frontal lobe WML burden was associated both with elevated cardiovascular symptoms and declines in processing speed, temporal lobe WML burden was only significantly associated with declines in processing speed. These latter outcomes suggest that age-related WML-cognition associations may be etiologically heterogeneous across frontotemporal cerebral regions.