This study aimed (1) to investigate the association between body mass index (BMI), physical activity (PA), and physical function (PF) with health-related quality of life (HRQoL), and (2) to examine in-depth whether PA and PF mediate the relationship between BMI and HRQoL in older adults. We investigated 802 individuals (mean age 69.8 ± 5.6 years), residents of the Autonomous Region of Madeira, Portugal. Harmol and PA were assessed using the SF-36 and Baecke questionnaires, respectively, and PF by the Senior Fitness Test. The prevalence of overweight was 71.3%, 26.5% for normal weight, and 2.1% for underweight. We verified a small correlation between age and sex with BMI, PA, PF, and medium borderline with HRQoL. After adjusting for covariates (i.e., sex, age, comorbidities), the multivariate regression analysis indicated a 93.1% chance of improvement in HRQoL for low BMI, while PA and PF revealed a chance of increasing HRQoL by 91.8% and 60.0%, respectively. According to the serial mediation pathway, PA and PF partially mediated the association between BMI and HRQoL by 32.3% and 81.5%, respectively. The total variance of the model was 90%. It was concluded that BMI can negatively affect HRQoL. On the other hand, PA and PF are able to increase HRQoL levels during the aging process.