The present study aimed to examine whether gait speed (GS), body balance (BB), and falls mediated the relationship between physical activity (PA) and health-related quality of life (HRQoL) in community-dwelling older adults. This is a cross-sectional study that included 305 men and 314 women (69.5 ± 5.6 years), residing in the Autonomous Region of Madeira, Portugal. HRQoL and PA were assessed using the SF-36 and Baecke Questionnaires, respectively. While BB was obtained by the Fullerton Advance Balance (FAB) scale, GS by the 50-foot (15 m) walk test, and the frequency of falls was obtained by self-report. According to the analyses, when GS and BB were placed concomitantly as mediators, the direct effect revealed by the model revealed a non-significant relationship between PA and falls. Thus, in the context of falls, GS and BB partially mediated the association between PA and HRQoL in approximately 29.7%, 56%, and 49.2%, respectively. The total HRQoL model explained a variance of 36.4%. The results can help to understand the role that GS, BB, and falls play in the relationship between PA and HRQoL of the vulnerable older adult population.