PURPOSE: To estimate the impact of traffic-related noise annoyance on health-related quality of life (HrQoL) in a population-based study and potential effect modification by gender. METHODS: The study included 5,021 participants of the Swiss Cohort Study of Air Pollution and Lung Disease in Adults second survey. The association between traffic-related noise annoyance, measured on an 11-point scale, and HrQoL, based on SF-36 scores, was investigated by multivariate regression analysis. Effect sizes were calculated, and interactions by gender and chronic disease status examined. RESULTS: Thirteen percentage of the study population reported high annoyance due to traffic. Women were more likely to report high noise annoyance (adjOR 1.23; 95%CI 1.01-1.48). Except for general health, all SF-36 scores showed a significant negative association with noise annoyance. The respective effect sizes ranged between 0.13 and 0.54. Significant effect modification by gender and chronic disease status was present in specific SF-36 domains. CONCLUSION: This paper presents first evidence of an inverse relationship of noise annoyance and HrQoL in a general population. Although the estimated effects are small to moderate for individuals, they may add up to a relevant public health impact.