COVID-19 vaccination raised concerns about its potential effects on physical performance. To assess the impact of COVID-19 vaccination on the perceived change in physical performance, we conducted an online survey among elite athletes from Belgium, Canada, France and Luxembourg, with questions about socio-demographics, COVID-19 vaccination, perceived impact on physical performance and perceived pressure to get vaccinated. Full vaccination was defined as two doses of mRNA or vector vaccine or a heterologous vaccine scheme. Among 1106 eligible athletes contacted, 306 athletes answered the survey and were included in this study. Of these, 72% perceived no change in their physical performance, 4% an improvement and 24% a negative impact following full COVID-19 vaccination. For 82% of the included athletes, the duration of the negative vaccine reactions was ≤3 days. After adjustment for potential confounding variables, practicing an individual sport, a duration of vaccine reactions longer than 3 days, a high level of vaccine reaction and the perceived pressure to get vaccinated were independently associated with a perceived negative impact on physical performance of more than 3 days after the vaccination. The perceived pressure to get vaccinated appears to be a parameter associated with the negative perceived change in the physical performance and deserves further consideration.