Ecosystem functioning is simultaneously affected by changes in community composition and environmental change such as increasing atmospheric carbon dioxide (CO2) and subsequent ocean acidification. However, it largely remains uncertain how the effects of these factors compare to each other. Addressing this question, we experimentally tested the hypothesis that initial community composition and elevated CO2 are equally important to the regulation of phytoplankton biomass. We full-factorially exposed three compositionally different marine phytoplankton communities to two different CO2 levels and examined the effects and relative importance(x2) of the two factors and their interaction on phytoplankton biomass at bloom peak. The results showed that initial community composition had a significantly greater impact than elevated CO2 on phytoplankton biomass, which varied largely among communities. We suggest that the different initial ratios between cyanobacteria, diatoms, and dinoflagellates might be the key for the varying competitive and thus functional outcome among communities. Furthermore, the results showed that depending on initial community composition elevated CO2 selected for larger sized diatoms, which led to increased total phytoplankton biomass. This study highlights the relevance of initial community composition, which strongly drives the functional outcome, when assessing impacts of climate change on ecosystem functioning. In particular, the increase in phytoplankton biomass driven by the gain of larger sized diatoms in response to elevated CO2 potentially has strong implications for nutrient cycling and carbon export in future oceans.