Heat pumps play an important role in decarbonizing the heating supply of buildings and they allow to increase the self-consumption of PV electricity, especially when supported by electricity or heat storage. In this study, we develop an open-source model to optimize PV-coupled heat pumps, with and without electricity and heat storage, and we compare their performance for three types of single-family houses with different thermal envelope quality paired with 549 electricity profiles. We analyze trade-offs between prosumer benefits and grid impacts, namely bill minimization, and maximum grid relief, depending on the type of storage and incentives for grid peak reduction (i.e., a capacity-based tariff). We conclude that the use of heat storage reduces the levelized cost of meeting the electricity demand between 13–26% compared to the baseline case without storage, in particular when heat pumps are used for both space heating and domestic hot water (DHW). Regarding total self-consumption rates, both storage technologies, namely batteries and hot water tanks (which supply both space heating and DHW) achieve similar rates ranging between 30–39%. In contrast, batteries are found to be very effective in reducing the peak demand (14–17% compared to the baseline scenario), but only if the retail tariff has a capacity-based component. Interestingly, the quality of the envelope plays a key role and heat pumps can double the power peak demand in poorly insulated houses, with thermal storage increasing the power peak demand further up to 8%, compared to the baseline, regardless of the storage technology. Thus, we conclude that policy makers should promote thermal retrofitting of the building stock to avoid the upgrading of the distribution grid.