We study the electronic structure of bulk 1T-TaSe$_2$ in the charge density wave phase at low temperature. Our spatially and angle resolved photoemission (ARPES) data show insulating areas coexisting with metallic regions characterized by a chiral Fermi surface and moderately correlated quasiparticle bands. Additionally, high-resolution laser ARPES reveals variations in the metallic regions, with series of low-energy states, whose energy, number and dispersion can be explained by the formation of quantum well states of different thicknesses. Dynamical mean field theory calculations show that the observed rich behaviour can be rationalized by assuming occasional stacking faults of the charge density wave. Our results indicate that the diverse electronic phenomena reported previously in 1T-TaSe$_2$ are dictated by the stacking arrangement and the resulting quantum size effects while correlation effects play a secondary role.