Spectral broadening of ultrashort laser pulses is simultaneously described by either self-phase modulation (SPM) or four-wave mixing (FWM). The latter implies the instantaneous conservation of both the photon number and energy, while the former describes a time-dependent frequency shift, implying a violation of the energy conservation if the number of photons is to be conserved in each time slice. We resolve this paradox by considering the transient energy storage in the propagation medium, that can be calculated in the SPM formalism via the dephasing between the incident pulse and the medium polarization leading to an effective imaginary part in the third-order susceptibility. In parallel, considering the temporal variation of the incident intensity in FWM offsets the instantaneous frequency.