We show that an entanglement swapping operation performed with spontaneous parametric down-conversion can be made faithful without postselection using sum-frequency generation. This invites us to revisit the sum-frequency process and from a proof-of-principle experiment, we demonstrate that it provides a realistic solution for nonlinear optics at the single-photon level. This opens the way to attractive alternatives to six-photon protocols based on linear optics used, e.g., for the heralded creation of maximally entangled pairs or for device-independent quantum key distribution.