We report on a method to certify a unitary operation with the help of source and measurement apparatuses whose calibration throughout the certification process needs not be trusted. As in the device-independent paradigm our certification method relies on a Bell test and requires no assumption on the underlying Hilbert space dimension, but it removes the need for high detection efficiencies by including the single additional assumption that non-detected events are independent of the measurement settings. The relevance of the proposed method is demonstrated experimentally by bounding the unitarity of a quantum frequency converter. The experiment starts with the heralded creation of a maximally entangled two-qubit state between a single $^{40}$Ca$^{+}$ ion and a 854 nm photon. Entanglement preserving frequency conversion to the telecom band is then realized with a non-linear waveguide embedded in a Sagnac interferometer. The resulting ion-telecom photon entangled state is assessed by means of a Bell-CHSH test from which the quality of the frequency conversion is quantified. We demonstrate frequency conversion with an average certified fidelity of ≥84% and an efficiency ≥3.1 × 10$^{−6}$ at a confidence level of 99%. This ensures the suitability of the converter for integration in quantum networks from a trustful characterization procedure.