Quantum key distribution (QKD) enables provably secure communication between two parties over an optical fiber that arguably withstands any form of attack. Besides the need for a suitable physical signalling scheme and the corresponding devices, QKD also requires a secret key distillation protocol. This protocol and the involved signal processing handle the reliable key agreement process over the fragile quantum channel, as well as the necessary post-processing of key bits to avoid leakage of secret key information to an eavesdropper. In this paper we present in detail an implementation of a key distillation engine for a QKD system based on the coherent one-way (COW) protocol. The processing of key bits by the key distillation engine includes agreement on quantum bit detections (sifting), information reconciliation with forward error correction coding, parameter estimation, and privacy amplification over an authenticated channel. We detail the system architecture combining all these processing steps, and discuss the design trade-offs for each individual system module. We also assess the performance and efficiency of our key distillation implementation in terms of throughput, error correction capabilities, and resource utilization. On a single-FPGA (Xilinx Virtex-6 LX240T) platform, the system supports distilled key rates of up to 4 Mbps.