This paper presents an evaluation conducted with end-users of translations produced by Swiss Post's in-house Language Service. The aim is to assess whether end-users i) would rate post-edited MT more highly than raw MT; ii) would find that Swiss Post's customized NMT system produces better results than a general-purpose, off-the-shelf NMT engine (DeepL) and, lastly, when aware of translation production metadata, iii) would be willing to pay for post-edited texts. This latter aspect in particular was intended to help determine whether the customers would still value human intervention or whether they would rather accept a lower quality translation and associated risks if this means they can save on costs. Results show that the post-edited texts are preferred by the majority of the participants, even when production metadata are revealed. The in-house customized engine seems to produce better results than DeepL, since the end-users choose the raw output from our system more often than from DeepL.