Proceedings chapter
Open access

Differences between SMT and NMT Output - a Translators' Point of View

Presented at Varna (Bulgaria), 5th-6th sept. 2019
Publication date2019

In this study, we compare the output quality of two MT systems, a statistical (SMT) and a neural (NMT) engine, customised for Swiss Post's Language Service using the same training data. We focus on the point of view of professional translators and investigate how they perceive the differences between the MT output and a human reference (namely deletions, substitutions, insertions and word order). Our findings show that translators more frequently consider these differences to be errors in SMT than NMT, and that deletions are the most serious errors in both architectures. We also observe there to be less agreement on differences to be corrected in NMT than SMT, suggesting that errors are easier to identify in SMT. These findings confirm the ability of NMT to produce correct paraphrases, which could also explain why BLEU is often considered to be an inadequate metric to evaluate the performance of NMT systems.

  • Statistical machine translation
  • Neural machine translation
  • MT
  • Machine translation
  • Post-editing
  • Swiss Post
  • Machine translation evaluation
Research group
Citation (ISO format)
MUTAL, Jonathan David et al. Differences between SMT and NMT Output - a Translators” Point of View. In: The Second Workshop on Human-Informed Translation and Interpreting Technology (HiT-IT 2019). Varna (Bulgaria). [s.l.] : [s.n.], 2019.
Main files (1)
Proceedings chapter (Accepted version)
  • PID : unige:123216

Technical informations

Creation09/16/2019 4:06:00 PM
First validation09/16/2019 4:06:00 PM
Update time03/15/2023 6:01:26 PM
Status update03/15/2023 6:01:26 PM
Last indexation01/17/2024 6:15:12 AM
All rights reserved by Archive ouverte UNIGE and the University of GenevaunigeBlack