Post-edited quality, post-editing behaviour and human evaluation: a case study

Ilse Depraetere (1) , Nathalie de Sutter , Arda Tezcan

1 STL - Savoirs, Textes, Langage (STL) - UMR 8163

In this chapter, we address the correlation between post-editing similarity and the human evaluation of machine translation. We were interested to find out whether a high similarity score corresponded to a high quality score and vice versa in the sample that we compiled for the purposes of the case study. A group of translation trainees post-edited a sample and a number of these informants also rated the MT output for quality on a five-point scale. We calculated Pearson's correlation coefficient as well as the relative standard deviation per informant for each activity with a view to determining which of the two evaluation methods appeared to be the more reliable measurement given the project settings. Our sample also enabled us to test whether MT enhances the productivity of translation trainees, and whether the quality of post-edited sentences is different from the quality of sentences translated 'from scratch'.

Mots clés: post-édition

Domaines: Linguistique

Ilse Depraetere, Nathalie de Sutter, Arda Tezcan. Post-edited quality, post-editing behaviour and human evaluation: a case study. Laura Winther-Balling, Lucia Specia, Michael Carl, Michel Simard, Sharon O'Brien. Post-editing of machine translation: processes and applications, Cambridge Scholars Publishing, pp.78-108., 2014. ⟨halshs-01060447⟩