Testing neural machine translation against different levels of specialisation. An exploratory investigation across legal genres and languages

This paper aims to shed light on the relationship between NMT and legal translation by presenting a small-scale study that looks into the performance of NMT in highly specialised legal genres to research whether, how, and to what extent the genre and the level of specialisation of the source text, and the target language affect the quality of the raw output. The data were extracted from two sets of target texts belonging to three different genres – i. e. a power of attorney, a memorandum opinion, and a share purchase agreement – which were machine-translated from English into two target languages – i. e. Spanish and Italian. The analysis considered the acceptability of the target-language solutions selected by DeepL through a sample revision procedure based on rich points (PACTE 2009) and focused on translation errors in the attempt to find any correlations between acceptability, errors, and the genre and/or the target language under scrutiny.