Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languages

Multilingual neural machine translation systems learn to map sentences of different lan-
guages into a common representation space. Intuitively, with a growing number of seen
languages the encoder sentence representation grows more flexible and easily adaptable to
new languages. In this work, we test this hypothesis by zero-shot translating from unseen
languages. To deal with unknown vocabularies from unknown languages we propose a
setup where we decouple learning of vocabulary and syntax, i.e. for each language we
learn word representations in a separate step (using cross-lingual word embeddings), and
then train to translate while keeping those word representations frozen. We demonstrate that
this setup enables zero-shot translation from entirely unseen languages. Zero-shot translat-
ing with a model trained on Germanic and Romance languages we achieve scores of 42.6
BLEU for Portuguese-English and 20.7 BLEU for Russian-English on TED domain. We ex-
plore how this zero-shot translation capability develops with varying number of languages
seen by the encoder. Lastly, we explore the effectiveness of our decoupled learning strat-
... mehr
egy for unsupervised machine translation. By exploiting our model’s zero-shot translation ca-
pability for iterative back-translation we attain near parity with a supervised setting.