KIT | KIT-Bibliothek | Impressum | Datenschutz

Efficient Speech Translation with Pre-trained Models

Li, Zhaolin 1; Niehues, Jan ORCID iD icon 1
1 Institut für Anthropomatik und Robotik (IAR), Karlsruher Institut für Technologie (KIT)

Abstract:

When building state-of-the-art speech translation models, the need for large computational resources is a significant obstacle due to the large training data size and complex models. The availability of pre-trained models is a promising opportunity to build strong speech translation systems efficiently. In a first step, we investigate efficient strategies to build cascaded and end-to-end speech translation systems based on pre-trained models. Using this strategy, we can train and apply the models on a single GPU. While the end-to-end models show superior translation performance to cascaded ones, the application of this technology has a limitation on the need for additional end-to-end training data. In a second step, we proposed an additional similarity loss to encourage the model to generate similar hidden representations for speech and transcript. Using this technique, we can increase the data efficiency and improve the translation quality by 6 BLEU points in scenarios with limited end-to-end training data.


Preprint §
DOI: 10.5445/IR/1000152224
Veröffentlicht am 17.11.2022
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Anthropomatik und Robotik (IAR)
Publikationstyp Proceedingsbeitrag
Publikationsdatum 03.11.2022
Sprache Englisch
Identifikator KITopen-ID: 1000152224
Erschienen in Conference on Neural Information Processing Systems (NeurIPS) 2022 : 2nd workshop on Efficient Natural Language and Speech Processing (ENLSP), 2nd December 2022, New Orleans
Veranstaltung 36th Annual Conference on Neural Information Processing Systems (NIPS 2022), Online, 28.11.2022 – 09.12.2022
Bemerkung zur Veröffentlichung 2nd workshop on Efficient Natural Language and Speech Processing (ENLSP), 2nd December 2022, New Orleans
Nachgewiesen in arXiv
KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft
KITopen Landing Page