KIT | KIT-Bibliothek | Impressum | Datenschutz

Findings of the WMT26 Multilingual Instruction Shared Task: Small Models Are Not Yet Polyglots

Schmidtova, Patricia; Artemova, Ekaterina; Aycock, Seth; Bafna, Niyati; Banga, Shobhit; Kaur, Manmeet; Kocmi, Tom; Koehn, Philipp; Liu, Danni ORCID iD icon 1; Luu, Nam; Papi, Sara; Savoldi, Beatrice; Shmatova, Mariya; Sidh, Hanuman; Zerminova, Evfrosiniya; Zouhar, Vilem; Züfle, Maike ORCID iD icon 1; Chen, Pinzhen; Haddow, Barry [Hrsg.]; ... mehr

Abstract:

We present the findings of the WMT26 Multilingual Instruction Shared Task (MIST), evalu-
ating small (≤10B parameter) open-weight language models across 24 languages on three subtasks: context-based QA, cross-lingual summarization, and open-ended generation. Submissions from eight teams explore fine-tuning, knowledge distillation, and prompt engineering. Our findings demonstrate that small models are not yet polyglots: while leading systems perform well on high-resource languages, per-
formance drops sharply on low-resource varieties, and comprehension degrades signifi-
cantly when transferring across non-English language pairs. In open-ended generation, human evaluation and automatic rule-based verification exhibit strong overall rank correlation (ρ=0.79) but diverge locally at the top: human evaluators favor distilled pipelines with superior fluency and cross-lingual equity, whereas automatic verifiers reward concise promptengineered systems. Furthermore, multi-stage pipelines suffer from prompt-language inertia on cross-lingual instructions, and summariza-
tion exhibits severe English leakage unless penalized by language gating. ... mehr


Verlagsausgabe §
DOI: 10.5445/IR/1000198780
Veröffentlicht am 04.11.2026
Originalveröffentlichung
DOI: 10.18653/2026.wmt-main.50
Cover der Publikation
Zugehörige Institution(en) am KIT Institut für Anthropomatik und Robotik (IAR)
Publikationstyp Proceedingsbeitrag
Publikationsmonat/-jahr 10.2026
Sprache Englisch
Identifikator ISBN: 979-8-89176-474-3
KITopen-ID: 1000198780
Erschienen in Proceedings of the Eleventh Conference on Machine Translation Hrsg.: Haddow, Barry; Kocmi, Tom; Koehn, Philipp; Monz, Christof
Veranstaltung 11th Conference on Machine Translation (WMT 2026), Budapest, Ungarn, 28.10.2026 – 29.10.2026
Verlag Association for Computational Linguistics (ACL)
Seiten 1024–1045
Serie Proceedings of the Eleventh Conference on Machine Translation
Externe Relationen Siehe auch
Nachgewiesen in OpenAlex
KIT – Die Universität in der Helmholtz-Gemeinschaft
KITopen Landing Page