Regressive Ensemble for Machine Translation Quality Evaluation

D 2021

Regressive Ensemble for Machine Translation Quality Evaluation

ŠTEFÁNIK, Michal, Vít NOVOTNÝ and Petr SOJKA

Basic information

Original name

Regressive Ensemble for Machine Translation Quality Evaluation

Authors

ŠTEFÁNIK, Michal (703 Slovakia, belonging to the institution), Vít NOVOTNÝ (203 Czech Republic, belonging to the institution) and Petr SOJKA (203 Czech Republic, guarantor, belonging to the institution)

Edition

Online and Punta Cana, Dominican Republi, Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21), p. 1041-1048, 8 pp. 2021

Publisher

ACL

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

60203 Linguistics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

References:

paper preprint

RIV identification code

RIV/00216224:14330/21:00122292

Organization unit

Faculty of Informatics

ISBN

978-1-954085-94-7

Keywords (in Czech)

strojový překlad; automatické vyhodnocení kvality překladu;

Keywords in English

machine translation; translation quality metrics; regressive ensemble for machine translation quality evaluation

Abstract

V originále

This work introduces a simple regressive ensemble for evaluating machine translation quality based on a set of novel and established metrics. We evaluate the ensemble using a correlation to expert-based MQM scores of the WMT 2021 Metrics workshop. In both monolingual and zero-shot cross-lingual settings, we show a significant performance improvements over single systems. In the cross-lingual settings, we also demonstrate that an ensemble approach is well-applicable to unseen languages. Furthermore, we identify a strong reference-free baseline that consistently outperforms the commonly-used BLEU and METEOR measures and significantly improves our ensemble's performance.

Links

MUNI/A/1549/2020, interní kód MU

Name: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 21 (Acronym: SKOMU)

Investor: Masaryk University

Citovat

ŠTEFÁNIK, Michal, Vít NOVOTNÝ and Petr SOJKA. Regressive Ensemble for Machine Translation Quality Evaluation. Online. In Loïc Barrault et al. Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21). Online and Punta Cana, Dominican Republi: ACL, 2021, p. 1041-1048. ISBN 978-1-954085-94-7.

@inproceedings{1790577,
   author = {Štefánik, Michal and Novotný, Vít and Sojka, Petr},
   address = {Online and Punta Cana, Dominican Republi},
   booktitle = {Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21)},
   editor = {Loïc Barrault et al.},
   keywords = {machine translation; translation quality metrics; regressive ensemble for machine translation quality evaluation},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Online and Punta Cana, Dominican Republi},
   isbn = {978-1-954085-94-7},
   pages = {1041-1048},
   publisher = {ACL},
   title = {Regressive Ensemble for Machine Translation Quality Evaluation},
   url = {https://aclanthology.org/2021.wmt-1.112},
   year = {2021}
}

TY  - JOUR
ID  - 1790577
AU  - Štefánik, Michal - Novotný, Vít - Sojka, Petr
PY  - 2021
TI  - Regressive Ensemble for Machine Translation Quality Evaluation
PB  - ACL
CY  - Online and Punta Cana, Dominican Republi
SN  - 9781954085947
KW  - machine translation
KW  - translation quality metrics
KW  - regressive ensemble for machine translation quality evaluation
UR  - https://aclanthology.org/2021.wmt-1.112
N2  - This work introduces a simple regressive ensemble for evaluating machine translation quality based on a set of novel and established metrics. We evaluate the ensemble using a correlation to expert-based MQM scores of the WMT 2021 Metrics workshop. In both monolingual and zero-shot cross-lingual settings, we show a significant performance improvements over single systems. In the cross-lingual settings, we also demonstrate that an ensemble approach is well-applicable to unseen languages. Furthermore, we identify a strong reference-free baseline that consistently outperforms the commonly-used BLEU and METEOR measures and significantly improves our ensemble's performance.
ER  -

ŠTEFÁNIK, Michal, Vít NOVOTNÝ and Petr SOJKA. Regressive Ensemble for Machine Translation Quality Evaluation. Online. In Loïc Barrault et al. \textit{Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21)}. Online and Punta Cana, Dominican Republi: ACL, 2021, p.~1041-1048. ISBN~978-1-954085-94-7.

Detailed Information on Publication Record