Regressive Ensemble for Machine Translation Quality Evaluation

D 2021

Regressive Ensemble for Machine Translation Quality Evaluation

ŠTEFÁNIK, Michal, Vít NOVOTNÝ a Petr SOJKA

Základní údaje

Originální název

Regressive Ensemble for Machine Translation Quality Evaluation

Autoři

ŠTEFÁNIK, Michal (703 Slovensko, domácí), Vít NOVOTNÝ (203 Česká republika, domácí) a Petr SOJKA (203 Česká republika, garant, domácí)

Vydání

Online and Punta Cana, Dominican Republi, Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21), od s. 1041-1048, 8 s. 2021

Nakladatel

ACL

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

60203 Linguistics

Stát vydavatele

Spojené státy

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Odkazy

paper preprint

Kód RIV

RIV/00216224:14330/21:00122292

Organizační jednotka

Fakulta informatiky

ISBN

978-1-954085-94-7

Klíčová slova česky

strojový překlad; automatické vyhodnocení kvality překladu;

Klíčová slova anglicky

machine translation; translation quality metrics; regressive ensemble for machine translation quality evaluation

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 28. 8. 2024 15:24, RNDr. Pavel Šmerk, Ph.D.

Anotace

V originále

This work introduces a simple regressive ensemble for evaluating machine translation quality based on a set of novel and established metrics. We evaluate the ensemble using a correlation to expert-based MQM scores of the WMT 2021 Metrics workshop. In both monolingual and zero-shot cross-lingual settings, we show a significant performance improvements over single systems. In the cross-lingual settings, we also demonstrate that an ensemble approach is well-applicable to unseen languages. Furthermore, we identify a strong reference-free baseline that consistently outperforms the commonly-used BLEU and METEOR measures and significantly improves our ensemble's performance.

Návaznosti

MUNI/A/1549/2020, interní kód MU

Název: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 21 (Akronym: SKOMU)

Investor: Masarykova univerzita, Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 21

Citovat

ŠTEFÁNIK, Michal, Vít NOVOTNÝ a Petr SOJKA. Regressive Ensemble for Machine Translation Quality Evaluation. Online. In Loïc Barrault et al. Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21). Online and Punta Cana, Dominican Republi: ACL, 2021, s. 1041-1048. ISBN 978-1-954085-94-7.

@inproceedings{1790577,
   author = {Štefánik, Michal and Novotný, Vít and Sojka, Petr},
   address = {Online and Punta Cana, Dominican Republi},
   booktitle = {Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21)},
   editor = {Loïc Barrault et al.},
   keywords = {machine translation; translation quality metrics; regressive ensemble for machine translation quality evaluation},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Online and Punta Cana, Dominican Republi},
   isbn = {978-1-954085-94-7},
   pages = {1041-1048},
   publisher = {ACL},
   title = {Regressive Ensemble for Machine Translation Quality Evaluation},
   url = {https://aclanthology.org/2021.wmt-1.112},
   year = {2021}
}

TY  - JOUR
ID  - 1790577
AU  - Štefánik, Michal - Novotný, Vít - Sojka, Petr
PY  - 2021
TI  - Regressive Ensemble for Machine Translation Quality Evaluation
PB  - ACL
CY  - Online and Punta Cana, Dominican Republi
SN  - 9781954085947
KW  - machine translation
KW  - translation quality metrics
KW  - regressive ensemble for machine translation quality evaluation
UR  - https://aclanthology.org/2021.wmt-1.112
N2  - This work introduces a simple regressive ensemble for evaluating machine translation quality based on a set of novel and established metrics. We evaluate the ensemble using a correlation to expert-based MQM scores of the WMT 2021 Metrics workshop. In both monolingual and zero-shot cross-lingual settings, we show a significant performance improvements over single systems. In the cross-lingual settings, we also demonstrate that an ensemble approach is well-applicable to unseen languages. Furthermore, we identify a strong reference-free baseline that consistently outperforms the commonly-used BLEU and METEOR measures and significantly improves our ensemble's performance.
ER  -

ŠTEFÁNIK, Michal, Vít NOVOTNÝ a Petr SOJKA. Regressive Ensemble for Machine Translation Quality Evaluation. Online. In Loïc Barrault et al. \textit{Proceedings of EMNLP 2021 Sixth Conference on Machine Translation (WMT 21)}. Online and Punta Cana, Dominican Republi: ACL, 2021, s.~1041-1048. ISBN~978-1-954085-94-7.

Podrobný výpis o publikaci