NOVOTNÝ, Vít. The Art of Reproducible Machine Learning: A Survey of Methodology in Word Vector Experiments. In Aleš Horák, Pavel Rychlý, Adam Rambousek. Proceedings of the Fourteenth Workshop on Recent Advances in Slavonic Natural Language Processing, RASLAN 2020. Brno: Tribun EU, 2020, p. 55-64. ISBN 978-80-263-1600-8.
Other formats:   BibTeX LaTeX RIS
Basic information
Original name The Art of Reproducible Machine Learning: A Survey of Methodology in Word Vector Experiments
Authors NOVOTNÝ, Vít (203 Czech Republic, guarantor, belonging to the institution).
Edition Brno, Proceedings of the Fourteenth Workshop on Recent Advances in Slavonic Natural Language Processing, RASLAN 2020, p. 55-64, 10 pp. 2020.
Publisher Tribun EU
Other information
Original language English
Type of outcome Proceedings paper
Field of Study 10201 Computer sciences, information science, bioinformatics
Country of publisher Czech Republic
Confidentiality degree is not subject to a state or trade secret
Publication form printed version "print"
WWW PDF Domovská stránka workshopu
RIV identification code RIV/00216224:14330/20:00117106
Organization unit Faculty of Informatics
ISBN 978-80-263-1600-8
ISSN 2336-4289
UT WoS 000655471300006
Keywords in English Machine learning; word vectors; word2vec; fastText; word analogy; reproducibility
Tags machine learning, word embeddings
Tags International impact
Changed by Changed by: RNDr. Vít Starý Novotný, Ph.D., učo 409729. Changed: 3/1/2023 13:53.
Abstract

Since the seminal work of Mikolov et al. (2013), word vectors of log-bilinear SVMs have found their way into many NLP applications as an unsupervised measure of word relatedness.

Due to the rapid pace of research and the publish-or-perish mantra of academic publishing, word vector experiments contain undisclosed parameters, which make them difficult to reproduce.

In our work, we introduce the experiments and their parameters, compare the published experimental results with our own, and suggest default parameter settings and ways to make previous and future experiments easier to reproduce.

We show that the lack of variable control can cause up to 24% difference in accuracy on the word analogy tasks.
Links
MUNI/A/1076/2019, interní kód MUName: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 20 (Acronym: SKOMU)
Investor: Masaryk University, Category A
MUNI/A/1411/2019, interní kód MUName: Aplikovaný výzkum: softwarové architektury kritických infrastruktur, bezpečnost počítačových systémů, zpracování přirozeného jazyka a jazykové inženýrství, vizualizaci velkých dat a rozšířená realita.
Investor: Masaryk University, Category A
PrintDisplayed: 1/5/2024 00:19