Compressed FastText Models for Czech Tagger

D 2022

Compressed FastText Models for Czech Tagger

NEVĚŘILOVÁ, Zuzana

Základní údaje

Originální název

Compressed FastText Models for Czech Tagger

Autoři

NEVĚŘILOVÁ, Zuzana (203 Česká republika, garant, domácí)

Vydání

Brno, Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2022, od s. 79-87, 9 s. 2022

Nakladatel

Tribun EU

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10200 1.2 Computer and information sciences

Stát vydavatele

Česká republika

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

tištěná verze "print"

Odkazy

Domovská stránka workshopu Plný text

Kód RIV

RIV/00216224:14330/22:00127484

Organizační jednotka

Fakulta informatiky

ISBN

978-80-263-1752-4

ISSN

Klíčová slova anglicky

model compression; FastText; embedding evaluation; Czech tagger

Změněno: 20. 12. 2022 12:43, Mgr. Jitka Nováčková

Anotace

V originále

We are building a new tagger for the Czech language that uses two models: the FastText model for word embeddings and a neural network that assigns tags to tokens. In the deployment, we are struggling with model sizes. Since the model size is a common obstacle in various tasks, several compression methods exist. Authors of the methods often claim that the impact on model performance is minimal. However, the evaluation is done on the two tasks the word embeddings are evaluated on: word analogy and word similarity. No information is provided for the evaluation of subsequent tasks. In this paper, we have trained a FastText word embedding model on more recent data. We retrained the tagger with the same parameters using compressed and uncompressed variants of the original FastText model and the new one. After comparing the results, we can see quantization methods are suitable, possibly together with pruning, without significant impact on the tagger performance. The precision dropped by 0.1 percentage point only in quantized models. All tested compression methods reduce the model size 10–100 times.

Návaznosti

LM2018101, projekt VaV

Název: Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy (Akronym: LINDAT/CLARIAH-CZ)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, LINDAT/CLARIAH-CZ - Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy

Citovat

NEVĚŘILOVÁ, Zuzana. Compressed FastText Models for Czech Tagger. In Aleš Horák, Pavel Rychlý, Adam Rambousek. Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2022. Brno: Tribun EU, 2022, s. 79-87. ISBN 978-80-263-1752-4.

@inproceedings{2240154,
   author = {Nevěřilová, Zuzana},
   address = {Brno},
   booktitle = {Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2022},
   editor = {Aleš Horák, Pavel Rychlý, Adam Rambousek},
   keywords = {model compression; FastText; embedding evaluation; Czech tagger},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Brno},
   isbn = {978-80-263-1752-4},
   pages = {79-87},
   publisher = {Tribun EU},
   title = {Compressed FastText Models for Czech Tagger},
   url = {https://raslan2022.nlp-consulting.net/},
   year = {2022}
}

TY  - CONF
ID  - 2240154
AU  - Nevěřilová, Zuzana
PY  - 2022
TI  - Compressed FastText Models for Czech Tagger
PB  - Tribun EU
CY  - Brno
SN  - 9788026317524
KW  - model compression
KW  - FastText
KW  - embedding evaluation
KW  - Czech tagger
UR  - https://raslan2022.nlp-consulting.net/
N2  - We are building a new tagger for the Czech language that uses two models: the FastText model for word embeddings and a neural network that assigns tags to tokens. In the deployment, we are struggling with model sizes. Since the model size is a common obstacle in various tasks, several compression methods exist. Authors of the methods often claim that the impact on model performance is minimal. However, the evaluation is done on the two tasks the word embeddings are evaluated on: word analogy and word similarity. No information is provided for the evaluation of subsequent tasks. In this paper, we have trained a FastText word embedding model on more recent data. We retrained the tagger with the same parameters using compressed and uncompressed variants of the original FastText model and the new one. After comparing the results, we can see quantization methods are suitable, possibly together with pruning, without significant impact on the tagger performance. The precision dropped by 0.1 percentage point only in quantized models. All tested compression methods reduce the model size 10–100 times.
ER  -

NEVĚŘILOVÁ, Zuzana. Compressed FastText Models for Czech Tagger. In Aleš Horák, Pavel Rychlý, Adam Rambousek. \textit{Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2022}. Brno: Tribun EU, 2022, s.~79-87. ISBN~978-80-263-1752-4.

Přehled o publikaci