Detailed Information on Publication Record
2021
Towards Domain Robustness of Neural Language Models
ŠTEFÁNIK, Michal and Petr SOJKABasic information
Original name
Towards Domain Robustness of Neural Language Models
Authors
ŠTEFÁNIK, Michal (703 Slovakia, guarantor, belonging to the institution) and Petr SOJKA (203 Czech Republic, belonging to the institution)
Edition
Brno, Recent Advances in Slavonic Natural Language Processing (RASLAN 2021), p. 91-103, 13 pp. 2021
Publisher
Tribun EU
Other information
Language
English
Type of outcome
Stať ve sborníku
Field of Study
10200 1.2 Computer and information sciences
Country of publisher
Czech Republic
Confidentiality degree
není předmětem státního či obchodního tajemství
Publication form
printed version "print"
References:
RIV identification code
RIV/00216224:14330/21:00123248
Organization unit
Faculty of Informatics
ISBN
978-80-263-1670-1
ISSN
Keywords in English
Generalization; Debiasing; Domain extrapolation; Domain adaptation; Domain robustness; Neural language models
Změněno: 15/5/2024 10:24, RNDr. Pavel Šmerk, Ph.D.
Abstract
V originále
This work summarises recent progress in generalization evaluation and training of deep neural networks, categorized in data-centric and model-centric overviews. Grounded in the results of the referenced work, we propose three future directions towards reaching higher robustness of language models to an unknown domain or its adaptation to an existing domain of interest. In the example propositions that practically complement each of the directions, we introduce novel ideas of a) dynamic objective selection, b) language modeling respecting the token similarities to the ground truth and c) a framework of additive component of the loss utilizing the well-performing generalization measures.
Links
| MUNI/A/1573/2020, interní kód MU |
|