AdaptOr: Objective-Centric Adaptation Framework for Language
Models

D 2022

AdaptOr: Objective-Centric Adaptation Framework for Language Models

ŠTEFÁNIK, Michal, Vít NOVOTNÝ, Nikola GROVEROVÁ and Petr SOJKA

Basic information

Original name

AdaptOr: Objective-Centric Adaptation Framework for Language Models

Authors

ŠTEFÁNIK, Michal (703 Slovakia, guarantor, belonging to the institution), Vít NOVOTNÝ (203 Czech Republic, belonging to the institution), Nikola GROVEROVÁ (203 Czech Republic) and Petr SOJKA (203 Czech Republic, belonging to the institution)

Edition

Dublin, Irsko, Proceedings of the 60th Conference of Association of Computational Linguistics, ACL 2022, p. 261-269, 9 pp. 2022

Publisher

Association for Computational Linguistics, ACL

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

United States of America

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

electronic version available online

References:

paper with code Preprint publisher's PDF

RIV identification code

RIV/00216224:14330/22:00125674

Organization unit

Faculty of Informatics

ISBN

978-1-955917-24-7

ISSN

DOI

http://dx.doi.org/10.18653/v1/2022.acl-demo.26

UT WoS

000828759800025

Keywords (in Czech)

plnotextové vyhledávání; doménová adaptace

Keywords in English

Adaptor library; domain adaptation; similarity search; vector space; embeddings

Abstract

V originále

Progress in natural language processing research is catalyzed by the possibilities given by the widespread software frameworks. This paper introduces the Adaptor library that transposes the traditional model-centric approach composed of pre-training + fine-tuning steps to the objective-centric approach, composing the training process by applications of selected objectives. We survey research directions that can benefit from enhanced objective-centric experimentation in multitask training, custom objectives development, dynamic training curricula, or domain adaptation. Adaptor aims to ease the reproducibility of these research directions in practice. Finally, we demonstrate the practical applicability of Adaptor in selected unsupervised domain adaptation scenarios.

Links

MUNI/A/1230/2021, interní kód MU

Name: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity 22 (Acronym: SKOMU)

Investor: Masaryk University

Citovat

ŠTEFÁNIK, Michal, Vít NOVOTNÝ, Nikola GROVEROVÁ and Petr SOJKA. AdaptOr: Objective-Centric Adaptation Framework for Language Models. Online. In Valerio Basile, Zornitsa Kozareva, Sanja Stajner. Proceedings of the 60th Conference of Association of Computational Linguistics, ACL 2022. Dublin, Irsko: Association for Computational Linguistics, ACL, 2022, p. 261-269. ISBN 978-1-955917-24-7. Available from: https://dx.doi.org/10.18653/v1/2022.acl-demo.26.

@inproceedings{1847597,
   author = {Štefánik, Michal and Novotný, Vít and Groverová, Nikola and Sojka, Petr},
   address = {Dublin, Irsko},
   booktitle = {Proceedings of the 60th Conference of Association of Computational Linguistics, ACL 2022},
   doi = {http://dx.doi.org/10.18653/v1/2022.acl-demo.26},
   editor = {Valerio Basile, Zornitsa Kozareva, Sanja Stajner},
   keywords = {Adaptor library; domain adaptation; similarity search; vector space; embeddings},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Dublin, Irsko},
   isbn = {978-1-955917-24-7},
   pages = {261-269},
   publisher = {Association for Computational Linguistics, ACL},
   title = {AdaptOr: Objective-Centric Adaptation Framework for Language Models},
   url = {https://paperswithcode.com/paper/adapt-mathcal-o-r-objective-centric},
   year = {2022}
}

TY  - JOUR
ID  - 1847597
AU  - Štefánik, Michal - Novotný, Vít - Groverová, Nikola - Sojka, Petr
PY  - 2022
TI  - AdaptOr: Objective-Centric Adaptation Framework for Language Models
PB  - Association for Computational Linguistics, ACL
CY  - Dublin, Irsko
SN  - 9781955917247
KW  - Adaptor library
KW  - domain adaptation
KW  - similarity search
KW  - vector space
KW  - embeddings
UR  - https://paperswithcode.com/paper/adapt-mathcal-o-r-objective-centric
N2  - Progress in natural language processing research is catalyzed by the possibilities given by the widespread software frameworks. This paper introduces the Adaptor library that transposes the traditional model-centric approach composed of pre-training + fine-tuning steps to the objective-centric approach, composing the training process by applications of selected objectives. We survey research directions that can benefit from enhanced objective-centric experimentation in multitask training, custom objectives development, dynamic training curricula, or domain adaptation. Adaptor aims to ease the reproducibility of these research directions in practice. Finally, we demonstrate the practical applicability of Adaptor in selected unsupervised domain adaptation scenarios.
ER  -

ŠTEFÁNIK, Michal, Vít NOVOTNÝ, Nikola GROVEROVÁ and Petr SOJKA. AdaptOr: Objective-Centric Adaptation Framework for Language Models. Online. In Valerio Basile, Zornitsa Kozareva, Sanja Stajner. \textit{Proceedings of the 60th Conference of Association of Computational Linguistics, ACL 2022}. Dublin, Irsko: Association for Computational Linguistics, ACL, 2022, p.~261-269. ISBN~978-1-955917-24-7. Available from: https://dx.doi.org/10.18653/v1/2022.acl-demo.26.

Detailed Information on Publication Record