People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts
Authors
NOVOTNÝ, Vít (203 Czech Republic, guarantor, belonging to the institution), Kristýna LUGER (203 Czech Republic, belonging to the institution), Michal ŠTEFÁNIK (703 Slovakia, belonging to the institution), Tereza VRABCOVÁ (203 Czech Republic, belonging to the institution) and Aleš HORÁK (203 Czech Republic, belonging to the institution)
Edition
Toronto, Canada, Findings of the Association for Computational Linguistics: ACL 2023, p. 14104-14113, 10 pp. 2023
Publisher
Association for Computational Linguistics
Other information
Language
English
Type of outcome
Stať ve sborníku
Field of Study
10201 Computer sciences, information science, bioinformatics
Although pre-trained named entity recognition (NER) models are highly accurate on modern corpora, they underperform on historical texts due to differences in language OCR errors. In this work, we develop a new NER corpus of 3.6M sentences from late medieval charters written mainly in Czech, Latin, and German.
We show that we can start with a list of known historical figures and locations and an unannotated corpus of historical texts, and use information retrieval techniques to automatically bootstrap a NER-annotated corpus. Using our corpus, we train a NER model that achieves entity-level Precision of 72.81-93.98% with 58.14-81.77% Recall on a manually-annotated test dataset. Furthermore, we show that using a weighted loss function helps to combat class imbalance in token classification tasks. To make it easy for others to reproduce and build upon our work, we publicly release our corpus, models, and experimental code.
Links
MUNI/A/1339/2022, interní kód MU
Name: Rozvoj technik pro zpracování dat pro podporu vyhledávání, analýz a vizualizací rozsáhlých datových souborů s využitím umělé inteligence
Investor: Masaryk University, Development of data processing techniques to support search, analysis and visualization of large datasets using artificial intelligence
NOVOTNÝ, Vít, Kristýna LUGER, Michal ŠTEFÁNIK, Tereza VRABCOVÁ and Aleš HORÁK. People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts. Online. In Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki. Findings of the Association for Computational Linguistics: ACL 2023. Toronto, Canada: Association for Computational Linguistics, 2023, p. 14104-14113. ISBN 978-1-959429-62-3.
@inproceedings{2289944, author = {Novotný, Vít and Luger, Kristýna and Štefánik, Michal and Vrabcová, Tereza and Horák, Aleš}, address = {Toronto, Canada}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2023}, editor = {Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki}, keywords = {natural language processing; nlp; historical documents; optical character recognition; ocr; named entity recognition; ner; czech; german; latin}, howpublished = {elektronická verze "online"}, language = {eng}, location = {Toronto, Canada}, isbn = {978-1-959429-62-3}, pages = {14104-14113}, publisher = {Association for Computational Linguistics}, title = {People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts}, url = {https://aclanthology.org/2023.findings-acl.887}, year = {2023} }
TY - JOUR ID - 2289944 AU - Novotný, Vít - Luger, Kristýna - Štefánik, Michal - Vrabcová, Tereza - Horák, Aleš PY - 2023 TI - People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts PB - Association for Computational Linguistics CY - Toronto, Canada SN - 9781959429623 KW - natural language processing KW - nlp KW - historical documents KW - optical character recognition KW - ocr KW - named entity recognition KW - ner KW - czech KW - german KW - latin UR - https://aclanthology.org/2023.findings-acl.887 N2 -
Although pre-trained named entity recognition (NER) models are highly accurate on modern corpora, they underperform on historical texts due to differences in language OCR errors. In this work, we develop a new NER corpus of 3.6M sentences from late medieval charters written mainly in Czech, Latin, and German.
We show that we can start with a list of known historical figures and locations and an unannotated corpus of historical texts, and use information retrieval techniques to automatically bootstrap a NER-annotated corpus. Using our corpus, we train a NER model that achieves entity-level Precision of 72.81-93.98% with 58.14-81.77% Recall on a manually-annotated test dataset. Furthermore, we show that using a weighted loss function helps to combat class imbalance in token classification tasks. To make it easy for others to reproduce and build upon our work, we publicly release our corpus, models, and experimental code.
ER -
NOVOTNÝ, Vít, Kristýna LUGER, Michal ŠTEFÁNIK, Tereza VRABCOVÁ and Aleš HORÁK. People and Places of Historical Europe: Bootstrapping Annotation Pipeline and a New Corpus of Named Entities in Late Medieval Texts. Online. In Anna Rogers, Jordan Boyd-Graber, Naoaki Okazaki. \textit{Findings of the Association for Computational Linguistics: ACL 2023}. Toronto, Canada: Association for Computational Linguistics, 2023, p.~14104-14113. ISBN~978-1-959429-62-3.