KOVÁŘ, Vojtěch, Pavel RYCHLÝ and Miloš JAKUBÍČEK. Low Inter-Annotator Agreement = An Ill-Defined Problem? In Eighth Workshop on Recent Advances in Slavonic Natural Language Processing. Brno: Tribun EU, 2014, p. 57-62. ISSN 2336-4289.
Other formats:   BibTeX LaTeX RIS
Basic information
Original name Low Inter-Annotator Agreement = An Ill-Defined Problem?
Authors KOVÁŘ, Vojtěch (203 Czech Republic, guarantor, belonging to the institution), Pavel RYCHLÝ (203 Czech Republic, belonging to the institution) and Miloš JAKUBÍČEK (203 Czech Republic, belonging to the institution).
Edition Brno, Eighth Workshop on Recent Advances in Slavonic Natural Language Processing, p. 57-62, 6 pp. 2014.
Publisher Tribun EU
Other information
Original language English
Type of outcome Proceedings paper
Field of Study 10201 Computer sciences, information science, bioinformatics
Country of publisher Czech Republic
Confidentiality degree is not subject to a state or trade secret
Publication form printed version "print"
WWW URL
RIV identification code RIV/00216224:14330/14:00077512
Organization unit Faculty of Informatics
ISSN 2336-4289
UT WoS 000374560500007
Keywords in English NLP; inter-annotator agreement; low inter-annotator agreement; evaluation
Tags International impact, Reviewed
Changed by Changed by: doc. Mgr. Pavel Rychlý, Ph.D., učo 3692. Changed: 7/6/2021 17:27.
Abstract
nnotation tasks where the inter-annotator agreement is low are usually considered ill-defined and not worth attention. Such tasks are also considered unsuitable for algorithmic solution and for evaluation of computer programs that aim at solving them. However, there is a lot of problems (not only) in the natural language processing field that are practically defined and do have this nature, and we need computer programs that are able to solve them. The paper illustrates such problems on particular examples and suggests methodology that will enable training and evaluating tools using data with low inter-annotator agreement.
Links
LM2010013, research and development projectName: LINDAT-CLARIN: Institut pro analýzu, zpracování a distribuci lingvistických dat (Acronym: LINDAT-Clarin)
Investor: Ministry of Education, Youth and Sports of the CR
7F14047, research and development projectName: Harvesting big text data for under-resourced languages (Acronym: HaBiT)
Investor: Ministry of Education, Youth and Sports of the CR
PrintDisplayed: 26/7/2024 07:30