Text Punctuation: An Inter-annotator Agreement Study

D 2017

Text Punctuation: An Inter-annotator Agreement Study

BOHÁČ, Marek, Michal ROTT and Vojtěch KOVÁŘ

Basic information

Original name

Text Punctuation: An Inter-annotator Agreement Study

Authors

BOHÁČ, Marek (203 Czech Republic), Michal ROTT (203 Czech Republic) and Vojtěch KOVÁŘ (203 Czech Republic, guarantor, belonging to the institution)

Edition

Cham, Text, Speech, and Dialogue: 20th International Conference, TSD 2017, p. 120-128, 9 pp. 2017

Publisher

Springer International Publishing

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Czech Republic

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

printed version "print"

References:

URL

Impact factor

Impact factor: 0.402 in 2005

RIV identification code

RIV/00216224:14330/17:00095096

Organization unit

Faculty of Informatics

ISBN

978-3-319-64205-5

ISSN

DOI

http://dx.doi.org/10.1007/978-3-319-64206-2_14

UT WoS

000449869200014

Keywords (in Czech)

doplňování čárek;mluvený jazyk;mezianotátorská shoda

Keywords in English

Comma adding;Spoken language;Inter-annotator agreement

Abstract

ORIG CZ

V originále

Spoken language is a phenomenon which is hard to be annotated accurately. One of the most ambiguous tasks is to fill in the punctuation marks into the spoken language transcription. Used punctuation marks are often dependent on how annotators understand the transcription content. This may differ as the spoken language often lacks clear structure (inherent to written language) due to the utterance spontaneity or due to skipping between ideas. Therefore we suspect that filling commas into the spoken language transcription is a very ambiguous task with low inter-annotator agreement (IAA). In this paper we analyze the IAA within group of annotators and we propose methods to increase it. We also propose and evaluate a reformulation of classical GT annotations for cases with multiple annotations available.

In Czech

Článek se zabývá problematikou doplňování čárek do mluveného textu, zejména mezianotátorskou shodou a přesností současných počítačových programů.

Links

GA15-13277S, research and development project

Name: Hyperintensionální logika pro analýzu přirozeného jazyka

Investor: Czech Science Foundation

LM2015071, research and development project

Name: Jazyková výzkumná infrastruktura v České republice (Acronym: LINDAT-Clarin)

Investor: Ministry of Education, Youth and Sports of the CR

MUNI/A/0897/2016, interní kód MU

Name: Rozsáhlé výpočetní systémy: modely, aplikace a verifikace VI.

Investor: Masaryk University, Category A

Citovat

BOHÁČ, Marek, Michal ROTT and Vojtěch KOVÁŘ. Text Punctuation: An Inter-annotator Agreement Study. In Ekštein, Kamil Matoušek, Václav. Text, Speech, and Dialogue: 20th International Conference, TSD 2017. Cham: Springer International Publishing, 2017, p. 120-128. ISBN 978-3-319-64205-5. Available from: https://dx.doi.org/10.1007/978-3-319-64206-2_14.

@inproceedings{1393439,
   author = {Boháč, Marek and Rott, Michal and Kovář, Vojtěch},
   address = {Cham},
   booktitle = {Text, Speech, and Dialogue: 20th International Conference, TSD 2017},
   doi = {http://dx.doi.org/10.1007/978-3-319-64206-2_14},
   editor = {Ekštein, Kamil Matoušek, Václav},
   keywords = {Comma adding;Spoken language;Inter-annotator agreement},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Cham},
   isbn = {978-3-319-64205-5},
   pages = {120-128},
   publisher = {Springer International Publishing},
   title = {Text Punctuation: An Inter-annotator Agreement Study},
   url = {https://link.springer.com/chapter/10.1007/978-3-319-64206-2_14},
   year = {2017}
}

TY  - JOUR
ID  - 1393439
AU  - Boháč, Marek - Rott, Michal - Kovář, Vojtěch
PY  - 2017
TI  - Text Punctuation: An Inter-annotator Agreement Study
PB  - Springer International Publishing
CY  - Cham
SN  - 9783319642055
KW  - Comma adding;Spoken language;Inter-annotator agreement
UR  - https://link.springer.com/chapter/10.1007/978-3-319-64206-2_14
N2  - Spoken language is a phenomenon which is hard to be annotated accurately. One of the most ambiguous tasks is to fill in the punctuation marks into the spoken language transcription. Used punctuation marks are often dependent on how annotators understand the transcription content. This may differ as the spoken language often lacks clear structure (inherent to written language) due to the utterance spontaneity or due to skipping between ideas. Therefore we suspect that filling commas into the spoken language transcription is a very ambiguous task with low inter-annotator agreement (IAA). In this paper we analyze the IAA within group of annotators and we propose methods to increase it. We also propose and evaluate a reformulation of classical GT annotations for cases with multiple annotations available.
ER  -

BOHÁČ, Marek, Michal ROTT and Vojtěch KOVÁŘ. Text Punctuation: An Inter-annotator Agreement Study. In Ekštein, Kamil Matoušek, Václav. \textit{Text, Speech, and Dialogue: 20th International Conference, TSD 2017}. Cham: Springer International Publishing, 2017, p.~120-128. ISBN~978-3-319-64205-5. Available from: https://dx.doi.org/10.1007/978-3-319-64206-2\_{}14.

Detailed Information on Publication Record