Annotation of Czech Texts with Language Mixing

D 2016

Annotation of Czech Texts with Language Mixing

NEVĚŘILOVÁ, Zuzana

Basic information

Original name

Annotation of Czech Texts with Language Mixing

Authors

NEVĚŘILOVÁ, Zuzana (203 Czech Republic, guarantor, belonging to the institution)

Edition

Switzerland, Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings, p. 279-286, 8 pp. 2016

Publisher

Springer International Publishing

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Switzerland

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

printed version "print"

Impact factor

Impact factor: 0.402 in 2005

RIV identification code

RIV/00216224:14330/16:00091344

Organization unit

Faculty of Informatics

ISBN

978-3-319-45509-9

ISSN

DOI

http://dx.doi.org/10.1007/978-3-319-45510-5_32

UT WoS

000389707400032

Keywords in English

language mixing; multi-word expression detection

Abstract

V originále

Language mixing (using chunks of foreign language in a native language utterance) occurs frequently. Foreign language chunks have to be detected because their annotation is often incorrect. In the standard pipelines of Czech texts annotation, no such detection exists. Before morphological disambiguation, unrecognized words are processed by Czech guesser which is successful on Czech words (e.g. neologisms, typos) but its usage makes no sense on foreign words. We propose a new pipeline that adds foreign language chunk and multi-word expression (MWE) detection. We experimented with a small corpus where we compared the original (semi-automatic) annotation (including foreign words and MWEs) with the results of the new pipelines. As a result, we reduced the number of incorrect annotations of interlingual homographs and foreign language chunks in the new pipeline compared to the standard one. We also reduced the number of tokens that have to be processed by the guesser. The aim was to use the guesser solely on potentially Czech words.

Links

LD15066, research and development project

Name: Rozhraní pro Linked Data v systému pro editaci slovníků DEB (Acronym: DEB LDI)

Investor: Ministry of Education, Youth and Sports of the CR

7F14047, research and development project

Name: Harvesting big text data for under-resourced languages (Acronym: HaBiT)

Investor: Ministry of Education, Youth and Sports of the CR

Citovat

NEVĚŘILOVÁ, Zuzana. Annotation of Czech Texts with Language Mixing. In Petr Sojka; Aleš Horák; Ivan Kopeček; Karel Pala. Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings. Switzerland: Springer International Publishing, 2016, p. 279-286. ISBN 978-3-319-45509-9. Available from: https://dx.doi.org/10.1007/978-3-319-45510-5_32.

@inproceedings{1358121,
   author = {Nevěřilová, Zuzana},
   address = {Switzerland},
   booktitle = {Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings},
   doi = {http://dx.doi.org/10.1007/978-3-319-45510-5_32},
   editor = {Petr Sojka; Aleš Horák; Ivan Kopeček; Karel Pala},
   keywords = {language mixing; multi-word expression detection},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Switzerland},
   isbn = {978-3-319-45509-9},
   pages = {279-286},
   publisher = {Springer International Publishing},
   title = {Annotation of Czech Texts with Language Mixing},
   year = {2016}
}

TY  - JOUR
ID  - 1358121
AU  - Nevěřilová, Zuzana
PY  - 2016
TI  - Annotation of Czech Texts with Language Mixing
PB  - Springer International Publishing
CY  - Switzerland
SN  - 9783319455099
KW  - language mixing
KW  - multi-word expression detection
N2  - Language mixing (using chunks of foreign language in a native language utterance) occurs frequently. Foreign language chunks have to be detected because their annotation is often incorrect. In the standard pipelines of Czech texts annotation, no such detection exists. Before morphological disambiguation, unrecognized words are processed by Czech guesser which is successful on Czech words (e.g. neologisms, typos) but its usage makes no sense on foreign words. We propose a new pipeline that adds foreign language chunk and multi-word expression (MWE) detection. We experimented with a small corpus where we compared the original (semi-automatic) annotation (including foreign words and MWEs) with the results of the new pipelines. As a result, we reduced the number of incorrect annotations of interlingual homographs and foreign language chunks in the new pipeline compared to the standard one. We also reduced the number of tokens that have to be processed by the guesser. The aim was to use the guesser solely on potentially Czech words.
ER  -

NEVĚŘILOVÁ, Zuzana. Annotation of Czech Texts with Language Mixing. In Petr Sojka; Aleš Horák; Ivan Kopeček; Karel Pala. \textit{Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings}. Switzerland: Springer International Publishing, 2016, p.~279-286. ISBN~978-3-319-45509-9. Available from: https://dx.doi.org/10.1007/978-3-319-45510-5\_{}32.

Detailed Information on Publication Record