Annotation of Czech Texts with Language Mixing

D 2016

Annotation of Czech Texts with Language Mixing

NEVĚŘILOVÁ, Zuzana

Základní údaje

Originální název

Annotation of Czech Texts with Language Mixing

Autoři

NEVĚŘILOVÁ, Zuzana (203 Česká republika, garant, domácí)

Vydání

Switzerland, Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings, od s. 279-286, 8 s. 2016

Nakladatel

Springer International Publishing

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Stát vydavatele

Švýcarsko

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

tištěná verze "print"

Impakt faktor

Impact factor: 0.402 v roce 2005

Kód RIV

RIV/00216224:14330/16:00091344

Organizační jednotka

Fakulta informatiky

ISBN

978-3-319-45509-9

ISSN

DOI

http://dx.doi.org/10.1007/978-3-319-45510-5_32

UT WoS

000389707400032

Klíčová slova anglicky

language mixing; multi-word expression detection

Štítky

firank_B

Příznaky

Mezinárodní význam, Recenzováno

Změněno: 26. 4. 2018 08:09, RNDr. Zuzana Nevěřilová, Ph.D.

Anotace

V originále

Language mixing (using chunks of foreign language in a native language utterance) occurs frequently. Foreign language chunks have to be detected because their annotation is often incorrect. In the standard pipelines of Czech texts annotation, no such detection exists. Before morphological disambiguation, unrecognized words are processed by Czech guesser which is successful on Czech words (e.g. neologisms, typos) but its usage makes no sense on foreign words. We propose a new pipeline that adds foreign language chunk and multi-word expression (MWE) detection. We experimented with a small corpus where we compared the original (semi-automatic) annotation (including foreign words and MWEs) with the results of the new pipelines. As a result, we reduced the number of incorrect annotations of interlingual homographs and foreign language chunks in the new pipeline compared to the standard one. We also reduced the number of tokens that have to be processed by the guesser. The aim was to use the guesser solely on potentially Czech words.

Návaznosti

LD15066, projekt VaV

Název: Rozhraní pro Linked Data v systému pro editaci slovníků DEB (Akronym: DEB LDI)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, Rozhraní pro Linked Data v systému pro editaci slovníků DEB

7F14047, projekt VaV

Název: Harvesting big text data for under-resourced languages (Akronym: HaBiT)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, Harvesting big text data for under-resourced languages

Citovat

NEVĚŘILOVÁ, Zuzana. Annotation of Czech Texts with Language Mixing. In Petr Sojka; Aleš Horák; Ivan Kopeček; Karel Pala. Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings. Switzerland: Springer International Publishing, 2016, s. 279-286. ISBN 978-3-319-45509-9. Dostupné z: https://dx.doi.org/10.1007/978-3-319-45510-5_32.

@inproceedings{1358121,
   author = {Nevěřilová, Zuzana},
   address = {Switzerland},
   booktitle = {Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings},
   doi = {http://dx.doi.org/10.1007/978-3-319-45510-5_32},
   editor = {Petr Sojka; Aleš Horák; Ivan Kopeček; Karel Pala},
   keywords = {language mixing; multi-word expression detection},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Switzerland},
   isbn = {978-3-319-45509-9},
   pages = {279-286},
   publisher = {Springer International Publishing},
   title = {Annotation of Czech Texts with Language Mixing},
   year = {2016}
}

TY  - JOUR
ID  - 1358121
AU  - Nevěřilová, Zuzana
PY  - 2016
TI  - Annotation of Czech Texts with Language Mixing
PB  - Springer International Publishing
CY  - Switzerland
SN  - 9783319455099
KW  - language mixing
KW  - multi-word expression detection
N2  - Language mixing (using chunks of foreign language in a native language utterance) occurs frequently. Foreign language chunks have to be detected because their annotation is often incorrect. In the standard pipelines of Czech texts annotation, no such detection exists. Before morphological disambiguation, unrecognized words are processed by Czech guesser which is successful on Czech words (e.g. neologisms, typos) but its usage makes no sense on foreign words. We propose a new pipeline that adds foreign language chunk and multi-word expression (MWE) detection. We experimented with a small corpus where we compared the original (semi-automatic) annotation (including foreign words and MWEs) with the results of the new pipelines. As a result, we reduced the number of incorrect annotations of interlingual homographs and foreign language chunks in the new pipeline compared to the standard one. We also reduced the number of tokens that have to be processed by the guesser. The aim was to use the guesser solely on potentially Czech words.
ER  -

NEVĚŘILOVÁ, Zuzana. Annotation of Czech Texts with Language Mixing. In Petr Sojka; Aleš Horák; Ivan Kopeček; Karel Pala. \textit{Text, Speech, and Dialogue 19th International Conference, TSD 2016 Brno, Czech Republic, September 12–16, 2016 Proceedings}. Switzerland: Springer International Publishing, 2016, s.~279-286. ISBN~978-3-319-45509-9. Dostupné z: https://dx.doi.org/10.1007/978-3-319-45510-5\_{}32.

Podrobný výpis o publikaci