Context Sensitive Pattern Based Segmentation: A Thai Challenge

D 2003

Context Sensitive Pattern Based Segmentation: A Thai Challenge

SOJKA, Petr and David ANTOŠ

Basic information

Original name

Context Sensitive Pattern Based Segmentation: A Thai Challenge

Authors

SOJKA, Petr and David ANTOŠ

Edition

Budapest, Proceedings of EACL 2003 workshop Computational Linguistics for South Asian Languages -- Expanding Synergies with Europe, p. 65-72, 8 pp. 2003

Publisher

Association for Computational Linguistics

Other information

Language

English

Type of outcome

Proceedings paper

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Hungary

Confidentiality degree

is not subject to a state or trade secret

Publication form

printed version "print"

References:

URL, URL of Proceedings, paper URL

Marked to be transferred to RIV

Yes

RIV identification code

RIV/00216224:14330/03:00008605

Organization unit

Faculty of Informatics

ISBN

1-932432-02-7

Keywords in English

segmentation Thai competing patterns; patgen

Abstract

In the original language

A Thai written text is a string of symbols without explicit word boundaries. A method for a development of a segmentation tool from a corpus of already segmented text is described. The methodology is based on the technology of competing patterns. A new UNICODE pattern generation program, OPATGEN, is used for the learning phase. We have shown feasibility of our methodology by generating patterns for Thai segmentation from already segmented text of the Thai corpus ORCHID: the segmentation algorithm quickly reaches F-score of 93 %. Finally, we enumerate possible new applications based on the pattern technique, and conclude with the suggestion of a general Pattern Translation Process. The technology is general and can be used for any other segmentation tasks as phonetic, morphologic segmentation, word hyphenation, sentence segmentation and text topic segmentation for any language.

Links

MSM 143300003, plan (intention)

Name: Interakce člověka s počítačem, dialogové systémy a asistivní technologie

Investor: Ministry of Education, Youth and Sports of the CR, Human-computer interaction, dialog systems and assistive technologies

Files attached

sojka-antos-eacl-thai-2003.pdf

w08.pdf

Request the author's version of the file

Cite

SOJKA, Petr and David ANTOŠ. Context Sensitive Pattern Based Segmentation: A Thai Challenge. In Pat Hall, Durgesh Rao. Proceedings of EACL 2003 workshop Computational Linguistics for South Asian Languages -- Expanding Synergies with Europe. Budapest: Association for Computational Linguistics, 2003, p. 65-72. ISBN 1-932432-02-7.

@inproceedings{484652,
   author = {Sojka, Petr and Antoš, David},
   address = {Budapest},
   booktitle = {Proceedings of EACL 2003 workshop Computational Linguistics for South Asian Languages -- Expanding Synergies with Europe},
   editor = {Pat Hall, Durgesh Rao},
   keywords = {segmentation Thai competing patterns; patgen},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Budapest},
   isbn = {1-932432-02-7},
   note = {EACL 2003 conference paper},
   pages = {65-72},
   publisher = {Association for Computational Linguistics},
   title = {Context Sensitive Pattern Based Segmentation: A Thai Challenge},
   url = {https://www.researchgate.net/publication/2868683_Context_Sensitive_Pattern_Based_Segmentation_A_Thai_Challenge},
   year = {2003}
}

TY  - CONF
ID  - 484652
AU  - Sojka, Petr - Antoš, David
PY  - 2003
TI  - Context Sensitive Pattern Based Segmentation: A Thai Challenge
PB  - Association for Computational Linguistics
CY  - Budapest
SN  - 1932432027
N1  - EACL 2003 conference paper
KW  - segmentation Thai competing patterns
KW  - patgen
UR  - https://www.researchgate.net/publication/2868683_Context_Sensitive_Pattern_Based_Segmentation_A_Thai_Challenge
N2  - A Thai written text is a string of symbols without explicit word boundaries. A method for a development of a segmentation tool from a corpus of already segmented text is described. The methodology is based on the technology of competing patterns. A new UNICODE pattern generation program, OPATGEN, is used for the learning phase. We have shown feasibility of our methodology by generating patterns for Thai segmentation from already segmented text of the Thai corpus ORCHID: the segmentation algorithm quickly reaches F-score of 93 %. Finally, we enumerate possible new applications based on the pattern technique, and conclude with the suggestion of a general Pattern Translation Process. The technology is general and can be used for any other segmentation tasks as phonetic, morphologic segmentation, word hyphenation, sentence segmentation and text topic segmentation for any language.
ER  -

SOJKA, Petr and David ANTOŠ. Context Sensitive Pattern Based Segmentation: A Thai Challenge. In Pat Hall, Durgesh Rao. \textit{Proceedings of EACL 2003 workshop Computational Linguistics for South Asian Languages -- Expanding Synergies with Europe}. Budapest: Association for Computational Linguistics, 2003, p.~65-72. ISBN~1-932432-02-7.

Přehled o publikaci