Annotation Game for Textual Entailment Evaluation

D 2014

Annotation Game for Textual Entailment Evaluation

NEVĚŘILOVÁ, Zuzana

Basic information

Original name

Annotation Game for Textual Entailment Evaluation

Authors

NEVĚŘILOVÁ, Zuzana (203 Czech Republic, guarantor, belonging to the institution)

Edition

Heidelberg, 15th International Conference, CICLing 2014, Kathmandu, Nepal, April 6-12, 2014, Proceedings, Part I, p. 340-350, 11 pp. 2014

Publisher

Springer

Other information

Language

English

Type of outcome

Stať ve sborníku

Field of Study

10201 Computer sciences, information science, bioinformatics

Country of publisher

Germany

Confidentiality degree

není předmětem státního či obchodního tajemství

Publication form

printed version "print"

Impact factor

Impact factor: 0.402 in 2005

RIV identification code

RIV/00216224:14330/14:00073212

Organization unit

Faculty of Informatics

ISBN

978-3-642-54905-2

ISSN

DOI

http://dx.doi.org/10.1007/978-3-642-54906-9_28

UT WoS

000342989200028

Keywords in English

annotation game; paraphrase; textual entailment; natural language generation

Abstract

V originále

Recognizing textual entailment (RTE) is a well-defined task concerning semantic analysis. It is evaluated against manually annotated collection of pairs hypothesis-text. A pair is annotated true if the text entails the hypothesis and false otherwise. Such collection can be used for training or testing a RTE application only if it is large enough. We present a game which purpose is to collect h-t pairs. It follows a detective story narrative pattern: a brilliant detective and his slower assistant talk about the riddle to reveal the solution to readers. In the game the detective (human player) provides a short story. The assistant (the application) proposes hypotheses the detective judges true, false or non-sense. Hypothesis generation is a rule-based process but the most likely hypotheses that are offered for annotation are calculated from a language model. During generation individual sentence constituents are rearranged to produce syntactically correct sentences. The game is intended to collect data in the Czech language. However, the idea can be applied for other languages. The paper concentrates on description of the most interesting modules from a language-independent point of view as well as the game elements.

Links

LM2010013, research and development project

Name: LINDAT-CLARIN: Institut pro analýzu, zpracování a distribuci lingvistických dat (Acronym: LINDAT-Clarin)

Investor: Ministry of Education, Youth and Sports of the CR

MUNI/A/0765/2013, interní kód MU

Name: Zapojení studentů Fakulty informatiky do mezinárodní vědecké komunity (Acronym: SKOMU)

Investor: Masaryk University, Category A

VF20102014003, research and development project

Name: Analýza přirozeného jazyka v prostředí internetu (Acronym: APJI)

Investor: Ministry of the Interior of the CR

Citovat

NEVĚŘILOVÁ, Zuzana. Annotation Game for Textual Entailment Evaluation. In Alexander F. Gelbukh. 15th International Conference, CICLing 2014, Kathmandu, Nepal, April 6-12, 2014, Proceedings, Part I. Heidelberg: Springer, 2014, p. 340-350. ISBN 978-3-642-54905-2. Available from: https://dx.doi.org/10.1007/978-3-642-54906-9_28.

@inproceedings{1164750,
   author = {Nevěřilová, Zuzana},
   address = {Heidelberg},
   booktitle = {15th International Conference, CICLing 2014, Kathmandu, Nepal, April 6-12, 2014, Proceedings, Part I},
   doi = {http://dx.doi.org/10.1007/978-3-642-54906-9_28},
   editor = {Alexander F. Gelbukh},
   keywords = {annotation game; paraphrase; textual entailment; natural language generation},
   howpublished = {tištěná verze "print"},
   language = {eng},
   location = {Heidelberg},
   isbn = {978-3-642-54905-2},
   pages = {340-350},
   publisher = {Springer},
   title = {Annotation Game for Textual Entailment Evaluation},
   year = {2014}
}

TY  - JOUR
ID  - 1164750
AU  - Nevěřilová, Zuzana
PY  - 2014
TI  - Annotation Game for Textual Entailment Evaluation
PB  - Springer
CY  - Heidelberg
SN  - 9783642549052
KW  - annotation game
KW  - paraphrase
KW  - textual entailment
KW  - natural language generation
N2  - Recognizing textual entailment (RTE) is a well-defined task concerning semantic analysis. It is evaluated against manually annotated collection of pairs hypothesis-text. A pair is annotated true if the text entails the hypothesis and false otherwise. Such collection can be used for training or testing a RTE application only if it is large enough. We present a game which purpose is to collect h-t pairs. It follows a detective story narrative pattern: a brilliant detective and his slower assistant talk about the riddle to reveal the solution to readers. In the game the detective (human player) provides a short story. The assistant (the application) proposes hypotheses the detective judges true, false or non-sense. Hypothesis generation is a rule-based process but the most likely hypotheses that are offered for annotation are calculated from a language model. During generation individual sentence constituents are rearranged to produce syntactically correct sentences. The game is intended to collect data in the Czech language. However, the idea can be applied for other languages. The paper concentrates on description of the most interesting modules from a language-independent point of view as well as the game elements.
ER  -

NEVĚŘILOVÁ, Zuzana. Annotation Game for Textual Entailment Evaluation. In Alexander F. Gelbukh. \textit{15th International Conference, CICLing 2014, Kathmandu, Nepal, April 6-12, 2014, Proceedings, Part I}. Heidelberg: Springer, 2014, p.~340-350. ISBN~978-3-642-54905-2. Available from: https://dx.doi.org/10.1007/978-3-642-54906-9\_{}28.

Detailed Information on Publication Record