English-French Document Alignment Based on Keywords and
Statistical Translation

D 2016

English-French Document Alignment Based on Keywords and Statistical Translation

MEDVEĎ, Marek, Vojtěch KOVÁŘ a Miloš JAKUBÍČEK

Základní údaje

Originální název

English-French Document Alignment Based on Keywords and Statistical Translation

Autoři

MEDVEĎ, Marek (703 Slovensko, garant, domácí), Vojtěch KOVÁŘ (203 Česká republika, domácí) a Miloš JAKUBÍČEK (203 Česká republika, domácí)

Vydání

Berlin, Proceedings of the First Conference on Machine Translation, Volume 2: Shared Task Papers, od s. 728-732, 5 s. 2016

Nakladatel

Association for Computational Linguistics

Další údaje

Jazyk

angličtina

Typ výsledku

Stať ve sborníku

Obor

10201 Computer sciences, information science, bioinformatics

Utajení

není předmětem státního či obchodního tajemství

Forma vydání

elektronická verze "online"

Kód RIV

RIV/00216224:14330/16:00088114

Organizační jednotka

Fakulta informatiky

ISBN

978-1-945626-10-4

Klíčová slova anglicky

bilingual document alignment

Změněno: 4. 9. 2016 22:05, RNDr. Miloš Jakubíček, Ph.D.

Anotace

V originále

In this paper we present our approach to the Bilingual Document Alignment Task (WMT16), where the main goal was to reach the best recall on extracting aligned pages within the provided data. Our approach consists of tree main parts: data preprocessing, keyword extraction and text pairs scoring based on keyword matching. For text preprocessing we use the TreeTagger pipeline that contains the Unitok tool (Michelfeit et al., 2014) for tokenization and the TreeTagger morphological analyzer (Schmid, 1994). After keywords extraction from the texts according TF-IDF scoring our system searches for comparable English-French pairs. Using a statistical dictionary created from a large English-French parallel corpus, the system is able to find comaparable documents. At the end this procedure is combined with the baseline algorithm and best one-to-one pairing is selected. The result reaches 91.6% recall on provided training data. After a deep error analysis (see section 5) the recall reached 97.4%.

Návaznosti

GA15-13277S, projekt VaV

Název: Hyperintensionální logika pro analýzu přirozeného jazyka

Investor: Grantová agentura ČR, Hyperintensionální logika pro analýzu přirozeného jazyka

LM2015071, projekt VaV

Název: Jazyková výzkumná infrastruktura v České republice (Akronym: LINDAT-Clarin)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, Projekt LINDAT-Clarin - Vybudování a provoz českého uzlu pan-evropské infrastruktury pro výzkum

7F14047, projekt VaV

Název: Harvesting big text data for under-resourced languages (Akronym: HaBiT)

Investor: Ministerstvo školství, mládeže a tělovýchovy ČR, Harvesting big text data for under-resourced languages

Citovat

MEDVEĎ, Marek, Vojtěch KOVÁŘ a Miloš JAKUBÍČEK. English-French Document Alignment Based on Keywords and Statistical Translation. Online. In Proceedings of the First Conference on Machine Translation, Volume 2: Shared Task Papers. Berlin: Association for Computational Linguistics, 2016, s. 728-732. ISBN 978-1-945626-10-4.

@inproceedings{1352922,
   author = {Medveď, Marek and Kovář, Vojtěch and Jakubíček, Miloš},
   address = {Berlin},
   booktitle = {Proceedings of the First Conference on Machine Translation, Volume 2: Shared Task Papers},
   keywords = {bilingual document alignment},
   howpublished = {elektronická verze "online"},
   language = {eng},
   location = {Berlin},
   isbn = {978-1-945626-10-4},
   pages = {728-732},
   publisher = {Association for Computational Linguistics},
   title = {English-French Document Alignment Based on Keywords and Statistical Translation},
   year = {2016}
}

TY  - JOUR
ID  - 1352922
AU  - Medveď, Marek - Kovář, Vojtěch - Jakubíček, Miloš
PY  - 2016
TI  - English-French Document Alignment Based on Keywords and Statistical Translation
PB  - Association for Computational Linguistics
CY  - Berlin
SN  - 9781945626104
KW  - bilingual document alignment
N2  - In this paper we present our approach to the Bilingual Document Alignment Task (WMT16), where the main goal was to reach the best recall on extracting aligned pages within the provided data. Our approach consists of tree main parts: data preprocessing, keyword extraction and text pairs scoring based on keyword matching. For text preprocessing we use the TreeTagger pipeline that contains the Unitok tool (Michelfeit et al., 2014) for tokenization and the TreeTagger morphological analyzer (Schmid, 1994). After keywords extraction from the texts according TF-IDF scoring our system searches for comparable English-French pairs. Using a statistical dictionary created from a large English-French parallel corpus, the system is able to find comaparable documents. At the end this procedure is combined with the baseline algorithm and best one-to-one pairing is selected. The result reaches 91.6% recall on provided training data. After a deep error analysis (see section 5) the recall reached 97.4%.
ER  -

MEDVEĎ, Marek, Vojtěch KOVÁŘ a Miloš JAKUBÍČEK. English-French Document Alignment Based on Keywords and Statistical Translation. Online. In \textit{Proceedings of the First Conference on Machine Translation, Volume 2: Shared Task Papers}. Berlin: Association for Computational Linguistics, 2016, s.~728-732. ISBN~978-1-945626-10-4.

Podrobný výpis o publikaci