Informatica logo


Login Register

  1. Home
  2. Issues
  3. Volume 22, Issue 2 (2011)
  4. Semi-Automatic Bilingual Corpus Creation ...

Informatica

Information Submit your article For Referees Help ATTENTION!
  • Article info
  • Related articles
  • Cited by
  • More
    Article info Related articles Cited by

Semi-Automatic Bilingual Corpus Creation with Zero Entropy Alignments
Volume 22, Issue 2 (2011), pp. 203–224
Algirdas Laukaitis   Olegas Vasilecas   Ricardas Laukaitis   Darius Plikynas  

Authors

 
Placeholder
https://doi.org/10.15388/Informatica.2011.323
Pub. online: 1 January 2011      Type: Research Article     

Received
1 October 2010
Accepted
1 April 2011
Published
1 January 2011

Abstract

In this paper, we describe a model for aligning books and documents from bilingual corpus with a goal to create “perfectly” aligned bilingual corpus on word-to-word level. Presented algorithms differ from existing algorithms in consideration of the presence of human translator which usage we are trying to minimize. We treat human translator as an oracle who knows exact alignments and the goal of the system is to optimize (minimize) the use of this oracle. The effectiveness of the oracle is measured by the speed at which he can create “perfectly” aligned bilingual corpus. By “Perfectly” aligned corpus we mean zero entropy corpus because oracle can make alignments without any probabilistic interpretation, i.e., with 100% confidence. Sentence level alignments and word-to-word alignments, although treated separately in this paper, are integrated in a single framework. For sentence level alignments we provide a dynamic programming algorithm which achieves low precision and recall error rate. For word-to-word level alignments Expectation Maximization algorithm that integrates linguistic dictionaries is suggested as the main tool for the oracle to build “perfectly” aligned bilingual corpus. We show empirically that suggested pre-aligned corpus requires little interaction from the oracle and that creation of perfectly aligned corpus can be achieved almost with the speed of human reading. Presented algorithms are language independent but in this paper we verify them with English–Lithuanian language pair on two types of text: law documents and fiction literature.

Related articles Cited by PDF XML
Related articles Cited by PDF XML

Copyright
No copyright data available.

Keywords
Viterbi alignments dynamic programming string alignments machine translation natural language processing rapid development low-density languages

Metrics
since January 2020
833

Article info
views

0

Full article
views

601

PDF
downloads

186

XML
downloads

Export citation

Copy and paste formatted citation
Placeholder

Download citation in file


Share


RSS

INFORMATICA

  • Online ISSN: 1822-8844
  • Print ISSN: 0868-4952
  • Copyright © 2023 Vilnius University

About

  • About journal

For contributors

  • OA Policy
  • Submit your article
  • Instructions for Referees
    •  

    •  

Contact us

  • Institute of Data Science and Digital Technologies
  • Vilnius University

    Akademijos St. 4

    08412 Vilnius, Lithuania

    Phone: (+370 5) 2109 338

    E-mail: informatica@mii.vu.lt

    https://informatica.vu.lt/journal/INFORMATICA
Powered by PubliMill  •  Privacy policy