Home arrow Products arrow Evaluation


Show Cart
Your Cart is currently empty.

Comparison of MetaMorpho with On-Line MT-Systems

MT systems translation between different languages can be compared. This can be done by Bleu-evaluation accepted widely by the MT community. It is done by the following steps. First they translate the text with three human translators. Then the result of the machine translation is compared to these translations. The comparison is done by paragraphs. They take the 1, 2, 3 and 4 long word sequences in the MT-result and count how many such sequence can be found in any of the human translations. Read more about Bleu: here.
In this evaluation we comapred our English-Hungarian translator to on-line English-German translators.
Date of the test is 17th March 2005.

MT Systems Compared

Systran babelfish.altavista.digital.com English-German
SDL www.freetranslation.com English-German
PROMT translation2.paralink.com English-German
WordLingo www.worldlingo.com English-German
MetaMorpho www.webforditas.hu English-Hungarian

Source, Reference and Result Texts

The English source text
The three human German translations: en-de_h1, en-de_h2, en-de_h3
The three human Hungarian translations: en-hu_h1, en-hu_h2, en-hu_h3
The translations of the English-German MT systems: Systran, SDL, PROMT, WordLingo,
The translation of the English-Hungarian MT system: MetaMorpho


The resulting Bleu-score is between 0 and 1. The number for MT systems is generally is between 0,2-0,5.

Systran SDL PROMT Wordlingo avarage of English-German MT systems MetaMorpho
Bleu-score 0,4206 0,4138 0,4616 0,4158 0,4280 0,4101
longest match 10 words 7 words 8 szó 10 words 8,75 words 12 words

The English-Hungarian MetaMorpho Bleu-score is only a little bit lower than the results of the English-German MT systems. MetaMorpho produced the longest (12 word long) match with one of the human translations.