Comparison of MetaMorpho with On-Line MT-Systems
MT systems translation between different languages can be compared. This can be done by Bleu-evaluation accepted widely by the MT community. It is done by the following steps. First they translate the text with three human translators. Then the result of the machine translation is compared to these translations. The comparison is done by paragraphs. They take the 1, 2, 3 and 4 long word sequences in the MT-result and count how many such sequence can be found in any of the human translations. Read more about Bleu: here.
In this evaluation we comapred our English-Hungarian translator to on-line English-German translators.
Date of the test is 17th March 2005.
MT Systems Compared
Source, Reference and Result Texts
The English source text
The three human German translations: en-de_h1, en-de_h2, en-de_h3
The three human Hungarian translations: en-hu_h1, en-hu_h2, en-hu_h3
The translations of the English-German MT systems: Systran, SDL, PROMT, WordLingo,
The translation of the English-Hungarian MT system: MetaMorpho
Results
The resulting Bleu-score is between 0 and 1. The number for MT systems is generally is between 0,2-0,5.
| |
Systran |
SDL |
PROMT |
Wordlingo |
avarage of English-German MT systems |
MetaMorpho
English-Hungarian |
| Bleu-score |
0,4206 |
0,4138 |
0,4616 |
0,4158 |
0,4280 |
0,4101 |
| longest match |
10 words |
7 words |
8 szó |
10 words |
8,75 words |
12 words |
The English-Hungarian MetaMorpho Bleu-score is only a little bit lower than the results of the English-German MT systems. MetaMorpho produced the longest (12 word long) match with one of the human translations.
|
Products 
