headers
Mirjam Sepesy Maucec | 10 Jul 2012 08:52
Picon

MERT tunning

Hi all,

I'm running experiments on subtitle corpora. My training corpus is 
OpenSub and I have dev and test sets from another source.
Before parameter tunning BLEU score was 21%. After running mert BLEU 
score improves to 24%.
Translation model weights after mert were:

[weight-t]
3.17108e-06
7.03552e-06
4.10145e-05
7.17818e-07
-0.999893

Small weights for 4 translation model components and large negative 
score for phrase penalty.
But phrase penalty in phrase table is always exp(1).
Does it mean that my dev/test set do not have much in common with 
OpenSub corpus?
I trained lm on OpenSub as well. Dev set perplexity was 228 an test set 
perplexity was 217 (OOV=1%). After mert the weight for language model 
was 0.01.
What do you suggest?

Mirjam

--

-- 
======================================
Mirjam Sepesy Maučec
(Continue reading)

Permalink | Reply |
headers
Barry Haddow | 11 Jul 2012 13:51
Picon
Picon

Re: MERT tunning

Hi Mirjam

These weights look very wrong. mert is effectively saying that it prefers your 
translation model to produce no output.

I suspect that ther is some misconfiguration in your setup, for example 
swapping of source and target files. What do the translations in the nbest 
lists generated by mert look like?

cheers - Barry

On Tuesday 10 Jul 2012 07:52:55 Mirjam Sepesy Maucec wrote:
> Hi all,
> 
> I'm running experiments on subtitle corpora. My training corpus is
> OpenSub and I have dev and test sets from another source.
> Before parameter tunning BLEU score was 21%. After running mert BLEU
> score improves to 24%.
> Translation model weights after mert were:
> 
> [weight-t]
> 3.17108e-06
> 7.03552e-06
> 4.10145e-05
> 7.17818e-07
> -0.999893
> 
> Small weights for 4 translation model components and large negative
> score for phrase penalty.
> But phrase penalty in phrase table is always exp(1).
(Continue reading)

Permalink | Reply |

Gmane