Skip to content

Latest commit

 

History

History
 
 

gmw-eng

README.md

opus-2020-06-27.zip

  • dataset: opus
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-06-27.zip
  • test set translations: opus-2020-06-27.test.txt
  • test set scores: opus-2020-06-27.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 26.2 0.535
news-test2008-deueng.deu.eng 25.1 0.528
newstest2009-deueng.deu.eng 24.4 0.524
newstest2010-deueng.deu.eng 27.4 0.560
newstest2011-deueng.deu.eng 24.7 0.532
newstest2012-deueng.deu.eng 26.0 0.542
newstest2013-deueng.deu.eng 28.9 0.559
newstest2014-deen-deueng.deu.eng 29.5 0.566
newstest2015-ende-deueng.deu.eng 30.6 0.570
newstest2016-ende-deueng.deu.eng 35.7 0.614
newstest2017-ende-deueng.deu.eng 31.5 0.578
newstest2018-ende-deueng.deu.eng 38.7 0.631
newstest2019-deen-deueng.deu.eng 35.1 0.602
Tatoeba-test.afr.eng 46.8 0.633
Tatoeba-test.afr-eng.afr.eng 61.1 0.744
Tatoeba-test.ang-eng.ang.eng 6.4 0.156
Tatoeba-test.deu-eng.deu.eng 47.1 0.648
Tatoeba-test.enm-eng.enm.eng 15.8 0.431
Tatoeba-test.frr-eng.frr.eng 14.1 0.224
Tatoeba-test.fry-eng.fry.eng 24.6 0.457
Tatoeba-test.gos-eng.gos.eng 13.4 0.301
Tatoeba-test.gsw-eng.gsw.eng 12.5 0.296
Tatoeba-test.ksh-eng.ksh.eng 8.2 0.242
Tatoeba-test.ltz-eng.ltz.eng 28.6 0.438
Tatoeba-test.multi.eng 46.8 0.633
Tatoeba-test.nds-eng.nds.eng 29.7 0.490
Tatoeba-test.nld-eng.nld.eng 56.9 0.720
Tatoeba-test.pdc-eng.pdc.eng 26.5 0.422
Tatoeba-test.sco-eng.sco.eng 31.3 0.514
Tatoeba-test.stq-eng.stq.eng 20.8 0.406
Tatoeba-test.swg-eng.swg.eng 12.2 0.294
Tatoeba-test.yid-eng.yid.eng 14.3 0.348

opus-2020-07-26.zip

  • dataset: opus
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-07-26.zip
  • test set translations: opus-2020-07-26.test.txt
  • test set scores: opus-2020-07-26.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 26.7 0.536
news-test2008-deueng.deu.eng 25.2 0.529
newstest2009-deueng.deu.eng 24.5 0.525
newstest2010-deueng.deu.eng 27.2 0.559
newstest2011-deueng.deu.eng 24.9 0.534
newstest2012-deueng.deu.eng 25.9 0.543
newstest2013-deueng.deu.eng 28.8 0.558
newstest2014-deen-deueng.deu.eng 29.5 0.565
newstest2015-ende-deueng.deu.eng 30.6 0.570
newstest2016-ende-deueng.deu.eng 36.1 0.617
newstest2017-ende-deueng.deu.eng 31.7 0.578
newstest2018-ende-deueng.deu.eng 38.8 0.630
newstest2019-deen-deueng.deu.eng 34.9 0.601
Tatoeba-test.afr-eng.afr.eng 61.3 0.748
Tatoeba-test.ang-eng.ang.eng 6.4 0.203
Tatoeba-test.deu-eng.deu.eng 47.5 0.649
Tatoeba-test.enm-eng.enm.eng 15.9 0.426
Tatoeba-test.frr-eng.frr.eng 6.1 0.073
Tatoeba-test.fry-eng.fry.eng 25.5 0.468
Tatoeba-test.gos-eng.gos.eng 12.5 0.308
Tatoeba-test.gsw-eng.gsw.eng 8.5 0.279
Tatoeba-test.ksh-eng.ksh.eng 8.4 0.261
Tatoeba-test.ltz-eng.ltz.eng 27.3 0.437
Tatoeba-test.multi.eng 47.5 0.637
Tatoeba-test.nds-eng.nds.eng 30.1 0.494
Tatoeba-test.nld-eng.nld.eng 57.5 0.723
Tatoeba-test.pdc-eng.pdc.eng 28.9 0.427
Tatoeba-test.sco-eng.sco.eng 45.3 0.602
Tatoeba-test.stq-eng.stq.eng 20.1 0.385
Tatoeba-test.swg-eng.swg.eng 13.8 0.284
Tatoeba-test.yid-eng.yid.eng 14.2 0.341

opus2m-2020-08-01.zip

  • dataset: opus2m
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus2m-2020-08-01.zip
  • test set translations: opus2m-2020-08-01.test.txt
  • test set scores: opus2m-2020-08-01.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 27.2 0.538
news-test2008-deueng.deu.eng 25.7 0.534
newstest2009-deueng.deu.eng 25.1 0.530
newstest2010-deueng.deu.eng 27.9 0.565
newstest2011-deueng.deu.eng 25.3 0.539
newstest2012-deueng.deu.eng 26.6 0.548
newstest2013-deueng.deu.eng 29.6 0.565
newstest2014-deen-deueng.deu.eng 30.2 0.571
newstest2015-ende-deueng.deu.eng 31.5 0.577
newstest2016-ende-deueng.deu.eng 36.7 0.622
newstest2017-ende-deueng.deu.eng 32.3 0.585
newstest2018-ende-deueng.deu.eng 39.9 0.638
newstest2019-deen-deueng.deu.eng 35.9 0.611
Tatoeba-test.afr-eng.afr.eng 61.8 0.750
Tatoeba-test.ang-eng.ang.eng 7.3 0.220
Tatoeba-test.deu-eng.deu.eng 48.3 0.657
Tatoeba-test.enm-eng.enm.eng 16.1 0.423
Tatoeba-test.frr-eng.frr.eng 7.0 0.168
Tatoeba-test.fry-eng.fry.eng 28.6 0.488
Tatoeba-test.gos-eng.gos.eng 15.5 0.326
Tatoeba-test.gsw-eng.gsw.eng 12.7 0.308
Tatoeba-test.ksh-eng.ksh.eng 8.4 0.254
Tatoeba-test.ltz-eng.ltz.eng 28.7 0.453
Tatoeba-test.multi.eng 48.5 0.646
Tatoeba-test.nds-eng.nds.eng 31.4 0.509
Tatoeba-test.nld-eng.nld.eng 58.1 0.728
Tatoeba-test.pdc-eng.pdc.eng 25.1 0.406
Tatoeba-test.sco-eng.sco.eng 40.8 0.570
Tatoeba-test.stq-eng.stq.eng 20.3 0.380
Tatoeba-test.swg-eng.swg.eng 20.5 0.315
Tatoeba-test.yid-eng.yid.eng 16.0 0.366

opus4m-2020-08-12.zip

  • dataset: opus4m
  • model: transformer
  • source language(s): afr ang_Latn deu enm_Latn frr fry gos gsw ksh ltz nds nld pdc sco stq swg yid
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus4m-2020-08-12.zip
  • test set translations: opus4m-2020-08-12.test.txt
  • test set scores: opus4m-2020-08-12.eval.txt

Benchmarks

testset BLEU chr-F
newssyscomb2009-deueng.deu.eng 27.4 0.541
news-test2008-deueng.deu.eng 25.8 0.534
newstest2009-deueng.deu.eng 25.5 0.533
newstest2010-deueng.deu.eng 28.5 0.569
newstest2011-deueng.deu.eng 25.8 0.542
newstest2012-deueng.deu.eng 27.0 0.552
newstest2013-deueng.deu.eng 30.0 0.568
newstest2014-deen-deueng.deu.eng 30.6 0.574
newstest2015-ende-deueng.deu.eng 31.5 0.577
newstest2016-ende-deueng.deu.eng 37.0 0.626
newstest2017-ende-deueng.deu.eng 32.8 0.588
newstest2018-ende-deueng.deu.eng 40.4 0.643
newstest2019-deen-deueng.deu.eng 36.6 0.615
Tatoeba-test.afr-eng.afr.eng 61.1 0.747
Tatoeba-test.ang-eng.ang.eng 7.4 0.217
Tatoeba-test.deu-eng.deu.eng 49.0 0.661
Tatoeba-test.enm-eng.enm.eng 16.3 0.430
Tatoeba-test.frr-eng.frr.eng 16.8 0.255
Tatoeba-test.fry-eng.fry.eng 28.5 0.482
Tatoeba-test.gos-eng.gos.eng 16.1 0.329
Tatoeba-test.gsw-eng.gsw.eng 11.8 0.289
Tatoeba-test.ksh-eng.ksh.eng 7.5 0.236
Tatoeba-test.ltz-eng.ltz.eng 29.1 0.443
Tatoeba-test.multi.eng 49.4 0.653
Tatoeba-test.nds-eng.nds.eng 32.5 0.522
Tatoeba-test.nld-eng.nld.eng 59.1 0.734
Tatoeba-test.pdc-eng.pdc.eng 29.3 0.438
Tatoeba-test.sco-eng.sco.eng 39.3 0.572
Tatoeba-test.stq-eng.stq.eng 21.4 0.405
Tatoeba-test.swg-eng.swg.eng 15.8 0.321
Tatoeba-test.yid-eng.yid.eng 17.8 0.378

opus1m+bt-2021-05-01.zip

Benchmarks

testset BLEU chr-F #sent #words BP
newssyscomb2009.deu-eng 26.2 0.534 502 11821 0.991
news-test2008.deu-eng 25.2 0.528 2051 49380 1.000
newstest2009.deu-eng 24.5 0.527 2525 65402 0.986
newstest2010.deu-eng 27.5 0.562 2489 61724 1.000
newstest2011.deu-eng 24.9 0.534 3003 74681 1.000
newstest2012.deu-eng 26.1 0.544 3003 72812 1.000
newstest2013.deu-eng 29.3 0.561 3000 64505 1.000
newstest2014-deen.deu-eng 29.8 0.566 3003 67337 0.980
newstest2015-ende.deu-eng 31.0 0.572 2169 46443 0.979
newstest2016-ende.deu-eng 36.1 0.616 2999 64126 1.000
newstest2017-ende.deu-eng 31.5 0.579 3004 64399 0.986
newstest2018-ende.deu-eng 39.3 0.633 2998 67013 0.982
newstest2019-deen.deu-eng 34.8 0.599 2000 39282 0.996
Tatoeba-test.afr-eng 61.8 0.746 1374 9622 0.991
Tatoeba-test.ang-eng 8.7 0.215 189 2070 0.990
Tatoeba-test.bar-eng 1.8 0.163 93 727 1.000
Tatoeba-test.deu-eng 47.6 0.649 10000 81213 0.979
Tatoeba-test.enm-eng 25.4 0.502 49 294 1.000
Tatoeba-test.frr-eng 8.4 0.106 2 9 1.000
Tatoeba-test.fry-eng 42.1 0.598 205 1500 1.000
Tatoeba-test.gos-eng 15.9 0.335 1152 5621 1.000
Tatoeba-test.gsw-eng 21.9 0.369 205 990 0.988
Tatoeba-test.hrx-eng 26.6 0.427 221 1235 1.000
Tatoeba-test.jam-eng 16.4 0.282 35 158 0.941
Tatoeba-test.ksh-eng 5.8 0.270 26 207 0.935
Tatoeba-test.ltz-eng 52.1 0.650 283 1751 1.000
Tatoeba-test.multi-eng 49.0 0.649 10000 72322 0.997
Tatoeba-test.nds-eng 43.1 0.609 2500 17584 1.000
Tatoeba-test.nld-eng 57.4 0.723 10000 69840 0.977
Tatoeba-test.pdc-eng 26.9 0.444 53 399 0.995
Tatoeba-test.sco-eng 38.1 0.587 27 216 1.000
Tatoeba-test.stq-eng 18.4 0.459 5 34 1.000
Tatoeba-test.swg-eng 12.6 0.294 33 251 1.000
Tatoeba-test.tpi-eng 19.1 0.430 49 228 1.000
Tatoeba-test.yid-eng 16.5 0.378 1168 7738 1.000

opus4m+btTCv20210807-2021-09-30.zip

Benchmarks

testset BLEU chr-F #sent #words BP
newssyscomb2009.deu-eng 25.5 0.527 502 11821 1.000
news-test2008.deu-eng 24.7 0.521 2051 49380 1.000
newstest2009.deu-eng 23.6 0.517 2525 65402 0.983
newstest2010.deu-eng 26.7 0.553 2489 61724 1.000
newstest2011.deu-eng 23.7 0.524 3003 74681 1.000
newstest2012.deu-eng 25.3 0.535 3003 72812 1.000
newstest2013.deu-eng 27.9 0.551 3000 64505 1.000
newstest2014-deen.deu-eng 28.6 0.557 3003 67337 0.982
newstest2015-ende.deu-eng 30.3 0.565 2169 46443 0.980
newstest2016-ende.deu-eng 35.4 0.608 2999 64126 1.000
newstest2017-ende.deu-eng 31.1 0.572 3004 64399 0.987
newstest2018-ende.deu-eng 38.0 0.622 2998 67013 0.982
newstest2019-deen.deu-eng 33.3 0.586 2000 39282 0.998
Tatoeba-test-v2021-08-07.afr-eng 55.1 0.705 1374 9622 1.000
Tatoeba-test-v2021-08-07.ang-eng 9.0 0.223 193 2086 1.000
Tatoeba-test-v2021-08-07.bar-eng 2.4 0.164 96 749 1.000
Tatoeba-test-v2021-08-07.deu-eng 42.2 0.608 17565 149415 0.985
Tatoeba-test-v2021-08-07.drt-eng 32.5 0.531 11 79 1.000
Tatoeba-test-v2021-08-07.eng-eng 61.2 0.770 12062 115099 0.974
Tatoeba-test-v2021-08-07.enm-eng 19.4 0.439 67 462 1.000
Tatoeba-test-v2021-08-07.frr-eng 4.5 0.202 197 1434 1.000
Tatoeba-test-v2021-08-07.fry-eng 36.2 0.544 220 1573 1.000
Tatoeba-test-v2021-08-07.gos-eng 19.2 0.373 1154 5634 1.000
Tatoeba-test-v2021-08-07.gsw-eng 25.3 0.395 205 990 0.952
Tatoeba-test-v2021-08-07.hrx-eng 26.6 0.419 221 1235 1.000
Tatoeba-test-v2021-08-07.jam-eng 24.0 0.340 35 158 0.935
Tatoeba-test-v2021-08-07.ksh-eng 10.4 0.298 26 207 0.945
Tatoeba-test-v2021-08-07.lim-eng 35.4 0.582 7 43 0.952
Tatoeba-test-v2021-08-07.ltz-eng 14.6 0.289 293 1840 1.000
Tatoeba-test-v2021-08-07.multi-eng 42.3 0.599 10000 76188 1.000
Tatoeba-test-v2021-08-07.multi-multi 42.3 0.599 10000 76188 1.000
Tatoeba-test-v2021-08-07.nds-eng 43.3 0.602 2500 17584 1.000
Tatoeba-test-v2021-08-07.nld-eng 54.6 0.695 12696 89970 0.993
Tatoeba-test-v2021-08-07.ofs-eng 3.9 0.164 2 15 1.000
Tatoeba-test-v2021-08-07.pdc-eng 29.3 0.453 53 399 1.000
Tatoeba-test-v2021-08-07.sco-eng 35.2 0.520 73 675 1.000
Tatoeba-test-v2021-08-07.srn-eng 63.4 0.445 2 8 0.867
Tatoeba-test-v2021-08-07.stq-eng 20.5 0.413 11 74 1.000
Tatoeba-test-v2021-08-07.swg-eng 10.4 0.319 34 259 1.000
Tatoeba-test-v2021-08-07.tpi-eng 18.5 0.423 49 228 1.000
Tatoeba-test-v2021-08-07.yid-eng 1.1 0.135 2483 15449 1.000
Tatoeba-test-v2021-08-07.zea-eng 35.3 0.611 3 20 1.000