Skip to content

Latest commit

 

History

History
 
 

roa-eng

README.md

opus-2020-06-28.zip

  • dataset: opus
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn glg ita lad lad_Latn lij lld_Latn lmo mwl oci osp_Latn pms por roh ron scn spa vec wln
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-06-28.zip
  • test set translations: opus-2020-06-28.test.txt
  • test set scores: opus-2020-06-28.eval.txt

Benchmarks

testset BLEU chr-F
Tatoeba-test.arg-eng.arg.eng 44.4 0.532
Tatoeba-test.ast-eng.ast.eng 34.3 0.515
Tatoeba-test.cat-eng.cat.eng 54.6 0.704
Tatoeba-test.cos-eng.cos.eng 53.7 0.670
Tatoeba-test.egl-eng.egl.eng 1.7 0.148
Tatoeba-test.ext-eng.ext.eng 40.2 0.576
Tatoeba-test.fra-eng.fra.eng 54.3 0.694
Tatoeba-test.frm-eng.frm.eng 27.1 0.448
Tatoeba-test.fvr-eng.fvr.eng 37.9 0.554
Tatoeba-test.glg-eng.glg.eng 54.8 0.702
Tatoeba-test.ita-eng.ita.eng 65.6 0.773
Tatoeba-test.lad-eng.lad.eng 11.0 0.376
Tatoeba-test.lij-eng.lij.eng 10.1 0.292
Tatoeba-test.lld-eng.lld.eng 13.4 0.308
Tatoeba-test.lmo-eng.lmo.eng 14.7 0.310
Tatoeba-test.multi.eng 57.3 0.715
Tatoeba-test.mwl-eng.mwl.eng 33.1 0.597
Tatoeba-test.oci-eng.oci.eng 18.8 0.393
Tatoeba-test.osp-eng.osp.eng 51.4 0.699
Tatoeba-test.pms-eng.pms.eng 11.9 0.324
Tatoeba-test.por-eng.por.eng 59.9 0.739
Tatoeba-test.roh-eng.roh.eng 17.3 0.397
Tatoeba-test.ron-eng.ron.eng 54.8 0.701
Tatoeba-test.scn-eng.scn.eng 47.5 0.421
Tatoeba-test.spa-eng.spa.eng 56.6 0.716
Tatoeba-test.vec-eng.vec.eng 16.4 0.334
Tatoeba-test.wln-eng.wln.eng 14.9 0.296

opus-2020-07-27.zip

  • dataset: opus
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn gcf_Latn glg hat ind ita lad lad_Latn lij lld_Latn lmo max_Latn mfe min mwl oci pap pms por roh ron scn spa tmw_Latn vec wln zlm_Latn zsm_Latn
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus-2020-07-27.zip
  • test set translations: opus-2020-07-27.test.txt
  • test set scores: opus-2020-07-27.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2016-enro-roneng.ron.eng 36.3 0.626
newsdiscussdev2015-enfr-fraeng.fra.eng 30.9 0.560
newsdiscusstest2015-enfr-fraeng.fra.eng 35.5 0.589
newssyscomb2009-fraeng.fra.eng 29.2 0.561
newssyscomb2009-itaeng.ita.eng 32.5 0.583
newssyscomb2009-spaeng.spa.eng 28.9 0.560
news-test2008-fraeng.fra.eng 25.1 0.531
news-test2008-spaeng.spa.eng 26.2 0.538
newstest2009-fraeng.fra.eng 28.2 0.553
newstest2009-itaeng.ita.eng 31.5 0.578
newstest2009-spaeng.spa.eng 28.6 0.557
newstest2010-fraeng.fra.eng 29.5 0.569
newstest2010-spaeng.spa.eng 32.9 0.593
newstest2011-fraeng.fra.eng 30.8 0.582
newstest2011-spaeng.spa.eng 31.8 0.582
newstest2012-fraeng.fra.eng 31.2 0.577
newstest2012-spaeng.spa.eng 34.8 0.604
newstest2013-fraeng.fra.eng 31.5 0.571
newstest2013-spaeng.spa.eng 32.5 0.589
newstest2014-fren-fraeng.fra.eng 34.0 0.606
newstest2016-enro-roneng.ron.eng 35.0 0.609
Tatoeba-test.arg-eng.arg.eng 45.7 0.549
Tatoeba-test.ast-eng.ast.eng 33.7 0.506
Tatoeba-test.cat-eng.cat.eng 53.7 0.696
Tatoeba-test.cos-eng.cos.eng 59.9 0.641
Tatoeba-test.egl-eng.egl.eng 3.2 0.184
Tatoeba-test.ext-eng.ext.eng 32.9 0.481
Tatoeba-test.fra-eng.fra.eng 53.3 0.687
Tatoeba-test.frm-eng.frm.eng 24.0 0.405
Tatoeba-test.gcf-eng.gcf.eng 14.3 0.287
Tatoeba-test.glg-eng.glg.eng 55.4 0.711
Tatoeba-test.hat-eng.hat.eng 35.1 0.523
Tatoeba-test.ita-eng.ita.eng 64.5 0.765
Tatoeba-test.lad-eng.lad.eng 13.4 0.413
Tatoeba-test.lij-eng.lij.eng 10.2 0.288
Tatoeba-test.lld-eng.lld.eng 19.3 0.336
Tatoeba-test.lmo-eng.lmo.eng 7.4 0.305
Tatoeba-test.mfe-eng.mfe.eng 63.1 0.805
Tatoeba-test.msa-eng.msa.eng 40.5 0.588
Tatoeba-test.multi.eng 54.5 0.694
Tatoeba-test.mwl-eng.mwl.eng 24.3 0.523
Tatoeba-test.oci-eng.oci.eng 20.8 0.408
Tatoeba-test.pap-eng.pap.eng 55.7 0.675
Tatoeba-test.pms-eng.pms.eng 11.6 0.322
Tatoeba-test.por-eng.por.eng 59.3 0.735
Tatoeba-test.roh-eng.roh.eng 15.2 0.402
Tatoeba-test.ron-eng.ron.eng 54.2 0.696
Tatoeba-test.scn-eng.scn.eng 40.0 0.396
Tatoeba-test.spa-eng.spa.eng 56.1 0.713
Tatoeba-test.vec-eng.vec.eng 12.7 0.316
Tatoeba-test.wln-eng.wln.eng 13.3 0.293

opus2m-2020-08-01.zip

  • dataset: opus2m
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn gcf_Latn glg hat ind ita lad lad_Latn lij lld_Latn lmo max_Latn mfe min mwl oci pap pms por roh ron scn spa tmw_Latn vec wln zlm_Latn zsm_Latn
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus2m-2020-08-01.zip
  • test set translations: opus2m-2020-08-01.test.txt
  • test set scores: opus2m-2020-08-01.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2016-enro-roneng.ron.eng 37.1 0.631
newsdiscussdev2015-enfr-fraeng.fra.eng 31.6 0.564
newsdiscusstest2015-enfr-fraeng.fra.eng 36.1 0.592
newssyscomb2009-fraeng.fra.eng 29.3 0.563
newssyscomb2009-itaeng.ita.eng 33.1 0.589
newssyscomb2009-spaeng.spa.eng 29.2 0.562
news-test2008-fraeng.fra.eng 25.2 0.533
news-test2008-spaeng.spa.eng 26.6 0.542
newstest2009-fraeng.fra.eng 28.6 0.557
newstest2009-itaeng.ita.eng 32.0 0.580
newstest2009-spaeng.spa.eng 28.9 0.559
newstest2010-fraeng.fra.eng 29.9 0.573
newstest2010-spaeng.spa.eng 33.3 0.596
newstest2011-fraeng.fra.eng 31.2 0.585
newstest2011-spaeng.spa.eng 32.3 0.584
newstest2012-fraeng.fra.eng 31.3 0.580
newstest2012-spaeng.spa.eng 35.3 0.606
newstest2013-fraeng.fra.eng 31.9 0.575
newstest2013-spaeng.spa.eng 32.8 0.592
newstest2014-fren-fraeng.fra.eng 34.6 0.611
newstest2016-enro-roneng.ron.eng 35.8 0.614
Tatoeba-test.arg-eng.arg.eng 38.7 0.512
Tatoeba-test.ast-eng.ast.eng 35.2 0.520
Tatoeba-test.cat-eng.cat.eng 54.9 0.703
Tatoeba-test.cos-eng.cos.eng 68.1 0.666
Tatoeba-test.egl-eng.egl.eng 6.7 0.209
Tatoeba-test.ext-eng.ext.eng 24.2 0.427
Tatoeba-test.fra-eng.fra.eng 53.9 0.691
Tatoeba-test.frm-eng.frm.eng 25.7 0.423
Tatoeba-test.gcf-eng.gcf.eng 14.8 0.288
Tatoeba-test.glg-eng.glg.eng 54.6 0.703
Tatoeba-test.hat-eng.hat.eng 37.0 0.540
Tatoeba-test.ita-eng.ita.eng 64.8 0.768
Tatoeba-test.lad-eng.lad.eng 21.7 0.452
Tatoeba-test.lij-eng.lij.eng 11.2 0.299
Tatoeba-test.lld-eng.lld.eng 10.8 0.273
Tatoeba-test.lmo-eng.lmo.eng 5.8 0.260
Tatoeba-test.mfe-eng.mfe.eng 63.1 0.819
Tatoeba-test.msa-eng.msa.eng 40.9 0.592
Tatoeba-test.multi.eng 54.9 0.697
Tatoeba-test.mwl-eng.mwl.eng 44.6 0.674
Tatoeba-test.oci-eng.oci.eng 20.5 0.404
Tatoeba-test.pap-eng.pap.eng 56.2 0.669
Tatoeba-test.pms-eng.pms.eng 10.3 0.324
Tatoeba-test.por-eng.por.eng 59.7 0.738
Tatoeba-test.roh-eng.roh.eng 14.8 0.378
Tatoeba-test.ron-eng.ron.eng 55.2 0.703
Tatoeba-test.scn-eng.scn.eng 10.2 0.259
Tatoeba-test.spa-eng.spa.eng 56.2 0.714
Tatoeba-test.vec-eng.vec.eng 13.8 0.317
Tatoeba-test.wln-eng.wln.eng 17.3 0.323

opus4m-2020-08-12.zip

  • dataset: opus4m
  • model: transformer
  • source language(s): arg ast cat cos egl ext fra frm_Latn gcf_Latn glg hat ind ita lad lad_Latn lij lld_Latn lmo max_Latn mfe min mwl oci pap pms por roh ron scn spa tmw_Latn vec wln zlm_Latn zsm_Latn
  • target language(s): eng
  • model: transformer
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus4m-2020-08-12.zip
  • test set translations: opus4m-2020-08-12.test.txt
  • test set scores: opus4m-2020-08-12.eval.txt

Benchmarks

testset BLEU chr-F
newsdev2016-enro-roneng.ron.eng 37.2 0.632
newsdiscussdev2015-enfr-fraeng.fra.eng 31.8 0.566
newsdiscusstest2015-enfr-fraeng.fra.eng 36.2 0.595
newssyscomb2009-fraeng.fra.eng 29.5 0.564
newssyscomb2009-itaeng.ita.eng 32.1 0.584
newssyscomb2009-spaeng.spa.eng 28.6 0.560
news-test2008-fraeng.fra.eng 25.3 0.533
news-test2008-spaeng.spa.eng 26.4 0.541
newstest2009-fraeng.fra.eng 28.7 0.557
newstest2009-itaeng.ita.eng 31.9 0.581
newstest2009-spaeng.spa.eng 29.1 0.561
newstest2010-fraeng.fra.eng 30.1 0.575
newstest2010-spaeng.spa.eng 33.3 0.595
newstest2011-fraeng.fra.eng 31.2 0.585
newstest2011-spaeng.spa.eng 32.2 0.584
newstest2012-fraeng.fra.eng 31.3 0.579
newstest2012-spaeng.spa.eng 35.1 0.605
newstest2013-fraeng.fra.eng 32.0 0.576
newstest2013-spaeng.spa.eng 32.9 0.591
newstest2014-fren-fraeng.fra.eng 34.6 0.610
newstest2016-enro-roneng.ron.eng 35.9 0.614
Tatoeba-test.arg-eng.arg.eng 44.6 0.535
Tatoeba-test.ast-eng.ast.eng 34.9 0.527
Tatoeba-test.cat-eng.cat.eng 55.1 0.708
Tatoeba-test.cos-eng.cos.eng 63.2 0.651
Tatoeba-test.egl-eng.egl.eng 5.6 0.212
Tatoeba-test.ext-eng.ext.eng 29.3 0.468
Tatoeba-test.fra-eng.fra.eng 53.9 0.692
Tatoeba-test.frm-eng.frm.eng 23.6 0.405
Tatoeba-test.gcf-eng.gcf.eng 18.0 0.316
Tatoeba-test.glg-eng.glg.eng 55.4 0.712
Tatoeba-test.hat-eng.hat.eng 37.3 0.553
Tatoeba-test.ita-eng.ita.eng 65.4 0.772
Tatoeba-test.lad-eng.lad.eng 14.0 0.441
Tatoeba-test.lij-eng.lij.eng 9.2 0.282
Tatoeba-test.lld-eng.lld.eng 8.7 0.267
Tatoeba-test.lmo-eng.lmo.eng 10.8 0.287
Tatoeba-test.mfe-eng.mfe.eng 66.0 0.805
Tatoeba-test.msa-eng.msa.eng 41.1 0.595
Tatoeba-test.multi.eng 55.1 0.699
Tatoeba-test.mwl-eng.mwl.eng 22.3 0.393
Tatoeba-test.oci-eng.oci.eng 21.7 0.410
Tatoeba-test.pap-eng.pap.eng 56.6 0.672
Tatoeba-test.pms-eng.pms.eng 12.9 0.335
Tatoeba-test.por-eng.por.eng 59.8 0.738
Tatoeba-test.roh-eng.roh.eng 19.4 0.429
Tatoeba-test.ron-eng.ron.eng 55.4 0.704
Tatoeba-test.scn-eng.scn.eng 35.8 0.388
Tatoeba-test.spa-eng.spa.eng 56.4 0.716
Tatoeba-test.vec-eng.vec.eng 13.6 0.343
Tatoeba-test.wln-eng.wln.eng 13.4 0.315

opus1m+bt-2021-05-01.zip

  • dataset: opus1m+bt
  • model: transformer-align
  • source language(s): arg ast cat cbk cos egl ext fra frm gcf glg hat ita lad lij lld lmo mfe mol mwl oci osp pap pms pob por roh ron scn spa vec wln
  • target language(s): eng
  • model: transformer-align
  • pre-processing: normalization + SentencePiece (spm32k,spm32k)
  • download: opus1m+bt-2021-05-01.zip
  • test set translations: opus1m+bt-2021-05-01.test.txt
  • test set scores: opus1m+bt-2021-05-01.eval.txt

Benchmarks

testset BLEU chr-F #sent #words BP
newsdev2016-enro.ron-eng 36.2 0.625 1999 49526 1.000
newsdiscussdev2015-enfr.fra-eng 30.9 0.559 1500 27759 0.961
newsdiscusstest2015-enfr.fra-eng 35.8 0.591 1500 26995 0.995
newssyscomb2009.fra-eng 29.7 0.563 502 11821 0.998
newssyscomb2009.ita-eng 32.3 0.582 502 11821 0.983
newssyscomb2009.spa-eng 29.4 0.562 502 11821 1.000
news-test2008.fra-eng 25.2 0.533 2051 49380 1.000
news-test2008.spa-eng 26.3 0.540 2051 49380 1.000
newstest2009.fra-eng 28.0 0.553 2525 65402 1.000
newstest2009.ita-eng 31.7 0.579 2525 65402 0.983
newstest2009.spa-eng 29.1 0.561 2525 65402 0.988
newstest2010.fra-eng 29.9 0.572 2489 61724 1.000
newstest2010.spa-eng 33.1 0.594 2489 61724 1.000
newstest2011.fra-eng 30.9 0.582 3003 74681 1.000
newstest2011.spa-eng 31.9 0.583 3003 74681 1.000
newstest2012.fra-eng 31.2 0.578 3003 72812 1.000
newstest2012.spa-eng 35.0 0.605 3003 72812 1.000
newstest2013.fra-eng 31.6 0.573 3000 64505 1.000
newstest2013.spa-eng 32.5 0.590 3000 64505 1.000
newstest2014-fren.fra-eng 33.9 0.607 3003 70708 1.000
newstest2016-enro.ron-eng 34.4 0.603 1999 47563 1.000
Tatoeba-test.arg-eng 33.5 0.465 105 451 1.000
Tatoeba-test.ast-eng 34.7 0.502 99 802 0.946
Tatoeba-test.cat-eng 54.2 0.700 1631 12625 0.964
Tatoeba-test.cbk-eng 18.6 0.414 1498 10024 1.000
Tatoeba-test.cos-eng 65.9 0.691 5 42 1.000
Tatoeba-test.egl-eng 3.6 0.163 84 444 1.000
Tatoeba-test.ext-eng 43.9 0.571 69 396 0.851
Tatoeba-test.fra-eng 52.4 0.681 10000 77165 0.977
Tatoeba-test.frm-eng 21.3 0.379 18 231 0.919
Tatoeba-test.gcf-eng 12.6 0.271 99 570 1.000
Tatoeba-test.glg-eng 54.8 0.703 1008 8364 0.969
Tatoeba-test.hat-eng 43.7 0.589 64 384 1.000
Tatoeba-test.ita-eng 64.9 0.768 10000 67384 0.979
Tatoeba-test.lad-eng 22.8 0.448 629 3456 1.000
Tatoeba-test.lad_Latn-eng 31.7 0.487 582 3200 0.989
Tatoeba-test.lij-eng 11.4 0.286 94 698 1.000
Tatoeba-test.lld-eng 15.7 0.288 21 226 0.964
Tatoeba-test.lmo-eng 10.3 0.295 17 132 0.953
Tatoeba-test.mfe-eng 72.0 0.861 7 35 1.000
Tatoeba-test.multi-eng 55.2 0.699 10000 74266 0.991
Tatoeba-test.mwl-eng 36.6 0.600 4 24 0.819
Tatoeba-test.oci-eng 20.2 0.391 841 5299 1.000
Tatoeba-test.osp-eng 51.4 0.699 3 21 0.951
Tatoeba-test.pap-eng 63.5 0.706 70 366 0.989
Tatoeba-test.pms-eng 9.8 0.301 268 2055 1.000
Tatoeba-test.por-eng 60.0 0.742 10000 75224 0.969
Tatoeba-test.roh-eng 16.6 0.392 16 214 0.981
Tatoeba-test.ron-eng 51.9 0.679 5000 37123 0.970
Tatoeba-test.scn-eng 54.3 0.481 4 44 0.953
Tatoeba-test.spa-eng 56.4 0.715 10000 79355 0.975
Tatoeba-test.vec-eng 17.2 0.352 19 127 1.000
Tatoeba-test.wln-eng 14.7 0.318 89 465 1.000
tico19-test.fra-eng 35.7 0.607 2100 56347 1.000
tico19-test.pob-eng 46.8 0.727 2100 56339 1.000
tico19-test.por-eng 46.8 0.727 2100 56339 1.000
tico19-test.spa-eng 43.9 0.710 2100 56339 1.000