Add Ukrainian models #409
Add Ukrainian models #409
Conversation
|
There's so many caveats that I don't know if we should implement it now. |
|
I will test it tomorrow morning. |
|
It looks pretty usable to me on a regular website like this one https://www.pravda.com.ua/ Of course, if you will try to translate huge texts on Wikipedia it will be noticeably slow but it's the same for our other models. |
|
I've pulled and tested the xpi built from CI and Ukrainian is not being displayed in the translation bar at least in a pt-br browser.
|
|
Same on english browser |
did other languages work? I can see it working using |
yes |
|
not sure what the issue was but I synced it with main and it looks working now |
The models were not quantized using browsermt marian (it produces so-called alpha models) and should be used with
gemm-precision: int8shiftAll. See browsermt/marian-dev config docs for an explanation of the difference. This mode is ~15% slower mozilla/translation-service#20 (comment).Also, we need to update Gecko to support non-alpha models, so they are slow for now ( I assume the same as usage on ARM).
We distinguish between different model types based on naming convention in the model registy.
The models include two different vocabularies for source and target language, unlike all the other models that currently reuse one vocabulary. I added two registry keys to support this.
Also, the model itself is a bit heavier and there's more data to download because of two vocabularies.
Despite all that it's good to have an extra language. It's marked as "Beta", so there shouldn't be expectations for speed or quality.
fixes #166