Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Patch int8PrepareBias to handle nullptr #82

Merged
merged 2 commits into from
Mar 16, 2022

Conversation

jelmervdl
Copy link
Member

@jelmervdl jelmervdl commented Mar 16, 2022

Description

This fixes base student models not working with the wasm version of bergamot-translator.

int8PrepareBias is being called from both PrepareBiasForBNodeOp and PrepareFakeBiasForBNodeOp. These are two different intgemm calls (callbacks differ) but they map to the same WASM surrogate function.

I'm using the fact that one of those calls has bias_input = nullptr as a way to differentiate between the two and call the correct intgemm function + callback pair in its implementation.

This will also need to be fixed in Mozilla's tree, @abhi-agg.

While on the topic of Mozilla's implementation, should the bounds check in Mozilla's implementation also pick up nullptrs?

See issue #81 for a more detailed explanation.

List of changes:

  • Fix int8PrepareBias fallback to call intgemm::Int8Shift::PrepareBias with the correct callback.
  • Also fixes a nullptr dereference that would be crash-worthy were it to run natively.

How to test

See #81 how I tested this. To test whether the fix itself works, easiest way would be:

  1. Check out https://github.com/jelmervdl/firefox-translations
  2. Change marian-dev submodule inside the bergamot-translator submodule to point to this branch
  3. Build the wasm module. There is a build-docker-wasm.sh script in the repository. You can use that, but you have to make sure it doesn't re-checkout the dependencies. There is a cmake flag, GIT_SUBMODULE, that stops cmake from doing that.
  4. After build-docker-wasm.sh (which also copies the .wasm and .js file) you need to go into the bergamot-translation-worker.js file and force it to always use the fallback implementation, not the optimised one in Firefox itself.
  5. Add the entries.reverse() as mentioned here: TranslateLocally _base_ models do not work jelmervdl/translatelocally-web-ext#14.
  6. Go to German wikipedia for example, and translate it to English.

Checklist

  • I have tested the code manually
  • I have run regression tests
  • I have read and followed CONTRIBUTING.md
  • I have updated CHANGELOG.md

…deOp` and `PrepareFakeBiasForBNodeOp`

Also this effectively is just a patch for a nullptr dereference error that did *not* cause a crash in the wasm interpreter? Worrying.
Copy link
Collaborator

@XapaJIaMnu XapaJIaMnu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor suggestion.

width,
cols_B,
intgemm::callbacks::UnquantizeAndAddBiasAndWrite(unquant_factor, input_bias, output));
if (input_bias == nullptr) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, stylistic. I prefer if (input_bias), and I have used this style of nullptr checking elsewhere.

@XapaJIaMnu XapaJIaMnu merged commit 844800e into browsermt:master Mar 16, 2022
@jelmervdl jelmervdl deleted the fix-wasm-intgemm-nullptr branch March 16, 2022 14:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants