Fix languages covered by M4Tv2 #28019
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
amyeroberts
approved these changes
Dec 14, 2023
src/transformers/models/seamless_m4t_v2/modeling_seamless_m4t_v2.py
Outdated
Show resolved
Hide resolved
| @@ -784,7 +790,7 @@ def update_generation(self, model): | |||
|
|
|||
| model.generation_config = generation_config | |||
|
|
|||
| def prepare_text_input(self): | |||
| def prepare_text_input(self, is_rus=False): | |||
There was a problem hiding this comment.
Why not just make this tgt_lang and then you can easily test with different target languages both in an outside of the supported languages with generate_speech as True and False?
Comment on lines
+857
to
+862
| # make sure that generating speech, with a language that is only supported for text translation, raises error | ||
| with self.assertRaises(ValueError): | ||
| model.generate(**input_text_rus) | ||
|
|
||
| # make sure that generating text only works | ||
| model.generate(**input_text_rus, generate_speech=False) |
There was a problem hiding this comment.
We should also make sure it works in both cases for a language supported for all tasks
…v2.py Co-authored-by: amyeroberts <[email protected]>
|
Thanks for the suggestion @amyeroberts ! I've integrated them and will merge when the CI is green! |
iantbutler01
pushed a commit
to BismuthCloud/transformers
that referenced
this pull request
Dec 16, 2023
* correct language assessment + add tests * Update src/transformers/models/seamless_m4t_v2/modeling_seamless_m4t_v2.py Co-authored-by: amyeroberts <[email protected]> * make style + simplify and enrich test --------- Co-authored-by: amyeroberts <[email protected]>
staghado
pushed a commit
to staghado/transformers
that referenced
this pull request
Jan 15, 2024
* correct language assessment + add tests * Update src/transformers/models/seamless_m4t_v2/modeling_seamless_m4t_v2.py Co-authored-by: amyeroberts <[email protected]> * make style + simplify and enrich test --------- Co-authored-by: amyeroberts <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Currently, M4Tv2 into-text tasks (ASR, S2TT, T2TT) do not work for languages outside of the 36 for which audio is supported. This is linked to a test at the beginning of the model generate. The model previously verified if the
tgt_langwas in a bunch of dictionaries, independently from the output modality. This PR aims to fix that.I've added a test to make sure it works.
cc @amyeroberts