[Feature Extractors] Fix kwargs to pre-trained

[sanchit-gandhi]

What does this PR do?

It was reported by @osanseviero that when instantiating a feature extractor from pre-trained, setting kwargs occasionally leads to incorrect behaviour. This is particularly the case for Whisper, where setting the argument feature_size can have no effect on the number of Mel-bins:

from transformers import WhisperFeatureExtractor

# initialise from kwargs -> mel filters matches feature size
feature_extractor = WhisperFeatureExtractor(feature_size=100)
print(f"From kwargs: feature size {feature_extractor.feature_size}, mel filters {feature_extractor.mel_filters.shape[1]}")

# initialise from pre-trained with kwargs -> mel filters not updated to new feature size
feature_extractor = WhisperFeatureExtractor.from_pretrained("openai/whisper-tiny", feature_size=100)
print(f"From pre-trained: feature size {feature_extractor.feature_size}, mel filters {feature_extractor.mel_filters.shape[1]}")

Print Output:

From kwargs: feature size 100, mel filters 100
From pre-trained: feature size 100, mel filters 80

This is because of the order in which we set the args in the feature extractor. We first pass all the arguments that we have saved in the preprocessor_config.json file:

transformers/src/transformers/feature_extraction_utils.py

Line 569 in 8127f39

feature_extractor = cls(**feature_extractor_dict)

And subsequently override any kwargs that we got from the user:

transformers/src/transformers/feature_extraction_utils.py

Lines 573 to 575 in 8127f39

	for key, value in kwargs.items():
	if hasattr(feature_extractor, key):
	setattr(feature_extractor, key, value)

The problem with this method is that for Whisper, we set some attributes based on the values of others in the init. E.g. we set mel_filters based on the value of feature_size:

transformers/src/transformers/models/whisper/feature_extraction_whisper.py

Lines 87 to 89 in 8127f39

	self.mel_filters = mel_filter_bank(
	num_frequency_bins=1 + n_fft // 2,
	num_mel_filters=feature_size,

These won't be updated properly with our current method, since we're only updating the attribute feature_size, and not subsequently recomputing the correct mel_filters.

The simple fix is to give priority to the user kwargs and ensure these are passed to the init.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[ArthurZucker]

LGTM good catch both!

[guynich]

Well done for this change. I was hitting the same issue with transformers 4.39.3 package - with transformers 4.41.0 I can use kwargs as expected. Thanks.