Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add MLX Whisper Backend for Apple Silicon #147

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

QuentinFuxa
Copy link

Hi,

As indicated in my previous pull request, this PR concerns exclusively the mlx-whisper backend:

The Whisper MLX backend is optimized specifically for Apple Silicon. It uses the mlx-whisper library, which leverages the native performance capabilities of Apple hardware for faster transcription without requiring a dedicated GPU.

Key Changes

  • MLXWhisper Backend: Added MLXWhisper as a new class in the backend options.

    • Automatically translates model names (e.g., large-v2) to MLX-compatible Hugging Face paths (e.g., whisper-large-v2-mlx).
    • No change to the --model parameter in the project CLI, model "translation" is done in the background.
  • Readme has been updated with information & how to install (pip install mlx-whisper)

Testing

  • The implementation has been tested on Apple M1. It works faster than faster whisper.

Usage Example

python whisper_streaming.py --model "large-v3-turbo" --backend "mlx-whisper"

The MLXWhisper class translates the model name to the Hugging Face path mlx-community/whisper-large-v3-turbo and loads the model efficiently.

Let me know if you have any questions or need further clarifications!

Best regards,
Quentin

@Gldkslfmsd
Copy link
Collaborator

OK, thanks. I'm changing my mind. I can't review and test this code but I'd like to ask any volunteer to do so and let me know. Then I'd merge it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants