Loading the model

[Mukish45]

Hello @nyadla-sys. You did a great job on making the whisper model working on Android. I'm also trying with the same, but struggling to load and run the inference using C/C++ library. I have tried to load the model, but the application crashes. Could you please share any references on setting up those C/C++ libraries.

[nyadla-sys]

I followed the steps from here to link the required libs. https://github.com/cuongvng/TF-Lite-Cpp-API-for-Android You can compile and run the attached zip file for your Android phone. https://github.com/usefulsensors/openai-whisper/blob/main/android_app/android_whisper_app.zip

…

On Sun, Feb 12, 2023 at 2:24 AM Mukish S ***@***.***> wrote: Hello @nyadla-sys <https://github.com/nyadla-sys>. You did a great job on making the whisper model working on Android. I'm also trying with the same, but struggling to load and run the inference using C/C++ library. Could you please share any references on setting up those C/C++ libraries. — Reply to this email directly, view it on GitHub <#20>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGL5SYQMYSWAYLUP2Z6WCODWXC26DANCNFSM6AAAAAAUZHFUGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Mukish45]

@nyadla-sys Is there any other alternative ways to run whisper model on android without using C++ libraries?

[nyadla-sys]

@Mukish45 Definitely, the app can be run using Java APIs, but I am not an expert in writing Android applications

[Mukish45]

@nyadla-sys Is it possible to transcript using whisper model on mobile. Eg: If I spoke in German, I should get its English translated text. Is it possible in mobile app?

[nyadla-sys]

Yes it is possible you need to use to different tflite model for translate in Android App

refer the below comments from this thread #15
This also might be of interest to you @nyadla-sys The base model do translation to english where as the tiny and small models just returned the language detected.

mycroft@OpenVoiceOS-e3830c:~/whisper $ minimal models/whisper-tiny.tflite de_speech_thorsten_sample03_8s.wav

n_vocab:50257

mel.n_len3000

mel.n_mel:80
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Inference time 7 seconds

[_extra_token_50258][_extra_token_50261][_extra_token_50359][BEG] Für mich sind alle Menschen gleich unabhängig von Geschlecht, sexuelle Orientierung, Religion, Hautfarbe oder Geo-Kordinaten der Geburt.[SOT]

mycroft@OpenVoiceOS-e3830c:~/whisper $ minimal models/whisper-base.tflite de_speech_thorsten_sample03_8s.wav

n_vocab:50257

mel.n_len3000

mel.n_mel:80
INFO: Created TensorFlow Lite XNNPACK delegate for CPU.
Inference time 12 seconds

[_extra_token_50258][_extra_token_50261][_extra_token_50358][BEG] For me, all people are equally independent

[nyadla-sys]

Actually some one is trying to implement multilanguage(transcribe and translate) support on Android APP using TFLite models

Refer the below code for more details..

https://github.com/ipsilondev/whisper-cordova/blob/main/android/cpp/native-lib.cpp

[Mukish45]

Thank you @nyadla-sys for your quick response also for sharing reference.