Using other tts models exported to onnxruntime with sherpa-onnx

[tn-17]

Besides VITS, can other tts models based on VITS and are exported to onnxruntime work with sherpa-onnx?

• https://github.com/p0p4k/vits2_pytorch
• https://github.com/fishaudio/Bert-VITS2
• https://github.com/RVC-Boss/GPT-SoVITS/tree/main

If they are not compatible out of the box, what needs to be done to make them compatible? I am interesting in learning how to implement support for those models for the sherpa-onnx on python.

[csukuangfj]

The first step is to convert them to onnx.

[yaambe]

I successfully exported a VITS model to ONNX after fine-tuning an MMS model using Hugging Face tools. You can find the tools I used here: https://github.com/ylacombe/finetune-hf-vits.

However, I'm having trouble finding documentation on how to convert these VITS models to Sherpa ONNX. Any guidance or resources would be greatly appreciated!

[csukuangfj]

Thank you!

Would you mind uploading the exported model to huggingface?

[yaambe]

Check out the following link.
https://huggingface.co/yaambe/mms-tts-spa/tree/main
Its a Spanish model.

[thewh1teagle]

Thank you so much! it worked for me with other mms models. I also needed to prepare vocab.json into tokens.txt but that's simple

[thewh1teagle]

@csukuangfj
I also exported the Hebrew model to hugging face, maybe it's useful mms-tts-heb

[willwade]

Thank you!

Would you mind uploading the exported model to huggingface?

Is there a neater way than me publishing them on my own huggingface account? Like a pr on a onnx-mms account or something? I’ll invest time in converting them but I feel I’d be best not hosting them

This way too the community know what’s been converted and hasn’t.

[willwade]

I’m currently converting all mms models at https://huggingface.co/willwade/mms-tts-multilingual-models-onnx/tree/main

they are going up via a big automated script.
code here https://github.com/willwade/mms-tts-multilingual-models-onnx

I’m creating wav files when I can