(Q) Multi/Single Speaker different language finetune

[mantrakp04]

I was wondering if anyone was successful with finetuning a styletts2 base model on a different language eg. French, Spanish, etc... and achieved good results, if so someone kind enough to share some outputs, training approach and dataset info (total duration, total speakers, language)

[Respaired]

yeah, a few individuals such as yours truly have done it.

[mantrakp04]

can you share some details on the model, language, duration, etc

[Respaired]

japanese on a 21hrs dataset, single speaker.

[Karesto]

I did multiple tries:

Around 30 hours of data.
French. (Mono speaker for now).

It is however, not a finetune, if you're training on a new language, i highly recommend restarting from scratch the training, albeit being very slow.

[mantrakp04]

What kind of hardware did you use n how long it take

…

On Thu, Sep 12, 2024 at 12:47 AM Karesto ***@***.***> wrote: I did multiple tries: Around 30 hours of data. French. (Mono speaker for now). It is however, not a finetune, if you're training on a new language, i highly recommend restarting from scratch the training, albeit being very slow. — Reply to this email directly, view it on GitHub <#282 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AUY3AOPJROURYDNHGWSGBY3ZWFBH5AVCNFSM6AAAAABNWYIUUOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNBVGUYTAMJXGU> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Karesto]

I used 2 x4090s. it took about 2 weeks (with a few stops for bugs here and there)

[martinambrus]

I would also recommend looking here for some pointers on multi-lingual StyleTTS2 creation: #257
It has some interesting points, especially about BERT and AuxiliaryASR models training for languages different from English.