Fix batch size 1 by specifying squeeze dims

[Sobsz]

Fixes #104 (as far as I can tell).

[yl4579]

Thanks for your fix, but what does .squeeze(0) do? Doesn’t it squeeze over the batch dimensions and cause issues?

[Sobsz]

...hm. I'll be honest, I don't remember my reasoning behind it; I just know it doesn't break.

Looking at the relevant code again, it seems if len(sp) <= 1: makes it return early anyway if the first dimension is squeezable. Perhaps the .squeeze(0)s should be removed entirely (maybe along with the early return?).

[yl4579]

So does it mean it's probably just redundant? Does it still work with batch size greater than 1? Also it would be great if you could implement gradient accumulate for the accelerate version of fine-tuning with 1 GPU but someone else could do it too. Sorry I am too busy to test the code nor adding new stuff.

[GUUser91]

@Sobsz
I tried your branch and I tried to finetune a model but I got the ZeroDivisionError: division by zero error message after 1 epoch. Batch size is set to 1.

[Sobsz]

Seems like the testing iterations are failing silently because of the except: continue at lines 673-674. (That, or your validation list is empty...?) Try replacing the continue with raise and see what error you get.

[GUUser91]

@Sobsz
Now I get the dimension out of range (expected to be in range of [-1, 0], but got 1) error message. I'm using the rocm5.7 nightly pytorch build if it helps.

[GUUser91]

@Sobsz
Here's a more clear version

Traceback (most recent call last):
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/train_finetune.py", line 707, in
main()
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/train_finetune.py", line 614, in main
d, p = model.predictor(d_en, s,
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/parallel/data_parallel.py", line 183, in forward
return self.module(*inputs[0], **module_kwargs[0])
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/models.py", line 469, in forward
d = self.text_encoder(texts, style, text_lengths, m)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/models.py", line 550, in forward
x = block(x.transpose(-1, -2), style).transpose(-1, -2)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/venv/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/run/media/user/e1745494-af46-4749-9e1a-89d2b2289699/StyleTTS2/models.py", line 431, in forward
h = h.view(h.size(0), h.size(1), 1)
IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

[korakoe]

Validation losses have a range mismatch, something likely has to be done here, as we cant't just not have validation loss

[korakoe]

turns out that this also breaks higher batch sizes

[brthor]

.squeeze(0) is going to squeeze the wrong dimension when batch size > 1, need to use .squeeze(dim=1) in most of these places (from manually stepping through with a debugger).

[korakoe]

.squeeze(0) is going to squeeze the wrong dimension when batch size > 1, need to use .squeeze(dim=1) in most of these places (from manually stepping through with a debugger).

@brthor can all of them be changed to dim=1 or do some need to remain 0? if so which ones?