very very slowly RTX4090

[qwerdf4]

--height 704 --width 480 --num_frames 121 RTX4090 need 2.5hours？？？

[bssrdf]

The current version of inference.py has text encoder occupying all the VRAM. Even 24GB on 4090 got overwhelmed and throttling. Try updated version https://github.com/KT313/LTX_Video_better_vram/tree/test which evicted text encoder after it is done. Much faster.

[eoffermann]

@bssrdf - I think the more impactful aspect of that change is converting the UNET model to bf16. That shaves a large amount off of the memory footprint and has negligible impact on the output quality. I didn't care for loading and unloading the model - it substantially reduced the memory footprint, again, but added unwanted reloads that had more of an impact on overall speed for me than was gained by freeing up GPU RAM. (YMMV naturally)

[bssrdf]

yes, --bfloat16 is also helping with VRAM reduction, but itself alone can not solve OP's problem. The LTX transformer + vae is quite small actually. T5's size is the problem.

[able2608]

Not sure if this repo will receive this much of an optimization, but if what you need to fix is only about T5, there are quantized version of it that significantly reduce the requirements down to (I guess) 6GB or less. I have been able to generate at 768x512x97 on 6GB VRAM (+16GB RAM, a pretty mediocre spec). I have written instruction on how to do so on ComfyUI at #4 (comment). If you really want to try it out now on your machine, I believe this is currently the only way to go.