Memory leak in shared GPU memory

[viper7882]

Hi admin,

After running Lag-Llama Demo, the dedicated GPU memory is freed up but the shared GPU memory is stuck at fully occupied. I suspect it has not been freed up properly.

One point to note is during the early execution of the demo, I saw the shared memory was fluctuating from zero to max and vice versa. Hence, I strongly believe shared memory is fully controllable by Lag-Llama.

Screenshot of demo post execution using Jupyter Notebook:

You might want to check it out.

[lsf123-ai]

Hello, I'd like to ask you for some advice. My computer is configured with CUDA and the corresponding PyTorch. When running script files, GPU available: True (cuda), used: True, but in fact, when I check my computer's performance, it's not using the GPU. Could you please tell me what the reason might be? I'm directly running a pre-trained script file without specifically modifying any code related to the GPU. If I need to modify the code, which part should I change? Your guidance would be greatly appreciated.