Running 'example. py' in a Docker container can cause the system to freeze or be forced to shut down

[ziyaxuanyi]

I am running it in a Docker container and nunkaku has been successfully compiled.

I replaced the schnell model in example. py with the dev model and ran it.
When the program reaches the prompt to load svdq-int4-flux.1-dev.safedetectors and outputs' Done '.
The system is stuck or forced to shut down

Warning of very high system memory usage detected when stuck or forced to shut down

May I ask if the current version of the code requires extremely high memory usage and approximately how much memory is needed
Or whether there are memory management or leakage issues

[nitinmukesh]

Are you facing this issue all the time or it is happening occasionally.

I am running on 8GB VRAM / Windows 11 and it happens randomly. It happens during loading of model, if model loads inference is smooth.

[ziyaxuanyi]

Are you facing this issue all the time or it is happening occasionally.

I am running on 8GB VRAM / Windows 11 and it happens randomly. It happens during loading of model, if model loads inference is smooth.

always.

[nitinmukesh]

Try this
After this code add cpu_offload.

pipeline = nunchaku_flux.from_pretrained(
        "black-forest-labs/FLUX.1-dev",
        torch_dtype=torch.bfloat16,
        qmodel_path="mit-han-lab/svdquant-models/svdq-int4-flux.1-dev.safetensors",
    )

pipeline.enable_sequential_cpu_offload()

[ziyaxuanyi]

Try this After this code add cpu_offload.

pipeline = nunchaku_flux.from_pretrained(
        "black-forest-labs/FLUX.1-dev",
        torch_dtype=torch.bfloat16,
        qmodel_path="mit-han-lab/svdquant-models/svdq-int4-flux.1-dev.safetensors",
    )

pipeline.enable_sequential_cpu_offload()

But I don't think it's a problem with the VRAM. My VRAM has 24GB, which is sufficient. It's likely a problem with the CPU