Inference speed get slower (Using two different pipeline with multi gpu)

[KihongK]

I am developing a video generation project on an 2-GPU server using two different pipelines:

• FluxPipeline for image generation.
• CogVideoX-Fun for image-to-video (I2V) generation.

The process works as follows:

1. Incoming generation requests are added to an image generation queue.
2. Once the image generation is complete, the request moves to the I2V queue.
3. If multiple requests are received, both pipelines run simultaneously on different GPUs, with GPU IDs assigned separately.
4. However, when both pipelines run simultaneously, the FluxPipeline slows down significantly.

I am looking for solutions to resolve this performance degradation.

[ukaprch]

Are they running on separate threads? Sharing the same queues? 2 GPU's and 1 CPU may not always equal 2X speed? Bottlenecks in memory on the PCI bus? You might want to run a performance profiler to see where any bottlenecks are.