Switching ControlNet and LoRAs

[haihp02]

Hi everyone, i am new to stable-fast, i have some questions that i hope i can get answers from you guys.
Currently i have a project that run on a single diffusers 's StableDiffusionXLPipeline,i also have another Im2ImagePipeline share all the component (initialized from the above Text2Im pipeline with .from_pipe() method). On top of these 2 pipes, i have 2 controlnet (each have a pipe for it own) and 3 LoRA (switch between LoRAs with set_adapters() method). My question are:

• Since all my pipes share most of the components (unet, te, vae) i only need to compile the StableDiffusionXLPipeline right? Or i have to compile each of them individually?
• If i need to compile them individually, will switching LoRAs takes long time, because i want my users to be able to generate with different styles, too long compiling will be bad experience and i only have 1 24GB GPU so i can't run multiple pipes in paralell.

Excuse me for my poor English. I very much appreciate any help. Thanks for advance.