StableDiffusionControlNetPipeline.controlnet.parameters() are not getting updated during training.

[venkateshtata]

In ControlNet architecture, the trainable copy of UNet from StableDiffusion (The right side network in the figure from ControlNet paper) can be accessed through StableDiffusionControlNetPipeline.controlnet right ? Or is it the StableDiffusionControlNetPipeline.unet ?

Also what is the most efficient way to get the capture the output latents for ControlNet that hold the condition image representations ?
ControlNet Alternate to below Unet version:
noise_pred = pipeline.unet(noisy_latents, torch.tensor([desired_timestep]).to("cuda"), encoder_hidden_states=text_embeddings).sample

[tolgacangoz]

Exactly which code are you using for training? These ones? Right, the trainable part is controlnet not unet.