IPadapter: multi IPAdapter and multi input

[thibaudart]

hi

since a while, i use on comfyui a workflow with multi ipadapter (mainly one for face and one for style with different ipadapter model, different weights and different input image).
moreover for the style one, i use a folder with 5 to 25 images.
Results achieved with that ensure a very coherent style (like a lora) and very great consistency for character (face).
replicating this process in pure diffusers could be very useful.

[asomoza]

there's an issue for it (#6318), you can follow the progress. I know you posted there but is to link both conversations.

[yiyixuxu]

Hi @thibaudart

the style one, i use a folder with 5 to 25 images.

For multi-inputs , do you mean that you would need to pass multiple inputs to the same IP adapter and split the weights equality?

e.g. would below API work?

pipeline.load_ip_adapter(["h94/IP-Adapter"], subfolder=["models"], weight_name=["ip-adapter_sd15.bin", "ip-adapter-full-face_sd15.bin"])
pipeline.set_ip_adapter_scale([0.3, 0.7])

images = pipeline(
    ...
    ip_adapter_image=[[style_img0, stype_img1], face_img],
    ...,
).images

it will be same as

pipeline.load_ip_adapter(["h94/IP-Adapter"], subfolder=["models"], weight_name=["ip-adapter_sd15.bin", "ip-adapter_sd15.bin", "ip-adapter-full-face_sd15.bin"])
pipeline.set_ip_adapter_scale([0.15, 0.15, 0.7])

images = pipeline(
    ...
    ip_adapter_image=[style_img0, stype_img1, face_img],
    ...,
).images

[thibaudart]

yes seems good! T Le sam. 13 janv. 2024 à 19:15, YiYi Xu ***@***.***> a écrit :

…

Hi @thibaudart <https://github.com/thibaudart> the style one, i use a folder with 5 to 25 images. For multi-inputs , do you mean that you would need to pass multiple inputs to the same IP adapter and split the weights equality? e.g. would below API work? pipeline.load_ip_adapter(["h94/IP-Adapter"], subfolder=["models"], weight_name=["ip-adapter_sd15.bin", "ip-adapter-full-face_sd15.bin"])pipeline.set_ip_adapter_scale([0.3, 0.7]) images = pipeline( ... ip_adapter_image=[[style_img0, stype_img1], face_img], ..., ).images it will be same as pipeline.load_ip_adapter(["h94/IP-Adapter"], subfolder=["models"], weight_name=["ip-adapter_sd15.bin", "ip-adapter_sd15.bin", "ip-adapter-full-face_sd15.bin"])pipeline.set_ip_adapter_scale([0.15, 0.15, 0.7]) images = pipeline( ... ip_adapter_image=[style_img0, stype_img1, face_img], ..., ).images — Reply to this email directly, view it on GitHub <#6544 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AYWNXDEK7HP2I4DZJKUAPEDYOLFM7AVCNFSM6AAAAABBXQ4UI2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DCMJZGU3DK> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[sayakpaul]

Can we also try marking the answers if that's the case? :)

[Nomination-NRB]

hi

since a while, i use on comfyui a workflow with multi ipadapter (mainly one for face and one for style with different ipadapter model, different weights and different input image). moreover for the style one, i use a folder with 5 to 25 images. Results achieved with that ensure a very coherent style (like a lora) and very great consistency for character (face). replicating this process in pure diffusers could be very useful.

Hi, I am also interested in multiple IPAdapter control, can you share your comfyui workflow