Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update SD1.5 TI embedding output format #103

Merged
merged 1 commit into from
Apr 11, 2024
Merged

Conversation

RyanJDick
Copy link
Collaborator

This PR addresses #100 (based on the Discord discussion here: https://discord.com/channels/1020123559063990373/1130288610520875049/1227012392416841789 )

Prior to this change, invoke-training produces TI embeddings for SD1.5 in a format based on diffusers with the main key being the placeholder token:

>>> sd = load_file("bruce_the_gnome.safetensors")
>>> sd.keys()
dict_keys(['bruce_the_gnome'])

This format is currently failing to import into InvokeAI via the Model Manager. I suspect that this is caused by a change in behaviour during the recent MM refactor, but I haven't gone to the effort to try and confirm this.

The old diffusers format is not very popular, so instead I have switched invoke-training to use a different format, which uses "emb_params" as the main state_dict key.

Note that only SD1.5 is affected. The format used for SDXL TI embeddings is unchanged.

… we were using the diffusers format, which the placeholder token as the main key in the state_dict. Now, we always use "emb_params" as the main key. This format is more widely supported. Also, InvokeAI does not support the old format. I'm not sure if this is a recent regression in InvokeAI, or if this has always been the case.
@RyanJDick
Copy link
Collaborator Author

Test failures are unrelated to this change and will be addressed in another PR.

@RyanJDick RyanJDick merged commit 11160dc into main Apr 11, 2024
1 check failed
@RyanJDick RyanJDick deleted the update-ti-sd1-format branch April 11, 2024 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants