[how to remove cropping] Is resolution=1024 required? Can I not train a LORA with Dreambooth for SDXL with 1920x1080 images?

[dkebudi]

I would ideally want the output to be the same aspect ratio. Thanks everyone!

[asomoza]

Hi, the bests results with SDXL will be with the same resolutions as it was trained with, which are in the paper:

Since you're training a Dreambooth LoRA, this will probably won't have any impact though, DB LoRAs are really small and practically it doesn't matter the resolution you use if it's not lower or a bad quality one, this will only have an effect for finetunes.

With SDXL I can still generate an image like this one without a problem:

For example I did an experiment when SDXL came out with this dataset which is just 1024px square images.

And with that training I could still generate an extreme landscape image like this one without any problems.

For DB LoRAs is a lot more important the quality and description of what you're training than the aspect ratio of the images which the model probably won't learn with just a few images.

[dkebudi]

@asomoza thank you! I am using this script and it seems like it forces the model to be 1024x1024. I want to force my own aspect ratio.

With the UI HuggingFace generated I was hoping to be able to make enough tweaks to the under lying code such that I allow for different aspect ratios in trianing.

Currently, the way the infra is set up, it center crops my data to fit a square with padding, in this instance 1920x1920.

Any ideas?

Any help would be appreciated.

[asomoza]

That's a two part question, the script has some arguments to set the resolution and cropping of the images:

--resolution and --center_crop which seems you're using.

I don't really use the default scripts because I'm always testing new stuff, but if you want to change it so it can accept non-square images you will need to change the Dataset object.

This line is the one that does the resizing so you'll need to change it. If you're using the same resolution images you can practically just delete the resizing from the dataset.

make enough tweaks to the under lying code such that I allow for different aspect ratios in trianing

Just as a heads up, If you're struggling with just the image resizing the "different aspect ratios training" will not be easy for you, this involves some more complex logic where you will need to use bucketing and know how to resolve when the batches don't match, so you'll practically need to code your own dataset object and how to retrieve the images from it.

[dkebudi]

This is really helpful, thank you @asomoza.

Lastly -- given the Lora-Ease wrapper I am using, is there a way for me to manipulate the Dataset object that code calls? Asking since Lora-Ease is also Hugging Face owned. A path forward would be phenomenal.

I will exclusively train for 1920x1080 for the time being.

[dkebudi]

Gotcha on the long training and using all the images.

Yeah weird that tensor-board is not working. Primarily because I have changed the code to force pushing images to TB in every run. Nevertheless, anything easy that comes to mind right now? Where in the code can I check that the images going into the model for training are 1920x1080? I will just add a print statement.

[asomoza]

not 100% sure in that script but you should be able to print the shape after this line if you're not caching the latents:

https://huggingface.co/spaces/dkebudi/lora-ease-dk/blob/main/train_dreambooth_lora_sdxl_advanced.py#L2070

print(f"{pixel_values.shape=}")

The last two values should be the height and width.

[dkebudi]

Awesome.

I opted to put it here since I am caching: https://huggingface.co/spaces/dkebudi/lora-ease-dk/blob/3c525b4aecf4fa3eecd273c5bbb7bb451fa1a169/train_dreambooth_lora_sdxl_advanced.py#L1896

And it is telling me that the images are 1080x1920!

Thank you @asomoza !

[ScilenceForest]

@asomoza Hello asomoza, I would like to ask for the training of SDXL Controlnet, the size of the image that I need to generate is 20482048, and my dataset image is also 20482048, but I am training Controlnet at 10241024 size (resize the dataset image), and I would like to know in this scenario Should I train Controlnet at 20482048? sorry this is my first reply, I'm not sure if this question needs to start a new discussion, I'm sorry if it's bothering you or the author of the original question (and please make me aware of it)

[asomoza]

Hi, training with 2048 probably won't give you any additional benefit, even though SDXL can generate at 2048x2048, most of the time it doesn't do it good and training a controlnet won't fix that. Your best option would be to train in 1024px and also generate at 1024px and then do an upscale.

If you need further help it would be best to start a new discussion, mostly so people and maybe even yourself in the future can find the answers and information better.