Prompt per "block" instead of entire image

[poisenbery]

tl;dr
Is it possible to give us the option to specify prompts for each block of the upscale process instead of 1 prompt for the entire image?

One of the biggest issues with SD upscaling is when the AI converts a chunk of the image into the prompt the user specifies, instead of enhancing what is actually supposed to be in that block.

Let's say that your base image is a picture of a person with a sky and clouds in the background.

You set a 4x upscale with 512x512 and put "a person with a sky and clouds in the background" as the prompt. The AI will then break up the image into the "blocks" and apply the prompt to every block. What ends up happening sometimes is that an entire block will consist of sky and clouds, with no person. But since the prompt says "a person with a sky and clouds in the background," the AI will sometimes create a person out of the sky and clouds. This is not what the user intended obviously. They wanted the clouds to become more detailed, but since the prompt has "person," the AI puts a person where it does not belong. It becomes more apparent when there are more prompts and can happen sometimes at relatively low denoise values. AFAIK, the only solution for this is to increase the tile size or use lower denoise values.

Would it be possible to implement something where the user is able to specify which block gets which prompt?
For example, if the first 3 blocks only has clouds and sky, the user can specify that the first 3 blocks prompts are "clouds and sky." Then if the 4th block has a person, they can specify that the 4th block uses "a person" for that prompt, and ect so on.

I think this one addition would change the entire script from "SD Upscale" to "Automatic Inpainting"

[Jopezzia]

Hello! Already thought about it, but so far this is a huge amount of work. So it won't be happening anytime soon. But we will gladly accept the PR with this feature

[DaveScream]

Maybe just apply clip checkbox, so each tile will be upscaled with prompt returted by clip for this only tile?