Ever a possibility for a model that attempts to correct or replace Text written by Stable Diffusion that is jumbled?

[CCpt5]

Hello,

I know this is probably not possible at this time, but I wanted to mention it just in case. It's well known that Stable Diffusion (and most of the AI txt to image tools) don't properly generate text. They put letters/numbers/symbols where text should be, but it's usually jumbled/etc.

I use Lama a lot to remove text from images SD generates that end up w/ text in them. Removing the text is pretty easy w/ Lama, and doing so makes images that had text look less obviously AI generated. That said, it'd be amazing if in the future a model trained on letters/fonts/etc might be able to recreate/replace text that SD has incorporated into an image and also maintain a similar style to how it appears on the image already (which is usually aesthetically correct).

Just thinking out loud. If anyone has thoughts on how likely this functionality/type of model may be in time I'd love to hear.

Thx.

[CCpt5]

Here are a couple examples of images that I generated today w/ SD that I would love to be able to modify the non-sensical text on while keeping the font/style.

[Sanster]

This is a very interesting idea for ai-assisted creation, after a quick search I found this project https://prasunroy.github.io/stefann/

[CCpt5]

Wow! That's exactly what I'm hoping for.

Is there any chance this could be incorporated into a future version of Lama?

[Sanster]

Did a quick test on your image, I would say the result is not good, and actually the 'replace process' is not easy(there are many algorithm parameters that need to be adjusted).

It would definitely be a good feature to have in Lama Cleaner if there is a model that works better and is easier to use. Let's wait and see.