TxT2Vid - content blurring out after first frame and gets progressively worse

[TheShadowNYC]

First off, absolutely fantastic work Volotat!

When generating via txt2vid I'm getting blurred results after the initial image. Attempted to try different models (inpainting included) as well as combos of different samplers/VAE/etc. Prompts are straight forward too and even when trying to add "in-focus, sharp, focused, etc" doesn't impact the result. Seems something else is going on or perhaps interacting with my setup somehow? Any thoughts? See samples below.

2023-05-12_08-46-18.mp4

2023-05-12_08-30-35.mp4

[volotat]

This may happen with the fixed seed. Make sure it is set to -1 as it is not automatically resets by the next frame as it should.

[TheShadowNYC]

Hi Volotat, thanks for looking into this further. Can't seem to replicate your result. I'm still seeing blurring occurring after frame one. Latest version throws a error now when CN is used btw... They released new version today with new "Reference Only" method.

"\controlnet.py", line 1063, in process
raise ValueError('controlnet is enabled but no input image is given')
ValueError: controlnet is enabled but no input image is given"

Could you be kind enough to confirm your model/VAE/ControNet setting used? I'm not changing anything else in your plugin settings btw.

Here are some results using same prompt but varying use of CN and models (RealisticVisionV20 and inpainting RVV20)

2023-05-14_16-13-55.mp4

2023-05-14_16-42-04.mp4

2023-05-14_16-45-24.mp4

[TheShadowNYC]

posting test from latest version today... still seeing blurring after couple frames. Anything I should try changing in settings?

2023-05-16_13-45-28.mp4

2023-05-16_13-22-50.mp4

[volotat]

Could you share your parameters? You can get them through the 'export' button below. And what model are you using? I need to reproduce this issue on my side to be able to fix it.

[TheShadowNYC]

Appreciate you looking into this!

Here is export.... and file it made. Using "RealisticVisionV20 safetensors" or "v1-5-pruned.safetensors" and Auto VAE or "vae-ft-mse-840000-ema-pruned" which gave same results.

2023-05-16_16-33-58.mp4

{
"prompt": "close up of flower",
"n_prompt": "text, letters, logo, brand, close up, cropped, out of frame, worst quality, low quality, jpeg artifacts, ugly, duplicate, morbid, mutilated, extra fingers, mutated hands, poorly drawn hands, poorly drawn face, mutation, deformed, blurry, dehydrated, bad anatomy, bad proportions, extra limbs, cloned face, disfigured, gross proportions, malformed limbs, missing arms, missing legs, extra arms, extra legs, fused fingers, too many fingers, long neck",
"steps": 15,
"sampler_index": 0,
"cfg_scale": 5.5,
"image_cfg_scale": 1.5,
"fix_frame_strength": 0.35,
"seed": -1,
"height": 512,
"width": 512,
"fps": 12,
"sdcn_process_mode": "txt2vid",
"processing_strength": 0.85,
"length": 10,
"ControlNets": []
}

[TheShadowNYC]

also if helpful at all....

Python 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)]
Version: v1.2.1
Commit hash: 89f9faa63388756314e8a1d96cf86bf5e0663045

torch 1.13.1+cu117

[Lalimec]

how is this issue going?

[TheShadowNYC]

checked again... similar results... still blurs after first frame unfortunately. @volotat any updates on other settings to try and change on my end to fix?

BTW Volotat, shout out to you over in Yerevan! One of my best friends is based there (traditional 2D animator has his own studio).

[lauretta91]

Have you tried increasing the Fix Frame Strength ?

[TheShadowNYC]

oh wow, this works now! Have to find time to test it more. But mark this solved! Thanks

[jfranecki]

What did you do to fix it? Was Fix Frame Strength too low?

[TheShadowNYC]

yes, that was new setting in recent version it seems, I got fixed results after increasing to 7-8... too high and it became erratic movements but not blurry still. Adding Control Net source really helps too! Can't wait to play with it more.