HowTo: https://civarchive.com/articles/19108
UPD: New version of Loop + Scenario + Loras
The most powerfull workflow in this series, highly recomend update to this one.
Single precondition text
Single postcondition text
Unlimited scenario steps (5 by default but you can add)
Unlimited lora for all (1 by default)
Unlimited SEPARATE Lora per step (1 by default)
Color correction
UPD: New version of Loop + Scenario + Color is here!
Separate precondition
One-line scenario
Separate postcondition
Color correction is works now : )
Updated version of https://civarchive.com/models/1826257
Now you can use Scenario with different prompt for each step.
This workflow contain For-Loop so you can create as much step as you wish just copy-past Prompt for each step and change what you need.
Do not forget to change precondition (for ex. If you undress wooman and take off sweater do not desribe next step as "weared in sweater" - it confuse model)
Lighting: https://huggingface.co/Kijai/WanVideo_comfy/tree/main/Lightx2v
FastVideo: https://huggingface.co/Kijai/WanVideo_comfy/tree/main/FastWan
Wan2.2: https://huggingface.co/QuantStack/Wan2.2-I2V-A14B-GGUF/tree/main
Description
Add ability to create scenario
FAQ
Comments (56)
Thanks for sharing the workflow! Oh neat, I didn't realize there was a conditioning switch node. Would've saved me a lot of time while I was tinkering locally with the original lol.
Locally the changes I did was first add a florence caption based on https://civitai.com/models/1687498/wan-2221-i2v-2-workflows-merge-fusionx-lora-2-sampler-florence-caption-last-frame-color-match?modelVersionId=2061133
Then I added a conditional change for the concatenated "after text" based on the index value in your Total Step Count node. This all gets evaluated with some if else logic, and the correct after text for the index gets chosen. That cond index switch case you have is wayyy better though. Much simpler!
I also added some color correction between the outputs to the source since I was noticing it get lighter and lighter.
Can you add auto prompt for each step? I can't do it. It either gives a loop error or a black screen
Auto prompt? More context I didn't catch
thesharque588 when the program itself analyzes the picture and creates a prompt based on this analysis, and the user only needs to write the action he wants. The idea is for the program to take the last frame of the video and, based on it, analyze and create a prompt for the next stage. It seems to me that this will significantly improve the quality of animation and there will be less spontaneity
thesharque588 you can do it with ollama qwen2.5:7b instruct q4 k m is pretty good at this stuff with a good instruction.
I have made something similar but did not expand on it:
https://pastebin.com/5tHbVuYw
houdh235914 Autoprompt that I've tested just garbage : ))) in danbooru it's better but textual description - just cry and scream : )
If I want to use it for t2v what should I change?
Thanks for your contributions!
I think you need to remove Image / image clip, Change WanVideo intent creator to something without input image, but I didn't see the way how you will connect step 1 and step 2? IMHO only first step could be complettly T2V and than you take last frame and continue.
Fantastic! I had commented on your original workflow about the prompts. Thanks four your hard work. This works great and your workflow is very clean and easy to add/subtract the number of steps/prompts. I feel like a movie producer now : )
hi, I don't find the fastwan lora you use, can you put the sources in descriptions please?
finally found it https://huggingface.co/Kijai/WanVideo_comfy/tree/main/FastWan
All of my individual steps loop for some reason.
Check seed in first sampler same text + same seed ≈ same movie. Use random seed.
thesharque588 No I mean whatever seed the individual steps start and end in the same position
lorenzo881743 oh! Take a version with image clip! It should helps.
I like OP's workflow so I tinkered with it.
https://pastebin.com/hGPi0zpj
Figured I'd share in case it helps anyone. After some messing with it, I made some adjustments for my own preference. Everything I changed:
- Auto captions per each index (the basic auto caption stuff copied from https://civitai.com/models/1687498/wan-2221-i2v-2-workflows-merge-fusionx-lora-2-sampler-florence-caption-last-frame-color-match?modelVersionId=2061133 ). I adjusted things so it should be an auto caption from last image + the after text box you add per iteration
- Individual Lora options per iteration
- Changed out ksampler for clownsharksampler in case you want to use res samplers
- Added color correction node
- Added auto size that keeps image proportion so you don't get mat errors
I didn't go crazy on the organization with the get and set nodes but I'm no expert. Hopefully it helps someone out there though!
where did you get takingoffclothes lora?!
UPDATE TO MY ORIGINAL COMMENT:
Passing along the info I saw from https://civitai.com/models/1307155/wan-22-experimental-wan-general-nsfw-model?modelVersionId=2070122&dialog=commentThread&commentId=887813 that made the workflow I posted a ton better after some tweaks:
FOLLOW THE STEPS BELOW: 1: Don't use acceleration/enhancement LoRas (such as Lightx2v or other) on "high noise"; use only on "low noise" with strength 1. (I'm using this version of Lightx2v LoRa): lightx2v_T2V_14B_cfg_step_distill_v2_lora_rank64_bf16 2: Use 'NSFW/SFW' LoRas on both "high noise" and "low noise" models. On "high noise," use strength 2, and on "low noise," use half the strength, with strength 1. You can vary the strengths as you like, but always keep the "low noise" strength at half the strength of the "high noise." 3: Use these settings in KSamplers: "high noise": steps: 9 / cfg: 3.5 / euler/simple / start_at_step: 0 / end_at_step: 4 "low noise": steps: 9 / cfg: 1.0 / euler/simple / start_at_step: 4 / end_at_step: 10000
From testing, it helps A LOT with the temporal consistency amongst iteration to the point that I couldn't even tell a difference from light shifts and such from iteration to iteration. Seriously, try it out.
gumpbubba721291 Do I understand correctly that high noise end_at_step should be set to 4, and not 1 as you advised before?
houdh235914 Well, hm. I'm still experimenting on the best high noise / low noise step ratios and amount. I'm adding more steps overall than the original comment but I'm also running on an H100. I can't say for certain but also seems a dependency on the lora for what's best. I don't have a good answer quite yet. All I can say is that it seems like the tweaks made in the other person's comment are looking better than what I had before.
Thank you for adding the LORA steps. I got your workflow running, I removed the LLM stuff and went back to the conditioning Index Switch but kept most of the rest. It's so great to have custom LORAs for each step. I went back to sde/sgm_uniform for the time being but need to experiment with sampler/schedulers. I still have bongmath toggled to true. Does that do anything if not using Bong Tangent?
iodrg244 still experimenting but from what I've seen so far, the best experience I've had is going to linear/euler + simple on both. then I got lightx2v going into low noise only. removed the other help loras. could probably experiment more with those still. In any case, euler+simple has been pretty damn stable comparatively.
highnoise:
eta: .5, sampler: linear/euler, scheduler: simple,. steps: 12, steps_to_run: 6, denoise: 1, cfg: 3.5, seed: I added a seed node that randomizes each time, control after generate: randomize, sampler_node: standard, bongmath: true
lownoise: eta - 0, steps_to_run: -1, cfg: 1, control after generate: fixed, sampler_node: resample. other stuff same as high noise sample.
To your question: Yes... I think. I was looking at the documentation earlier. "BONGMath is an algorithm unique to RES4LYF that vastly improves sampling quality and coherence in most cases, with little to no effect on sampling speed. It has no effect when eta is set to 0. " That said, I haven't done tests to check for differences, but hey, if it doesn't hurt the time and can't hurt, eh, why not use it, right? lol
gumpbubba721291 consider posting your own workflow with updates :)
awesomesnipee there are still some things I want to add onto it, like options for face detailers and nodes for specifying the direction drawn in (something I saw in an AFO workflow but need to learn more). Once I got everything where I want it, I'll consider posting it later if I complete it 🫡. But for now, there aren't really a whole lot of changes I've done since my last.
gumpbubba721291 I hope everything works out for you! In my opinion, this is one of the best (or the best) workflow on this site!
gumpbubba721291 sounds great, we need nicely optimized, de-facto AIO loop workflow for wan 2.2 🫡
hi, you should try using "setNode" and "getNode" when you can, it's game changer for clean workflows
argotic I know about it. I think I even made a comment in the workflow acknowledging it, but I was too lazy to do it since I was working through things fast LOL. In my current version I used it more to clean it up
gumpbubba721291 ok my bad. I tried yesterday the comment you saw, about putting acceleration lora on low noise only, change steps etc. I tried it again I obtain very bad quality, burned picture.
gumpbubba721291 Strength 2 for the loras seems a lot, sure the Loras with such strength will work great but if you have 3 or 4 of them activated, isn't it the best way to lose face consistency ? I'm just wondering...
Skuuurt I mean, each lora is going to have it's own best strength level. per the following sentence: "You can vary the strengths as you like, but always keep the "low noise" strength at half the strength of the "high noise."
gumpbubba721291 gatcha, you were talking about ratio and not actual numbers per say, thanks a lot for your time and your crystal clear explanation ;) gotta test your settings !
gumpbubba721291 also, do you know if there is an actual difference in rendered quality between lightx2v_T2V_14B_cfg_step_distill_v2_lora_rank64_bf16 and lightx2v_T2V_14B_cfg_step_distill_v2_lora_rank32_bf16 (rank64 and rank32) ? Until now, I am using rank32 which is lighter
Skuuurt I haven't ran any tests on it myself. From my shallow understanding of it, lower rank is going to be a smaller file and faster but lower quality (though you also need to take into mind compatibility of things, if you get errors). I guess the only way to know is if you try both and see what you like 🙂
Skuuurt AFAIK there just a diff like in Q2 vs Q8 - bigger = better + slower : ) But in some place I've found comparation video (lost link) about 1.3B fast-video diff rank, and actually I didn't catch what exactly diff - there no linear changes like (bad -> best)
awesomesnipee houdh235914 Pastebin kept stopping the upload so I put it here. Will probably just add more versions on it as I clean it up or add more features like vace + spline integration
Skuuurt Check this out... https://huggingface.co/Kijai/WanVideo_comfy/blob/main/Lightx2v/lightx2v_lora_rank_comparison.mp4
gumpbubba721291 I also try Wan2.2 WanFunVideo - but... it's broken for the moment. The Idea was simple - create small long video -> extrapolate to big use small. (TLDR: Wan2.1 - doesn't work like that)
gumpbubba721291 appreciated!
@gumpbubba721291 Add your workflow to "suggested" because many who ask about "independed loras" : )
Thanks for the workflow. Is there anything i can do to prevent the face from changing from scene to scene? cause after 3 steps the face is changing. not all the time. When the last frame of a step if the girl have her head down or look to the side the next steps is a different face.
Check your prompts. I did a couple generations with multiple steps with this workflow and had consistency across segments.
use reactor
Have you tried using a face replacer node pointing towards your original image? It can sometimes make the faces look "painted on" but sometimes it can make it look better. I typically keep it off for that reason, but I suspect it's more so my configuration somewhere down the line.
One thing to be aware of is if in the last frame the face isn't visible, you're going to get a different face since it doesn't have the original in the last frame to begin the next step. I try to work my prompts so the face is visible but when it isn't you might need to manually choose a frame. Maybe a future model will allow us to post a reference image of a character separately from the first/last frames for consistency sort of like LLMs have system prompts to help them.
Thanks for making this, you're pretty clever. I do have a question though. When I pick an image to use as the starting image, it doesn't really copy what she looks like or the room she's in. Does the starting image do much or is it like a 'hint'?
AFAIK The Wan model just take your first picture as starting point but rewrite it. Why I thing so? Just take any random picture from internet like "bus on the road" and try prompt "girl take off her clothes" 😂
wow this actually worked, quality could be better and there's a clear degradation on each new scene, but as a concept it's amazing and i hope you keep developing it, amazing workflow, highly recommended.
I got error "For loop-terminated
'78.0.0.3.0.0.63'", only run the first time, second one raised this error. Any idea?
Update ComfyUI - Same was here. Update ComfyUI, update modules, reboot - and... it desapear.
And check --no-cache (or something like that)
Is here a way to add frame overlap between steps to improve consistency?
I didn't find the way... But I think you could decode (img->latent) few last steps and add some emty latent (concat latent) I have Intel i9 arc - so 256x256 - 61 frame is my limit : ))) Overlaped frames is not my story : )
Is there a way, and what would I add, to make the video more clear? It seems to get blurry with fast motion.
steps - It's just a 3+3 steps and Lightining, you can remove it and use 12+12 steps. Spend time -> better quality