v3.0 Is Finally Here!
Entirely new layout with grouped step-by-step sections.
New detailed Notes section with links to essential model downloads.
Added Res4Lyf custom node pack for Sigma previews and access to more sampler/sheduler combinations.
Thanks to BillyWicked for their collaborative efforts in testing and refining this version!
The workflow is still fully capable of NSFW content if you wish to use it for those purposes. Feel free to share any of the videos you are able to generate using this workflow!
I HIGHLY RECOMMEND READING THE FULL INFORMATIONAL ARTICLE REGARDING THIS MAJOR UPDATE - https://civarchive.com/articles/21684/torstens-wan22-i2v-gguf-workflow-version-30
v2.0 - still simplified and very efficient
added more quality-enhancing features without impacting rendering speed!
v1.0 - basic workflow, extremely fast, not as many features
TL;DR
It's a great workflow, some have said it's the best they've ever seen. I didn't say that, but other people have. I think it's the best workflow anyone has ever uploaded. You know what we need on this platform? We need to Make Workflows Great Again!
Description
After several weeks, and dozens of hours tweaking and understanding each parameter of the previous workflows, I'm proud to announce: VERSION 3.0 IS FINALLY HERE!
What's new in this version:
Added RES4LYF node pack. This allows users to preview and adjust de-noise schedules and sigma values, enhancing user control over the quality of their video generation!
Created grouped sections to break the process down into easy steps
Created a detailed instruction manual to the left of the WF
Re-worked the order and formatting of some node in/out links to prevent video errors
FAQ
Comments (53)
Thank you, Sir, for your contributions. I have a 12GB VRAM card and have been using Torsten's v2.0 with great success. 5s vids take about 4–8 minutes to generate, depending on resolution.
Been looking forward to v3.0. Just downloaded it tonight! First off, I must say, it looks amazing. I love the layout, organization, and notes—they are greatly appreciated.
That said, it's taking about twice as long for videos so far. Could this be because the lightning LORAs are not used this time around?
In any case, thanks again!
make sure you re-download it if you got the file within the first few hours. I've made some last-minute tweaks and updated the file as soon as I could
@TorstenTheNord Guess I jumped on it too fast! Yes, I downloaded the v3 update, and speed is back and actually faster... loving it! Thank you!
Also been using it a lot, tweaked it a little tough but getting similar times on 12Gb 4070
As for the samplers and sigmas why ure not using the moe custom sampler where u can insert the sigma boundary? 0.875 for t2v and 0.900 for I2V
I just like the simplicity of KSampler nodes. Feel free to add/remove nodes as you see fit!
First of all V2 is awmazing, want to test V3 now but wondering where you got the wan_lightx2v_high_noise_model.safetensors? I'm guessing you renamed it?
he put the link on her workflow, yes he renamed
@KuraW Yea I missed that but it was what I expected :p
Can I just ask what the point is of saving the starting image for every output? Saving the last frame seems more useful so you can use that to continue the clip.
@Asaghon I do it because I run multiple iterations of the image using different prompting techniques and/or different samplers, schedulers, and shift values for testing. if I start with the same image each time, I can use the scientific process to document and adjust for changes
what lora is the detailz-wan.safetensor? I dont find its link, can someone send the link
It’s a wan2.1 Lora that still works in Wan2.2. Just keep the strength of Detailz Lora on the lower end.
link: https://civitai.com/models/1385506/detailz-wan-detail-enhancer-for-wan-videos
This may sound weird but could this work on old GTX1080 8Gb VRAM?
yeah, with 8GB RAM on an older card like that I suggest using the Q4 quantized (GGUF) models and instead of using the full fp8_scaled text encoders, there are quantized (GGUF) text encoder models you can plug into the formula as well. Your GTX1080 will likely have significantly slower generation times as well as some potential loss in quality and prompt adherence because of the structure and age of the technology (GDDR5X) behind the current generation (GDDR7)
@TorstenTheNord Thank you for your answer, I might give it a go just to try, before upgrading and see what the potato-pc can output.
@TorstenTheNord: Do you have any suggestions about adding a face swapper within your v3 workflow to maintain face consistency across clips? I've been trying to modify your workflow to include the face swapper from this workflow, but the workflows are so different that I cannot make sense of it.
I haven't been able to get face-swap or face-restore nodes to work 100% of the time. Therefore, I do not add them to my publicly released versions of my workflows. You might want to try disconnecting the Color Match node to utilize a face node more effectively, or set up the face-swap/restore nodes in a separate post-processing flow/extension.
@TorstenTheNord Thank you for the input, Torsten!
Hello. I've been trying to use this workflow on my 12GB 3060 but I'm still getting OOM errors. I'm using Q4 and tried using lowvram and novram params and making sure there is nothing else on my PC that's using the GPU and it's still not enough to get it to work. Could you share any advice for my situation? lmk if you need any particular information or spec.
I have experienced this before when my ComfUI got too bloated with other models and cached files/images. I would recommend a fresh installation of ComfyUI Windows Portable, or alternatively a fresh Manual Install of ComfyUI with its own virtual environment. Make a backup of ONLY your most important models, unet, text encoders, and vae. Everything else (especially Custom Nodes) should be completely deleted until a new fresh install is complete. Make sure you also delete temporary files and shader cache from your computer system prior to reinstalling ComfyUI.
@TorstenTheNord Sorry for the late reply. I have basically just made a fresh comfyui-cli installation not too long ago, but I probably already overloaded it with too much stuff. I'll make a new clean install today after I do some maintenance on my PC, I am unsure how to clean the temp and cache files, but I'm sure I'll be able to find something in comfy's documentation. I'll let you know how it goes afterwards.
Just one last question. Do you recommend the portable version of Comfy over venv version?
@Renaphine I usually recommend a virtual environment, because it prevents "system creep." That's where your python dependencies can "creep" into the rest of your computer and accidentally override or overwrite other necessary dependencies in your computer. If you work with multiple AI tools and programs, it's especially important to be mindful of that. Therefore, a Virtual Environment (Manual Install) of ComfyUI is my recommendation. It creates a completely separate environment for that specific system and set of files. It blocks "system creep" by requiring that Virtual Environment to be activated before any files can be added or removed within it.
@TorstenTheNord It's been a while since the last time I tried working with this workflow. I monitored it with task manager after my last comment and it turns out the VRAM was overflowing and the amount of RAM I had free to use as "shared GPU Memory" was also filling up entirely (I had been running Comfy and everything AI on 16 GB up until this point). Today I got a new RAM stick to double my RAM to 32 GB which was long overdue anyway, and now the whole workflow works like a charm. I'm leaving this here in case anyone else has the same issue as me with the OOM errors.
Thank you so much for the feedback, recommendations and the workflow, really amazing work!
@Renaphine I’m also on a 3060 12gb with 32gb system ram.
Can you run this workflow more then once? If I run it until it completes, then tweak the prompt and attempt to run again, it crashes comfyUI turning the window completely black. Sometime screen turns black.
It also seems to be holding GPU ram at like 11gb and system ram at like ~20gb after finishing.
So I think it’s a memory problem, but at the time it crashes I don’t see either of them spike in task manager, but I feel like it prolly just happens too fast to be seen.
@chunksmobile756 try updating your Python Dependencies if needed, and/or you may need to delete everything (keep a backup of your most important LoRAs, diffusion models, unet models, vaes, and generated content) and re-install with a clean copy of ComfyUI Portable or Manual Installation methods. I recommend manual for reducing these types of issues. CogniBuild aka Get Going Fast on YouTube is very informative on how to do it.
After trying several other wf I came back and still prefer this one. I'm still not sure how to interpret the graph though. Am I supposed to be aiming for something? I did make some tweaks to the layout for my personal preference.
It means a lot to know that you’ve found this workflow to be preferable!
The Sigmas Preview graph is a visualization of the de-noise process. If you're disabling the Lightning LightX2V Lora, you can play around with more schedulers such as Bong Tangent and Beta57. To adjust for better quality with less noise, you can change the Model Shift weights (make sure they are equal in both High and Low noise settings). Your scheduler with a proper shift weight should make the graph look something like the phot I posted in the Full Info Article about this version: https://civitai.com/articles/21684/torstens-wan22-i2v-gguf-workflow-version-30
***If you are ENABLING the LightX2V Speed/Lightning Lora, I do not recommend changing model weights or schedulers too much***
Firstly, Thank you for the amazing work! Been using your workflow since V1. Your work is fantastic! I tried V3 and increased the number of frames to >120, the video seems to loop back to original state in the end. I'm using the workflow with lightx included and followed your instructions on the workflow. Is this expected?
Firstly, Thank you for the amazing work! Been using your workflow since V1. Your work is fantastic! I tried V3 and increased the number of frames to >120, the video seems to loop back to original state in the end. I'm using the workflow with lightx included and followed your instructions on the workflow. Is this expected?
Thank you for the kind words! To prevent loop-back, make sure you've enabled the RifleXRoPE nodes for both high-noise and low-noise sections. Try to increase or decrease the value by increments of 1-2 until you get satisfactory results. Wan models tend to try to loop back no matter what, and RifleXRoPE can lower the amount of loop-back. It may not get rid of it completely, but you should end up with more usable frames if you're looking to do a sequential video generation (like telling a story through multiple videos strung together for a final project)
@TorstenTheNord Thank you for the guidance, will try this out and report results. I am noticing after the latest NVIDA driver updates the Clean VRAM and Clean Cache nodes cause the workflow to terminate on 5090. Not sure if anyone else has experienced this. It prints a message about Comfyui trying to install NVIDA drivers and the workflow is terminated.
@iamstreamin that's odd. Luckily you don't NEED to enable those VRAM and Cache cleaning nodes as long as you keep randomizing the seed or changing some parameters in the flow on a fixed seed for testing.
I may also recommend updating your ComfyUI. If that doesn't work, you may want to try updating your Python Dependencies... but be careful with Python Dependency updates. Depending on your specific version of ComfyUI and how you installed it, they may try to overwrite each other which would then require a re-installation. If you have to re-install, try the Manual method with virtual environment if possible.
@TorstenTheNord based on some light research, it appears to be an issue with 50 series cards. I moved the clean nodes to the end of the workflow after the 30fps creation and the issue is infrequent now. I played around with the values in the Rife node, it was initially not fixing the issue. I changed the prompts to be more descriptive and logical (i.e. the characters movement in sequence) and changed the Rife values to 2 and 2. The looping seems to have stopped, tried for 12 sec videos.
Your workflow is superb, it's become my top one for I2V. My RTX 4070 SUPER and I thank you!
Will you make a similar one for T2V?
I’m considering making one for t2v based on what I now know! I’m just a hobbyist with a busy full time career so it may take a bit to get it out there publicly. There are multiple new projects in the works that I want to do.
I think instead of 2+2 for hi/low, you should use 2+3. Change shift to 6.
Turn off detailz lora at hi, and raise to 0.6 at low (beause it is a 2.1 WAN Lora).
@vicaut I’ve adapted my steps and lora strategies recently. Enable LightX2V on both high and low, 8 steps total, (4 high, 4 low) dpmpp_2m_sde sampler with beta scheduler, plus a very detailed prompt with descriptive instructions for realism, camera lens and angle…. I barely need any detail-enhancing loras anymore.
@TorstenTheNord 8 steps take too long, maybe you can add sage attention for possible speed up?
I think i have some good values: Remove detailz lora. For Hi: use 3 Steps, Euler beta57, CFG 1-2, Lightning lora at 0.75 strength, model shift 4.5; for low: use 6 steps, Euler beta57, CFG 1, Lightning lora at 1 strength, model shift 5. Install sage attention and triton and start comfyui with --use-sage-attention parameter. On a 3070 this takes about 17 Minutes for a 7 second video. 544x960 resolution
@vicaut The reason I use 8 steps is because for my specific project that I'm currently working on, I am looking for maximum quality out of the Quantized models (currently using Q8 on my 16-GB NVidia RTX 5070) - but your values seem like a good balance as well, so if it works for you, then go for it! I'm happy that people are getting some versatility with my workflows, and using it to test new ideas and methods!
@TorstenTheNord if you want best quality you should use res_2s (or res_2m for hi and res_2s for low) with bong_tangent. But computation will take significantly longer.
You can calculate the correct model shift value S, depending on your total steps and when you want to change from hi to low. This should happen at Sigma=0.9; t = 1-(Value of Change to low after this chosen Step Number/total steps); S=Sigma*(1-t)/[t*(1-Sigma)]. Simplified: For Hi shift: 9*desired high steps / desired low steps = correct model shift value for I2V Hi, set the shift value for low to 5 (regardless of your number of low steps)!
I have tried installing windows portabel comfyui, windows version , downloaded different models, downgraded power lora..
Can someone help me out on this ..
Ok got it running.. changed the unet loader node to GGUFLoaderKJ
@rajivenator639 yeah, I believe the other GGUF loader is now deprecated based on the new Comfyui-Front-End version and Node2.0 update. I had to use the same fix on mine.
thanks
thanks
The first workflow that worked on my poor man's machine 2070 + 32GB RAM. Of course very small videos (848 x 480), using Q4_M, and it takes around 15 minutes, but it works, if someone in the same shitty setup as me is curious.
"Workflows The Epic : I tried a lot... and then I found one - the End." Just wanted to give you my thanks man! It works very well while not being convoluted and it feel really comprehensible for fresh starters.
So great and easy to use! Thank you! Any plans for updates in the future?
When running the RIFE VFI (Video Frame Interpolation) node, the GPU barely uses any resources, yet it consumes most of my memory. Is this normal?