V1.0 was setup with Load Model Only Loras. V2.0 have Load Lora with Clip and added nodes to be able to add a second Lora.
This is a simple COMPACT and easy to use WAN 2.2 workflow that I put together. I was tired of opening workflows and be greeted by a convoluted mess of boxes and nodes so I created my own. I put in Loaders, LoRA loaders, Prompts, 3 Samplers and Skip Layer Guidance in one simple arrangement that easily fit in one screen with no subgraphs.
I think it is relatively fast, even without Lightning LoRAs. My setup is a somewhat weak GPU (3070 Ti 8Gb VRAM) and I was able to consistently render 5 seconds videos at around 700 seconds (~12 mins) at 512x512 resolutions or any resolution with the same amount of pixels. I was able to push it to 10 seconds but it takes significantly longer.
Feel free to play around with it and if you like it, show us your results here! Also a like is always appreciated!
Description
The V1.0 was setup with load Lora Model only. For this version I've changed it for a Load Lora with clip and added nodes to add a second Lora
FAQ
Comments (22)
What is a benefit of triple sampling instead of duble?
Because of the poor processing power on my GPU, all workflows that were 2 samplers resulted in fuzzy/blurry video generations. Searching around the net, multiple sources said that 3 sampler method could alleviate those quality problems. As I understand it, it gives more time for the AI to refine the quality of the video.
Thanks for asking the question, I was curious about that too.
This is really cool, I'll definitely give this a shot today. Thanks for sharing!
My pleasure my friend, there's 2 versions, if you need more lora nodes use V2.0, actually version 2.0 is better, the prompts are loaded through the lora instead of skipping them straight to the samplers
Did it work? This looks like a simplified way of smoothing things up! Thanks for the info man.
My ComfyUI keeps failing to load the GGUF models, it says there is an invalid node ID. Any solutions?
I haven't had that error happen to me but searching around I found that it's commonly missing or badly installed custom nodes, do you know which node is faulty, it's usually highlighted in red?
Just replace the Load GGUF nodes with regular Load Diffusion nodes and hook them accordingly. This also got rid of a weird bug with the LoRA loader for me.
Sorry if this is basic, Other than adding a single lora for the high and low, can I add multiple pairs or Loras in this, Any help with that?
thanks
Hi there! Yes you can, you just need to connect them the same way the first two are connected. Remember though, every LoRA you add to the workflow you add complexity and it's going to slow down generation time and resources needed.
Hi, I have a few questions regarding this workflow:
1) Why is the "Lora Load" node that comes after the second "UNet Loader (GGUF)" is clip-connected to another LoRA? In other words: Is there a specific reason why the CLIP is connected in a zig-zag way between those 4 "Lora Load" nodes instead of having two separate paths for low and high noise?
2) What do the “High SLG”, “High SLG 2”, and “Low SLG” groups do? I roughly understand what the "ModelSamplingSD3" node does, but what about the "SkipLayerGuidanceDiT", "UNetTemporalAttentionMultiply", and "CFGZeroStar" nodes?
3) Why does this workflow use three KSamplers, while most workflows use two: one for high noise and one for low noise?
Thank you.
Hi there!
1) The model and loras are loaded in 2 nodes in their own specific instance, 1 for high noise and 2 for low noise. As for the clip, it zig-zags between all the loras because only 1 node can go into the positive prompt and 1 node in the negative prompt, so yeah the clip goes around all loras before going into the rest of the process.
2) The Skip Layer Guidance is a way to direct the AI to ''skip'' certain parts of the composition. In this instance I directed it to focus more on structure to the ''middle'' frames of the video. It's a way to alleviate artefacts, fuzzy or blurry body parts like hands and face. It's not perfect but usually helps generate better looking videos on WAN. Also provides a small boost to the speed of the generation since is skips some of it.
3) Because of the poor processing power on my GPU, all workflows that were 2 samplers resulted in fuzzy/blurry video generations. Searching around the net, multiple sources said that 3 sampler method could alleviate those quality problems. As I understand it, it gives more time for the AI to refine the quality of the video.
@CarterMac Thank you for your reply! Is there any place to read more about "Skip Layer Guidance" and how to adjust it according to my needs?
Thank you very much for your workflow, just having an issue running, keep getting the following errors:
CLIPLoader: - Value not in list: clip_name: 'umt5_xxl_fp16.safetensors' not in [] UnetLoaderGGUF: - Value not in list: unet_name: 'DasiwaWAN22I2V14BTastysinV8_q8High.gguf' not in [] UnetLoaderGGUF: - Value not in list: unet_name: 'DasiwaWAN22I2V14BTastysinV8_q8Low.gguf' not in [] LoraLoader: - Value not in list: lora_name: 'Wan22_ThroatV3_High.safetensors' not in [] LoraLoader: - Value not in list: lora_name: 'Wan22_ThroatV3_Low.safetensors' not in []
I tried using "manage" to install missing items but just comes up blank. Sorry I am very new to this :)
I googled each and downloading them, I think I know which folders to put in:
umt5_xxl_fp16.safetensors - ComfyUI/models/diffusion_models/
DasiwaWAN22I2V14BTastysinV8_q8High.gguf - ComfyUI/models/unet/
DasiwaWAN22I2V14BTastysinV8_q8Low.gguf - ComfyUI/models/unet
and the rest Loras, is that right.
Sorry for the hassle.
@Dracken1986 Hi there, no worries, we're all new at this at some point, So yeah the .gguf files go in unet folder, but umt5_xxl_fp16.safetensors goes into text_encoders folder, and yes, loras goes into the lora folder.
@CarterMac Thank you, I will try now, I ended up putting the umt5 into clip >_<, will try a prompt now, I was getting some odd results but nothing like what was in the prompt so maybe moving the file to the right place might help. (but honestly thank you for your time, nice to have actually support when new :)
@Dracken1986 My pleasure my friend, hope it works!
素晴らしいワークフローをありがとう。ノイズがとても改善された。しかし、まだ、少しのノイズがある。ノイズを減らすにはどうすればいいかな。
Google translate told me you have a bit of difficulties with noise, I would suggest increasing the number of steps. It will take longer to generate videos, you'll have to experiment between the number of steps (Time) and Quality. Increase the number of steps by numbers divisible by 3... you know, 12 (4+4+4) 15 (5+5+5) etc...
@CarterMac ありがとう。やってみる。