This is my comfy workflow for using my NF4 quants of Wan2.1. Requires a few custom nodes, so use Manager to install them.
I highly recommend the Sage Attention version if you can run it, as it runs about 25% faster.
Description
FAQ
Comments (34)
what do the Kanji mean in the negative prompt?
I ran it through google translate and it's pretty standard negative prompt stuff, although I don't remember exactly what all of it was.
I left it in there as the default because that's what the default was from the developer's github.
it is all standard stuff, since the model is chinese and has chinese training, figure its better to use them than to switch to english
It is a Chinese trained model, and people have found it best to leave the negative in the native Chinese of the image become over exposed
, but they mean:
"bright colors, overexposed, static, blurred details, subtitles, style, artwork, painting, picture, still, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, malformed limbs, fused fingers, still picture, cluttered background, three legs, many people in the background, walking backwards"
@_Envy_ I'm not seeing the NF4 loader node in comfyui manager. Can you link to the git repo? Thanks.
https://github.com/silveroxides/ComfyUI_bnb_nf4_fp4_Loaders
Also added it to the description.
@_Envy_ Very cool, thanks.
I am missing the UNETLoaderNF4 node for the workflow. where can I find the node?
I get following error in ksampler:
All input tensors need to be on the same GPU, but found some tensors to not be on a GPU: [(torch.Size([655360, 1]), device(type='cpu')), (torch.Size([1, 256]), device(type='cuda', index=0)), (torch.Size([1, 5120]), device(type='cuda', index=0)), (torch.Size([20480]), device(type='cpu')), (torch.Size([16]), device(type='cpu'))]
Try using this version https://github.com/excosy/ComfyUI_bnb_nf4_fp4_Loaders
This will probably OOM on your first gen if under 12gb VRAM but try it again after and it should run
I tried to run ComfyUI with argument:
--gpu-only
but then I get another error in ksampler:
mat1 and mat2 shapes cannot be multiplied (154x768 and 4096x5120)
@MikuMasterRace I was reading around with the original nf4 implementation on comfy it seems just inherently broken from people talking about it in issues and most moved to GGUF after so they never fixed it. The link I posted before is a workaround and honestly not that good but if you have to use it that's probably the way to go.
Well I will try GGUF checkpoint then...
Sometimes when I get that error I just run it again and it works.
How much GPU ram do you have? What resolution and frame count?
@_Envy_ 8 GB VRAM, I tried to generate 17 frames at 512x512 resolution
https://github.com/excosy/ComfyUI_bnb_nf4_fp4_Loaders
Both Checkpoint and UNET loader are included.
Same as https://github.com/silveroxides/ComfyUI_bnb_nf4_fp4_Loaders, but fixed a RuntimeError.
RuntimeError: All input tensors need to be on the same GPU, but found some tensors to not be on a GPU.
@windblow1024770 Was able to get going with his but does it work with Lora’s?
I saw the same thing after the first time running it, had to reboot ComfyUI to get it to work again.
I know what's causing this. I'll push out a fix tomorrow.
@Griphen116 New error encountered after reboot: Expected a cuda device, but got: cpu
any idea on how to get those umt5.... clip models?
I think this is the one you need:https://huggingface.co/Kijai/WanVideo_comfy/blob/main/umt5-xxl-enc-fp8_e4m3fn.safetensors
But I am having trouble getting Wan 2.1 to run as well, so I am not 100% sure.
For some reason I have a conflict with Models/Clip_vision/Vae that work on other WAN2.1 variants.
If you could link to all the necessary files to run this workflow, that would be awesome.
where is the nf4 model...
@_Envy_ thx
I am missing the UNETLoaderNF4 node form the workflow. The manager has been updated with all custom nodes needed for the workflow. Where can this node be found or can the diffuser node be used in place of the UNET? TIA
KSampler
too many values to unpack (expected 4)
cant seem to get it to works