V1.1 - Sorry for the quick release wanted to get this out before i was done for the day.
added batching (works but its not perfect)
added hand masks (sam 1 mask) Option to add lineart hands to control
added model upscale and frame interpolation.
added a few more notes.
added many missing switches
V1
Removed pony.
Start frame and control video. If you make a start frame it works well, if you dont you can still get good results
All examples dont use a custom start frame and just a cropped image ref.
Beta
- warning this works on 12gb. i dont know if it can do less. Pony takes A lot in itself. This has very few notes, and while its all grouped properly its a complex flow that has many moving parts. I am working on finishing this but wanted to see if there was any real interest before i did much more.
I used the 1.3b control model.
- multi step wan video to video with image ref.
-Depth and openpose controls
-full masking support with florence and sam 2
-full pony/xl groups for first frame image generations
-Teacache
-as low as 8 steps video renders
-2 upscale options
-bg replace
-lots of other things
https://huggingface.co/alibaba-pai/Wan2.1-Fun-1.3B-Control/tree/main
https://huggingface.co/alibaba-pai/Wan2.1-Fun-14B-Control/tree/main
Description
FAQ
Comments (1)
how i can use loras? or i can't? all time get error
ERROR lora diffusion_model.blocks.0.cross_attn.k.weight shape '[1536, 1536]' is invalid for input of size 26214400
Looks like we don't have an active mirror for this file right now.
CivArchive is a community-maintained index — we catalog mirrors that volunteers upload to HuggingFace, torrents, and other public hosts. Looks like no one has uploaded a copy of this file yet.
Some files do get recovered over time through contributions. If you're looking for this one, feel free to ask in Discord, or help preserve it if you have a copy.