Wan 2.2!
Re-trained the dataset on Wan 2.2, make sure to grab BOTH Loras (High and Low)!
Getting great results with K3NK's workflow - https://civarchive.com/models/1824027/wan-22-t2v-i2v4-stepskijais-wrapper-workflowk3nk
Update 6.17
I realized while trying to improve this LoRA that I never uploaded a T2V version, so... here it is. Enjoy!
-- -- --
My second LoRA... I've made some drastic changes to my captioning approach, and the results show some nice improvement.
Please share anything you make with this, I'd love to see some better prompts!
This was trained on WAN 14B I2V, on 45 videos normalized to 480P / 24FPS and trimmed to 3 seconds using diffusion-pipe. However, it works okay with the T2V model, and I've included some examples from that.
My captioning approach:
45 videos, resized to 480P, 3 seconds, 24FPS.
Ran each one through ComfyUI_Qwen2-VL-Instruct to generate a base video description, but this unfortunately doesn't pick up on any NSFW bits. This usually took a few tries on the same image, since the LLM almost seemed "disgusted" by the suggestion. :D
Grabbed my "favorite" frame and ran that through Joy Caption 2, then I manually combined the Qwen description and the Joy Caption Two caption to make the final .txt file.
Description
Behold, the T2V version that I never quite got around to baking all the way! Surprised how great it looks at 40 epochs, so I'm uploading. Enjoy!
FAQ
Comments (8)
GYAT DAMN! Been waiting for this type of lora MONTHS!
Thanks for releasing this - this is now my number one lora!
Nice... thanks, I try. It's comments like this that make all those hours captioning and trimming down porn clips alllll worth it. :-D
Amazing lora, works great for i2v. Seems to change character likeness in t2v though - will play around with strength and see if I can sort that
There's two versions, I2V and T2V. In my experience the T2V likes to give everyone long hair at full strength, but likeness is still pretty good.
I'll give it another try, might be my character lora. Was definitely using the new T2V. Hair was ok with prompting though. Thanks anyway, the i2v at least for me is a gem.
There is always a hand appearing out of nowhere... I even tried with "(hands:1.4), (hands appearing from offscreen:1.4)" in neg prompt and a lower weight of the LoRA (down to 0.55)...
Can someone help out an absolute noob here?
From what I understand, I need to use a checkpoint for this to work, but I’ve never worked with one before. When I load a checkpoint, it seems like my existing diffusion model, CLIP, and VAE get excluded.
Is that what’s supposed to happen?
Just noticed this was from 2 months ago lol.
Here's a link to the wiki for anyone that may still have this question. Also plenty of good stuff on YT, aitrepreneur is a good one. And start with WAN2.1 if you're just beginning, don't worry about LoRas yet. https://comfyui-wiki.com/en/tutorial/advanced/video/wan2.1/wan2-1-video-model#example-of-wan21-comfyui-native-workflow