Yoshiaki Kawajiri was a producer, director and character designer for Madhouse Studio's. Back in the 80's and 90's. He is best known for being a director of Wicked City, Ninja Scroll, and Vampire Hunter D: Bloodlust. This style was based around his early works.
The videos here are made with both high and low noise with wan2.2. I also used lightx2v wan2.1 t2v lora to generate the samples for v2 and v1. v3 is using only the base model (I still recommend using the distill lora's for speed up) You should be able to set the low and high loras to strength 1 but can be increased.
Samples generated with
sampler: lcm
scheduler: beta
additional lora: wan2.1 t2v lightx2v
4 steps of high noise
4 steps of low noise
Low noise lora took about 36 hrs of training on a 6000 pro blackwell with lr 3e-5
High noise lora too about 8 hrs with lr 4e-5. I technically trained high noise up to epoch 32 but it was clearly overcooked and found epoch 16 to be the best results.
Version 3 updates: Trained for 32 more epochs and adjusted the dataset to include more short clips. I am rendering most of these examples without Lightning/FusionX/Lightx2v to show the base model capabilities. If you are unsure, I am putting the number of steps as 20 for the raw model rendering.
Description
This is the high noise lora.
FAQ
Comments (13)
Test footage looks great. Thanks for this, and I'll be keeping my eyes open for the high model.
Thanks, I just published it. The biggest difference I see is the lighting. The high noise model likes its dark, higher contrast scenes.
How do you feel about the High model results on top of the Low?
Using just the low model seems to give a lot more detail to the scene. Using both high and low gives a very clean look which is needed for the dramatic shots.
The high model obeys the camera positions a lot more than the low model does. So if you want a character to run towards the viewer or do a zoom in at a specific time, then high noise is useful.
tarnished3029 They look great. Did you train on only images? Or videos too?
Jellai 231 images 171 video clips.
tarnished3029 So much data! Way to go.
Is this image to video, text to video, or both?
Its trained as a t2v. If you want to use it as an I2v you will need to add extra noise to the image so it can do a style transfer when it processes.
That's awesome ! You do a great job.
I got a 4090 did you think it's possible to train a animé lora with it ?
It's possible but it will be slow. You would be better off cost wise to rent a 6000 rtx from runpod and train on that. Then use the 4090 to generate video using the lora.
Is it possible to train a lora from animated gif ? I suppose it's not possible to train wan style from only images or maybe ?