The purpose of this lora is to create a general anime style lora for anime from the 90's.
V2 info:
Trained on 12 different anime's from the 90s. A total of 766 clips. Each show has it's own trigger word and there is a general trigger word that's in all captions. I recommend using the ANIMSTY trigger word at minimum. If style doesn't trigger. As a last resort use "anime" at end of the prompt but I think it will make the output not as good.
General Trigger word: ANIMSTY
Source Trigger-Word
Akira: SHW_AKIRA
Cowboy bebop: SHW_CWBY
Evangelion: SHW_EVA
GitS: SHW_GITS
Goldenboy: SHW_GLBY
Gunbuster: SHW_GNBR
Kite: SHW_KTE
Outlawstar: SHW_OUT
Perfectblue: SHW_PFTBL
Slayers: SHW_SLYR
Tenchi Universe: SHW_TNCH
Irresponsible Captain Tylor: SHW_TYL
This lora should give not just the style of the atheistic of 90's anime, it will also bring the motion. (The base model of LTX basically gives you 3D style motion, this gives it a more 2D anime style). You may lose some things like camera control as the dataset has pretty static camera beyond camera pans.
Since this model is trained on a large variety of data (50-100 clips per show), you can use it as a base to train future anime loras. But maybe it might not work so well with modern anime. I might do a modern anime one next.
You can access the save state files on my hugging face here. Download the folder and then do this argument on musubi tuner fork: --resume "/PATH/anime90s/anime90s-step00040000-state". And use your own anime dataset from another show and train it to that show's style as a shortcut. This took around 40 hours of training to reach this state on a 5090 locally. I also will attach some sample data and my training configs. Feel free to ask if you need help.
https://huggingface.co/comfyuiman/anime90s_ltx/tree/main/anime90s-step00040000-state
https://huggingface.co/comfyuiman/anime90s_ltx/tree/main/anime90s-step00053000-state
I'd love feedback on this. I think I will do a modern anime style lora next. Then maybe try a fine tune of the actual model with more data. (long-term future goal).
I recommend using my example workflow, and its best on 50FPS, though 24FPS is ok too.
OLD:
The purpose of this lora is to create a general anime style lora for anime from the 90's. It's trained on 5 different anime from that era. It's goal is not just aesthetics but also to teach the motion of 2d anime from that era since its trained entirely on clips.
This lora was trained for the hackathon at ados paris (https://ados.events/). So the preparation was a bit messy due to the 8 or so hour timelimit. And a lot of issues which will be fixed in v2. But it was more of a proof of concept and its in good enough shape to release to public. All samples are t2v.
The dataset
Akira 75 clips
Cowboy Bebop 60 clips
Ghost in the shell (movie) 50 clips
Perfectblue 60 clips
Irresponsible Captain Tylor 60 clips
Total 305 clips
I used pyscene to split these into clips between 9-121 frames and converted them 25fps
I forgot to separate Cowbebop into its own group and as a result sometimes you get black bars on 16:9 generations (because my source for that was 4:3). This will be fixed in the next version. I wanted to do 10 shows and 700 clips but I ran out of time. So next version will have more variety of data source. And I might do 100 clips per show. As usual I captioned without mentioning the style to remove bias from the base model which doesn't understand so well.
Training
I was going to train on H100 but I couldn't get the instance working so I trained locally on my 5090. I will provide sample training data and config files later. The training resolution was 832x480 and max 121 frames. I used default LR and trained video only. I used rank 64. I trained to around 18k steps, but I like the result of 14k steps the most. But I didn't have much time to check earlier checkpoints between 8-13k steps.
Prompting
Use the style trigger word ANIMSTY to trigger the style.
I also have trigger words for each source material:
SHW_AKIRA, SHW_GITS, SHW_TYL, SHW_CWBY, SHW_PFTBL
The theory is that you combine the main styler trigger with a show trigger word and it will lean the model to generate in that shows style or you can remix by adding both. Or exclude show keyword for more general look. I need more time to test if that is the case, but I think maybe not. Give it a try.
For the most part it should trigger as anime style without any style keywords. But if that fails you can add "anime" to the end but I recommend only doing it as a last resort. Most of my prompts did not need it at 14K steps. And again sorry for black bars that may appear, it will be fixed in v2.
I recommend using my recommend workflow and 50fps.
Description
I trained the model up to 53,000 steps, I noticed model started to over train after this.
FAQ
Comments (9)
I have to say that LTX Video is trained on anime and is capable of displaying animation like traditional 2D anime on its own, instead the typical 3D celshading animation style, and you can achieve good results using only images in the training, as I did with my Mature Anime lora. The key is to start the prompt with the phrase 'An anime scene of...' or 'Anime scene of...'. It's also worth noting that there are noticeable differences depending on whether the videos are generated with ComfyUI or Wan2GP, with the latter sometimes producing a better 2D animation appearance. At least that's the experience I've had...
I've even had the specific case where LTX has generated some videos with subtitles, like in anime fansubs, even though there are no images with subtitles in my lora's training dataset.
Yes there is some training data but not enough in the base model. Prompting "anime" will just invoke the stiff ai like movement which doesn't look like how 2d style anime moves which is why I avoid it in my captions and Prompting. In terms of a still image ltx can produce something that looks like an a screen capture from 2d anime but the moment it moves there is a 3d cg style motion to it that is a dead give away. Actually my trigger word ANIMSTY is not good because it picks up anim as animated or anime. I wish I had chosen differently. I think the only way to truly fix will be a fine tune of the base model.
@tazmannner379 Well, take a look to the examples of my lora's gallery... No 3D CGI movement... And 'Anime' only is not enought, you have to write 'An anime scene of...' or 'Anime scene of...'
https://civitai.red/models/2516247/mature-anime-screencap-style-ltx-23-edition
@sirdrak your lora has the same issues I mentioned. It doesn't look like traditional 2d style animation in motion
@sirdrak Looks quite a bit like 3D CGI movement. I had to squint pretty hard to see anything that was even a little away from that. It looks great for what it is though. It just doesn't look anything like drawn cel animation.
This looks really good. Why 50fps as opposed to 24? Does 50 take twice as long to generate? if so, why target that when 2D animation is mostly based on 24fps? Wouldn't 24fps make for more than twice as fast generations, with zero drop in quality, or giving you twice as long clips for whatever your VRAM limitation is?
Its a problem with LTX base model. It introduces motion blur/distortion with quick movements. And with 2d animation it's even worse. The current best fix is to give it more fps. Ltx directly mentioned high resolution and 50 fps as ideal. Though it comes with its own set of issues. 50 fps videos in the model tend to be 3d animation so it also bias the model towards that. With this lora its trained so many steps that you get the 2d look with the high fps. Though motion still a bit 3d, I think this is as close as we can get for now. And yeah it takes twice as long to generate.
@tazmannner379 Ah, I wonder if that can be improved with later LTX versions. It's pretty unfortunate. At least they don't make you train at 50fps.
@Jellai Maybe, a bigger model trained on more anime would help. They said they're working on a potentially huge upgrade to parameters. They also said some IC lora may help (a huge one they're working on, dunno if they release it). Some other people have said training the lora on specific sigmas only (no idea how that works in relation to ltx though). And another idea I've heard is training on an anime dataset at 50 fps. Also another idea is an IC lora where you have smooth motion target video and blurred ltx distortion motion as the reference video.