For those don't want to read: This is a distilled model, you need to lower CFG scale to 1~1.5.
RDBT [Anima]
See the main page for more info. (usage, faq, int8 ckpt...)
See this page for update log and version info.
Sharing merges using this model is not allowed. This "restriction" won't affect anyone. It's only aimed at those who steal others' models to sell.
This is a fully finetuned model, but in LoRA. It needs the correct base model to function properly. Don't mismatch base models. If you don't know how LoRA works or what this means, use the ckpt version.
Links to the LoRA base models:
AnimaYume (ym) (hf link) (civitai link) (int8 version).
Anima pretrained (b1,p3,p2,p1) (hf link)
FYI: "Merged" base model usually is NOT recommended. It's weights might be very messy. Probably merged different pretrained versions. A nightmare for all LoRAs, not just this LoRA.
Description
FAQ
Comments (6)
I think you did it again, boss. Amazing stuff.
What's the relationship between this and the anima turbo lora when it comes to stabilizing and style adherence? if my intention is to use both and preserve the base model's style knowledge as much as possible, should I lower turbo?
_ relationship between this and the anima turbo lora
I don't know. They didn't mention which method they used.
_ should I lower turbo
A simple guide would be making sure the sum of those strengths <= 1.
A more practical guide is to check if the model starts to show repeating patterns (more common in background like trees, cloud, buildings), as the distribution/diversity collapsed too much, and that's why people feel the "sloppiness".
Hey there, could you teach me how I can add realistic data to my lora datasets without them dominating the style aftewards? For me its a huge issue. I have like 1 photo among 20-30 normal drawn images and it will just dominate the overall default style of the lora. Afaik you use lots of photo data and add it to 2d data. How do you do it without causing issues? Thanks for any advice
My advice would be large dataset and don't overtrain. If you follow my rdbt model you might notice v0.37~0.24 are overfitted and has similar issues. v0.39 was trained with -60% steps and -50% lr, and no stage 2 finetuning, and is considered the best version so far...
Although I'm not sure why "1 photo can dominate other 20~30 drawings". Sounds impossible to me. Anyways, when I training a small dataset, I add trigger words, e.g. "@main, @realistic_photo...", "@main, @anime drawing", and apply very aggressive (>50%) tag/caption dropouts, just keep trigger words. Ensure the captions are not ambiguous, so the model won't mix styles. I use update batch: 16, update steps: 200, adawm and lr 0.00005 for my small dataset (~50 imgs) lora
@reakaakasky Thanks, I will look into that. It may be that my rank is just too high (64). Maybe it just taps into cosmos backbone knowledge too much when photoreal is part of the dataset, with a high rank, dunno.
I forgot to mention: uniform timesteps with shift 3.
Paired with aggressive tag dropouts = let the model just memorize the data with minimum conditions. Sounds bad. But afaict, this is the best settings I can get when training on small dataset. Converge fast and clean.