AnyLoRA
Add a ❤️ to receive future updates.
Do you like what I do? Consider supporting me on Patreon 🅿️ or feel free to buy me a coffee ☕
For LCM read the version description.
Available on the following websites with GPU acceleration:
Remember to use the pruned version when training (less vram needed).
Also this is mostly for training on anime, drawings and cartoon.
I made this model to ensure my future LoRA training is compatible with newer models, plus to get a model with a style neutral enough to get accurate styles with any style LoRA. Training on this model is much more effective compared to NAI, so at the end you might want to adjust the weight or offset (I suspect that's because NAI is now much diluted in newer models). I usually find good results at 0.65 weigth that I later offset to 1 (very easy to do with ComfyUI).
This is good for inference (again, especially with styles) even if I made it mainly for training. It ended up being super good for generating pics and it's now my go-to anime model. It also eats very little vram.
Get the pruned versions for training, as they consume less VRAM.
Make sure you use CLIP skip 2 and booru style tags when training.
Remember to use a good vae when generating, or images wil look desaturated. Or just use the baked vae versions.
Description
Do NOT use this for LoRa training.
LCM version. Use it with 5-15 steps, ~2 cfg. IT WORKS ONLY WITH LCM SAMPLER (to get it on Auto1111 you currently need a plugin).
Being a distilled model it has lower quality compared to the base one. However it's MUCH faster and perfect for video and real time applications.
FAQ
Comments (10)
I guess this is not a story to be told individually, but when I adopted and changed any LORA from anythingv5 to any LORA as a training model (Pre-trained model) for LORA, it generated LORA that was not trained at all (almost the same image as the one with no LORA applied). Both types.
I don't know why at all because I didn't change the settings at all, but I guess it's not for me because many results have been reported.
In general, you don't train anime loras on anything else than NAI model (anime full pruned). Otherwise you end up with poop generations if you change between models.
oh no, the NAI dogma still lives in 2024
@Lykon ofc, if there's no better alternative what do you expect
all the top anime lora creators still work with nai model
@LittleJelly I used to be the top anime lora creator and I've switched away from NAI very early. NAI is just a SD finetune like all the others, nothing special about it. It's super old and outdated at this point that most style loras trained on it won't work accurately on most models. Not to mention it still uses the SD1.4 text encoder so it's not on a fine-tuned CLIP and not even the final version used by stability.
People often think that NAI is better because most settings end up making loras that are good at weight 1, while training on AnyLoRA is much faster and you end up with a lora that should be used at around 0.65 weight and you have to offset.
There is also a lot of misinformation around, like that NAI is better because it has a text encoder finetuned on tags, but that's not true. It has objectively a worse text encoder compared to most well made models, and definitely worse than SD1.5.
Once people learn how to train on anything that's not NAI, they never go back. This is anecdotal, but I had countless people telling me that on Discord.
Plus there are some concept you can't train on NAI (or AnyLoRA) at all, you should adapt the base model for different use cases.
@Lykon I've been training on NAI since I came to SD, and never had an issue with it on most of the models
Also never had an issue that i had to use 0.65 on lora, always around 0.8-1.
It's definitely outdated, but it still works the best.
On the other hand, I've tried training with other models and the results never looked good on any model.
@LittleJelly Wrong! Sometimes a 3d esque style will bake in better using AnyLora. Lora training is an entire preference. EVEN THE TOP ANIME CREATORS WILL LET YOU KNOW THIS. (I know some of them so stop harping like you know everything) -- This is why AnyLora and NAI are in many of the colab notebooks for training.
@duskfallcrew for 2.5D style sure why not, but focusing solely on anime style NAI is better
FallenIncursio, ChameleonAI, GoofyAi, all of them still use NAI (and they still make nice 2.5Ds which work around 0.8-1 weight).
I'm not saying i know everything. Just saying my opinion, after a few tests I've done.
@LittleJelly anime styles trained on NAI will work on NAI and some anime models. Anime styles trained on AnyLoRA work on DreamShaper and all derived models too. Anime characters trained on AAM from screencaps will work even on photorealistic models.
Maybe you can achieve the same on NAI too, it's just much easier for me to swap model according to the use case and how much compatibility I need.
@Lykon I have no idea about realistic results on NAI. Not a fan of realistic AI, looks too plastic.
I'll give one more try training on AnyL if it gives better results than NAI trained model.