Update - 28/02/24 - There is a new mix model, instead of the Pony model. I have updated links.
Going to keep this brief because I'm too lazy to make a full writeup. This model and training method far surpasses anything we've seen with NovelAI from 2023. It trains at 1024x1024 based on SDXL.
You can refer to https://civarchive.com/models/62125/lora-lazy-training-guide for tagging etc.
Refer to https://github.com/bmaltais/kohya_ss for the kohya link.
Download https://civarchive.com/models/288584/autismmix-sdxl
Download https://files.catbox.moe/dvx8me.json or the .json from the attachments - When opening Kohya, use this as your configuration (click on config at the top on the LoRA tab, open, select this json).
Point the training model to the Pony model.
Change your save location to wherever you have your LoRAs.
Tag all of your images with score_9, source_anime. Set your folder repeats to 1.
Train.
When generating on the above pony model, put scores 6 through 4 into your negative prompts, and add 'score_9, score_8, score_7, source_anime' to your prompt.
ISTRONGLYrecommend using an artist LoRA. Tryhttps://rentry.org/ponyxl_loras_n_stufffor LoRAs made specifically for the pony model.
NOTE - You no longer NEED to use an artist LORa with the fintune of the 'autismmix' model for Pony. It generates anime out the gate.
Generate. It should look good. The results speak for themselves.
The pony model is the best we have for local anime at the moment. It blows base NovelAI out of the water.
Description
FAQ
Comments (7)
Hi, first of all, thank you for this.
I'm training a style which is less flat than anime, more like semi-realistic so... I'm a bit worried about those "score" that comes with PonyXL models, isn't that going to affect too much the style I'm training ? I tend to not add any score in the captions for training and then use them when I generate images, along with the finished LoRA. I want to try another training using scores as you suggested but... I kinda don't like doing things that I don't understand. HOW did you figure out that score6-4 should be in the negative, are those forcing the realistic styles in Pony ? Is that documented somewhere or there's no way way other than "figure it out yourself" lol
If you have any info/links about those mysterious tags please let me know !
Also when you say to use a LoRA, does that mean during the training or after ? Because now I'm wondering how I'm supposed to add a LoRA in training if Kohya_ss doesn't know how to handle them and where they are located.. you know ?
ok that's is with the questions lol
In your config "caption extension" is set to NOTHING, should be ".txt" you really need to fix that I lost 2 days because I overseen it x) Thanks
I got 13s/it and 4hrs cooking time on my rtx 3080, it seems to me that this should not be the case on Lora with 1200 steps. What could be the problem?
In the json you supply for training pony/autism , you have a clip skip of 2. SDXL is meant to be trained with no clip skip, yet the links in the Pony information guide you also link has clip skip 1 listed. Which is it? For SD1.5 it was established that 1 was for realistic and 2 is for NAI based. Is this the case for SDXL vs Pony?
When you say "Tag all of your images with score_9, source_anime" do you mean naming all the images?
Do you have an idea how you would try to minimize style learning if you were trying to train a concept (for example a non-specific subject interacting with a specific object) but had to work with a limited dataset, in which one style is predominantly present?
Hi, the JSON link is broken. Can you share again :)