Twi’lek party member from Star Wars: Knight of the Old Republic now in Anima flavor. Made to test 3D models as training data for Anima.
To see all my LoRAs, please (temporarily) disable X and XXX visibility, otherwise a lot will be hidden at the whims of credit card companies. You can help support making models like this by posting stuff you've made with it to the gallery using the "add post" button, that helps me earn back the training cost at no cost to you.
Tags: M_V_O_R, Twi’lek, blue skin, black headgear
Outfit: utility vest, open vest, sleeveless shirt, black armband, belt, black gloves, grey pants, thigh boots
V2: Added 2 fanart+Galaxy of Heroes 2d art (so dataset isn't monostyle) and reduced dim/alpha to 16.
Description
FAQ
Comments (16)
Edit: V2 and its lower dim seem to work better.
If anyone can get better results from this, please show me. Otherwise I'm going to conclude Anima, at least with Civitai's default training settings, can't eat 3D data like Illustrious could.
I made 2 anima loras with 3D data and I think they came out fine. There are way to many factors at play to know what's wrong here
@rotechacon667 Good to know. Wonder what's wrong with this then. My checkpoint?
I think I can't give you a good advice here, since I haven't really used 3d images in training. Looks a bit overbaked to me. Maybe dim 32 is too much for this kind of dataset.
Here's another great example of 3d anima lora: https://civitai.red/models/2210210/zootopia-1-and-2-style-krea-2-il-anima?modelVersionId=2867287
@RisingV A possibility, though that's worrying if I get overbaking with 29 different pics. I'll have to try one of my other 3D datasets and see how it goes since I didn't save any older epochs and (even though I have the blue buzz) don't really want to blow more on training with no changes on a "maybe".
@NanashiAnon doesnt look like you actually put "3D" in any of your prompts
also you are using a 2d stylized checkpoint that isnt anima, just anima based
@nogo 3D data based Illustrious (and Pony) LoRAs become style agnostic if the data is tagged "3D" and will readily adopt any style given in prompts since the 3dness is isolated to the 3D tag. I was hoping it would function the same in Anima and make a LoRA suitable for 2D generations. The Illustrious version of this in recommended resources and used the exact same dataset+tags. I did try tagging 3D in generation and using BlenderMix but that came out even weirder so I didn't post it.
So everything I am saying here is guessing (with a bit of experience): Anima's text encoder works just very different than CLIP (illustrious te). Even when not training the text encoder it is used to process the image captions of course. So not everything that works with illustrious works with anima. The CLIP text encoder of illustrious basically encodes every syllable of the prompt/caption as separate token. The anima text encoder on the other hand encodes more like full sentences/longer tag sequences at once. So while CLIP probably encodes "3d" as a single token, it will be encoded with anima te in a longer sequence with other tags weakening the binding of style to it. So what can you do about it?
Text encoder training.... no, I'm joking, I know it is not in the civit trainer right now. But yeah, you used text encoder training with illustrious version, so it is also naturally working better bc of that.
What you can do without text encoder training: I am not 100% sure about how the style was trained in the anima base model, but looking at the training data of the greg rutkowski demo style lora trained by cirlcestonelabs, it used "@greg rutkowski." as prefix of the (natural language) caption. So using "@3d" in the caption will probably bind that tag to the style (maybe you should also use the recommended tag order like M_V_O_R, @3d, ... ). Though I think that demo lora was trained with text encoder so no idea if it works as good without it (I used "@hibikileon" in the caption of this lora since all the images were by that artist, but I only trained one version and can't compare it).
Another thing you can try is being more descriptive about the style. I haven't tried this, but since the anima te is an llm and was trained partially with natural language captions it should be able to understand something like "a screenshot of a 3d video game with polygonic optics" or something like that (I am bad at describing styles...). This will work better with te training of course.
I can totally understand, if you don't want to use 3d data with anima as long as the training options are not expanded in the civitai trainer, but these are the ideas I have about this.
@RisingV I'll probably train another one (and some of the other weird styles I've got data for) just to see if the issue is consistent, though if that comes out poor too I won't throw more 3D datasets until/unless there's improvements to the trainer.
I'd try to train a 3d style (or character based on 3d) myself with anima, but I don't have a dataset for that...
Maybe I'll try booting up Pokemon Colosseum and make some screencaps :D
@RisingV Could work, though the auto-changing camera in battle would make trainers annoying to capture in anything but intro and defeat pose. I suppose Shadow Lugia (only unique unique non-trainer between the two) is possible for XD (GameFAQs has multiple saves that claim to have it captured and unpurified if needed), though that has the added wrinkle of being a not at all humanoid (though I suppose "works on Illustrious" is really the only test needed).
Provided you have a google account you can take your data set and run it for free on Google colab, just have to have a zip of your files and tags together. It is limited to no more than 1,000 steps so you will get like 5 epochs out of a 30-40 picture set. This might help in determining whether this is a Civitai problem or more a dataset/ tagging or just a Anima related problem.
@carneil1000 I guess, but I really don't want to touch anything Alphabet.
Anima v2.0 works much better than v1.0. For some reason it still produces 3d style when using hybrid (tag+nl) prompting, but putting "3d" in negative prompt fixes it for me.
Good. I set a retrain of one of the underwear models with dim+alpha set to 8. Epochs 8+9 seem to have the overly real appearance but epoch 10 doesn't. We'll see how that works tomorrow.