A character LoRA for Pavi from Avatar: Seven Havens. Pavi is the protagonist of the third coming of Avatar Legends series and the successor of Aang and Korra as the Avatar. Trained with screencaps of the first official trailer of the series and a few artworks I found. Since neither anima nor illustrious seems to know what a peg leg is, I had to do a retrain using "prosthetic leg" instead, but it seems like this is a concept that is generally hard to get. Works now ok, but the peg leg often shows up on the wrong side. I also managed to train in her cat monkey Geet more or less and two other characters were in the dataset, Jae and Nisha, but I haven't really tested if they can be generated.
trigger words: 1girl, pavi, purple hair, low ponytail, multi-tied hair, green eyes, cape, tunic, pants, prosthetic leg, single boot, leg bow
Feel free to comment, if you have something to say about the lora and share your images on the model page. Sharing your images on the model page gives the creator a small blue buzz bonus. But more important I like to see, what people do with my LoRas.
If you like the LoRa please have a look at the showcase pics and leave a reaction.
Description
Comments (9)
Can you make illustrious model plss
Yeah, I actually trained an illustrious version when I was doing the anima version, but I was facing the same problem as in the anima version that her peg leg did not train in and it's harder for illustrious to get such a thing. So illustrious version needs some more training.
@RisingV damn, but its still possible right?
@Otherthings I hope so. I noticed a flaw in my dataset settings that was training the original and a vertical flipped image and probably contributed to the difficulties. I am doing a full retrain of illustrious version without that setting and some additional images, but it's not finished yet.
@RisingV Alright, thanks for you effort keep trying! Tho since I'm kinda a newbie to ai stuff, whats really the difference between anima and illustrious models? For me from what I've seen so far when generating, anima takes forever to generate
@Otherthings When I generate locally anima is a bit slower for me, but not that large a margin. I haven't really measured how illustrious and anima compare in onsite generation though. So if you are refering to onsite generation it's probably lower server capacity or something for anima on civitais end.
What I have experienced in LoRA training so far is that anima gets things easier than illustrious. E.g. this doesn't work perfect with anima, but I doubt it would work with illustrious at all.
Using that LoRA as example, another thing anima is in general better than illustrious is prompt adherence. You can see that I did showcase images prompting 5 different characters and anima manages to display them more or less correctly, e.g. this image. Illustrious sometimes already struggles with two characters.
Then the ability to use natural language. If there is no danbooru tag for describing a concept/scene/pose you can try to describe it with natural language, e.g. this did not work out with just tags. Illustrious does not understand natural language at all. That's because it's using CLIP as text encoder (the part of the model translating your prompt to computer language) and not a Large Language Model (LLM) as most modern text-to-image models.
There are other things I could mention, but one big reason to use anima is that illustriousXL is based on the SDXL (Stable Diffusion XL) model, which is already three years old. This is like stone age in terms of AI. I am not saying there are no reasons to use Illustrious, it's still a capable model if you want to make 1girl images. However in the last years models have gotten better every month, so the only real reason why such an old model is still in use is probably the generation cost is lower than anima (and people hold on to what they know).
@RisingV I see, thanks for the explanation... I guess it depends on hardware specs then too? I'm generating locally, using invoke AI, RTX 2080... Last question if you don't mind does danbooru style prompt work on Anima for you? When I tried, the generations look worse somehow compared to natural language prompting. I kinda find natural language a lot more of a hassle compared to danbooru style
@Otherthings RTX 2080 should be good enough to run anima at decent speed. I don't know about invoke AI, I am using Forge Neo. With a RTX 3080 I have ~20s per image generation time with anima at 1k and 11-12s per image with illustrious. Forge Neo also allows to use int8convrot versions of models. It has nearly a 100% accuracy compared to the normal model, but runs faster with less VRAM usage. For me anima with int8convrot is ~14s per image at 1k. And you could also try to use the anima-turbo version that needs only 8 steps instead of 30, so it is significantly faster.
I use booru tags only most of the time with anima, just look at the prompts of my showcase images. Sometimes I am using hybrid prompts, e.g. describing a character and it's traits with tags and then a natural language sentence to describe what they are doing like (quality tags omitted) 1girl, black hair, long hair, twintails, black eyes, oversized shirt, she is sitting on a landing stage looking at the ocean. I think it's a strength of anima you can use both. If you are using the correct tag order and quality tags there should not be a problem with image quality (though you may need to use a style tag or a style lora with anima base, since it does not have a strong default style like many illustrious models).
@RisingV Okay. Seems like I learned some new things, thank you for answering my questions! Hopefully that Pavi illustious version works^^









