V2:
Trigger:
tomie, mole under eye
extra face
A cleaner version that more consistently puts a single mole on the right side of her face.
V1:
Trigger:
tomi3
split face, mutated face
'tomi3' for normal Tomie
For the split face, add 'split face, mutated face', but be careful about tags like '1girl' or 'solo' especially with embeddings because it will cause her face not to mutate as the 1girl tag gets confusing with 2 faces and embeddings usually try to stop a mutated face from happening.
If you're having trouble, it may be your default negative prompts to stop bad anatomy as well.
Description
FAQ
Comments (5)
Thanks for the second training ;)
Given the training dataset tags, you captioned the mole under eye. Did you try not captioning such details inherent to the character?
Training on Pony, I ended up with slightly better results training without captioning things like mole. At they end they become part of the concept trained rather than a variable such as hair color, etc. People can always get rid of a mole inpainting. Or eventually negative prompting it.
Could you please share what worked best in your IL training tests please: captioning the mole or not at all? If you tried ;)
Cheers.
sdxl based models are definitely not my forte, If I'm strong at any loras, it would be for flux. The big thing for redoing this one is that the initial advice I got lead to me severely overtraining, I was only testing the latter epochs, and then i realized how overtrained everything was because I forget things should still work at CFG 7 on SDXL based models. I got too used to basically only using flux guidance of 3.5-4 on flux. The other big mistake I made was flip augmentation when the character has a distinctive feature on a specific side of her face. I normally use flip augmentation when my dataset is small. I also did not specifically caption the mole on the first version. I did with with this one, and I also looked up the danbooru tag used for images of tomie with a split face. I then enabled shuffle tags while keeping top 3 tokens, which would be the keyword, mole (if visible) and 'extra face', where applicable. If I do more experimentation, I'll let you know, but i don't think i have the information to answer your question because the reason I retrained it was separate from the things you were talking about.
Is there a way to isolate the split face concept? I want to reuse it with different characters, and perhaps from behind too.
I think it would probably have to be its own lora for it to transfer well to other characters, I know that the effect is stronger in the first version, maybe playing with the weights using that one will allow you to accomplish that. I don't know if there is enough training data out there but it sounds like an interesting concept lora on its own. If I find enough with other characters, I may make another lora
I am able to create the effect on different characters by using a lower weight and a controlnet, so I am able to create my own training data, it will take some time but I will make a model that pertains specifically to the split face concept