Note that the first model is trained with clip skip 1. If you prefer to use clip skip 2, please use the second model.
The two models should perform quite similarly in most situations (as long as you use the clip skip each model is trained on). Some comparisons are provided in the last two example images.
--
The updated models are trained with the entire anime series + fan arts that I collect up to this date 2023.04.30
I think it should improve in particular for Miyo and Asahi
There are few images of FujimiNemu and TenkawaNayuta but I don't think they are properly learned
Checkpoints of other steps can be found in the associated hugging face repository https://huggingface.co/alea31415/onimai-characters and in particular https://huggingface.co/alea31415/onimai-characters/tree/main/loha_all_0430
As usual the networks are trained on top of ACertainty and should work on most popular anime models
For LoHa I have both clip skip 1 and clip skip 2 versions while the old LoRa was trained on clip skip 1 (as far as I remember)
You can try to switch between several styles as well: aniscreen, fanart, edstyle, manga cover. Note that however
aniscreen may conflict with style model
edstyle unfortunately comes with text most of the time and it is hard to get rid of if
manga cover may also produce some text occasionally but this can be mitigated with proper prompts and sampler
most of the time putting nothing may be just the best thing to do
More:
The clothes seem to be properly learned. I am not sure what is the minimum you need to trigger each of them. Just use common sense. For example for uniform you have either "school uniform, jacket" or "school uniform, suspenders" and for Mihari you can put "bolo tie, labcoat". Play with negative prompt to avoid blending.
I only remove tags about eyes during training. Therefore you may need more description about hair style in your prompt. I made this decision because each character appears with several different hair styles.
While the choice of sampler mainly depends on your flavor. It can sometimes results some weird effect. For example a lot of strange things appear with manga cover style when I use DDIM. On the other hand Euler a and DPM++ 2S Karras give better results in general. I do not have any text in the training images of this style, so it is quite mysterious that these stuffs appear.
--
I don't need support or credit, but I would be glad to know that you are using the models I trained and find it useful.
Moreover, I would like to advocate for more franchise models.
You can take a look at my workflow https://github.com/cyber-meow/anime_screenshot_pipeline if you are interested.
I just want to spread the fact that there is no reason to encode a single concept in each lora.
Description
For some reason people seem to prefer loras over full checkpoint, so as an experiment I decide to make a lora based on the same dataset that I used for the model https://civitai.com/models/7848/onimai-characters
Civitai does not allow me to mark the repository as both full checkpoint and lora so I can only create a new one for this.
Naturally I am able to get the characters trained but not the styles. Unlike the full checkpoint, the outfits are not learned either.
FAQ
Comments (16)
with proper tagging and dataset, you can train both style + characters (and their outfits) in a LORA.
check my HxH Lora for example
Thanks. I know that outfits can be trained if tagged properly, but they are just not automatically learned as in the full checkpoint. As for the style I guess I would need to play around with the parameters.
The "some reason" why people prefer LoRAs over full checkpoints is that LoRAs are portable between checkpoints so they can use their preferred mix of base and LoRA styles. It's also a significantly smaller download compared to an entire checkpoint.
The fact that they are portable between checkpoints is just partially true. They work for common models NAI/Anything/Orange as they are close enough but for the ones that are trained further the portability gets decreased. Moreover, lora basically stores the weight difference of the spatial transformer blocks, so computing the difference of an entirely trained model and its base model would give you a little bit more (the convolution blocks are trained as well). You can also convert this difference into lora in scarifying the weight difference of the convolution blocks.
The size is a valid point and of course the big reason to prefer lora, but I would say training it for a single concept is a bad practice and goes against this advantage (especially with dimension 128).
I'm having problems with extracted LORAs from old dreambooth models. Its no surprise to me you can't get the style from your model but training a LORA on that style should be quite easy since you already have the necessary database.
That being said, the anime is still airing and some of the characters only made their debut quite recently - please don't lose your motivation to make a decent model+LORAs before it finishes.
Also, thank you for working on this. This anime is highly underrated outside of Japan and deserves more love.
The anime style is apparently trained into the loras, but the other styles are not. Maybe I just need to find the right parameters.
I may try to make an update once the anime finishes (though my priority will be tensei oujo).
If you don't mind me asking, do you have a link for the "nep" model?
https://mega.nz/file/WgEzxZYA#tsUqBeE9ChdTfPCqIkoTDuZyrqOtuaX8hQMO7bIirqk
@alea31415 Thank you!
@alea31415 reup possible?
@alea31415 I myself would request the merge_dwd_kohaku-v2-fp16 model, it looks amazing!
@alea31415 could we also get a link for the anyloli v8 model if possible?
I knew this was coming...
Unfortunately I am the authors of none of these models
They are shared privately and the authors prefer to keep them private after asking for their opinion
@alea31415 understandable. appreciate the response
@alea31415 fair enough. is there a place that is better to look? do you have a telegram group or anything? I have things to share as well.
@HereForTheAI These three specific models are from a Taiwanese discord server. Ping me on discord (cybermeow#1385) if you want to know more.
Otherwise in the future I may consider using multi-style models such as ALunarDream https://huggingface.co/lunachan/ALunarDream or VBP https://mega.nz/folder/M0twDRRR#yLh3JctQrP8S6J5dnUStbA
The latter comes from a chinese telegram channel https://t.me/StableDiffusion_CN
There is probably no need to mention all the major English discord servers such as touhouai?
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.