Pokemon LoRA (Ken Sugimori style for Pokemon and trainers)
Making models can be expensive. Do you like what I do? Consider supporting me on Patreon 🅿️ or feel free to buy me a coffee ☕
V3 UPDATE: trained on all Pokemon currently released or announced as of today (all the ones until S/V and some of the DLC ones that have been announced).
V3 CAPTION STYLE UPDATE: trained on all Pokemon currently released or announced and their Bulbapedia descriptions (shortened using ChatGPT). Read "About this version" on the right for more.
This started out as a fun experiment. I know there is already a Pokémon style model which is decently good, but I wanted to see how things could work with a LoRA instead. For v1 I didn't pay much attention to the dataset, but the result was still better than I expected, so I made a v2 which is much more consistent and can also do Pokémon characters and style in general.
V3 is offset and should be used at weight 1.
V1 and V2 trigger words are sugimori ken \(style\), pokemon \(creature\) (optional if you don't want to make humans) and all Pokémon names until XY (plus some more).
I've used it with AnyLoRA at 0.5 or 0.6 weight. Also use CLIP skip 2 and remember to generate relatively small images (max 768 w/h) and then upscale.
V2 was trained on like 800+ characters inside (up to XY + some random recent ones and many human characters), it cannot replicate characters perfectly, but it's still god enough to make Pokémon fusions (like Blastoise + Venusaur in the examples), especially with inpainting as a guide (which I didn't use for the examples). It's also good at creating completely new Pokémon and trainers. Use no humans, pokemon \(creature\) if you want to create Pokémon only.
How to use LoRA's in auto1111:
Update webui (use
git pulllike here or redownload it)Copy the file to
stable-diffusion-webui/models/loraSelect your LoRA like in this video
Make sure to change the weight (by default it's
:1which is usually too high)
Description
This version is harder to use but offers MUCH more control. It has been made using Bulbapedia descriptions and ChatGPT to generate captions for the dataset. Every Pokémon in the training set has a short visual description. As such, the model is much more able to follow prompt directives. The counter effect is that if you don't prompt any description it will not make a decent Pokémon.
In the examples my file is called v4_mix. Ignore it.