Pokemon LoRA (Ken Sugimori style for Pokemon and trainers)
Making models can be expensive. Do you like what I do? Consider supporting me on Patreon 🅿️ or feel free to buy me a coffee ☕
V3 UPDATE: trained on all Pokemon currently released or announced as of today (all the ones until S/V and some of the DLC ones that have been announced).
V3 CAPTION STYLE UPDATE: trained on all Pokemon currently released or announced and their Bulbapedia descriptions (shortened using ChatGPT). Read "About this version" on the right for more.
This started out as a fun experiment. I know there is already a Pokémon style model which is decently good, but I wanted to see how things could work with a LoRA instead. For v1 I didn't pay much attention to the dataset, but the result was still better than I expected, so I made a v2 which is much more consistent and can also do Pokémon characters and style in general.
V3 is offset and should be used at weight 1.
V1 and V2 trigger words are sugimori ken \(style\), pokemon \(creature\) (optional if you don't want to make humans) and all Pokémon names until XY (plus some more).
I've used it with AnyLoRA at 0.5 or 0.6 weight. Also use CLIP skip 2 and remember to generate relatively small images (max 768 w/h) and then upscale.
V2 was trained on like 800+ characters inside (up to XY + some random recent ones and many human characters), it cannot replicate characters perfectly, but it's still god enough to make Pokémon fusions (like Blastoise + Venusaur in the examples), especially with inpainting as a guide (which I didn't use for the examples). It's also good at creating completely new Pokémon and trainers. Use no humans, pokemon \(creature\) if you want to create Pokémon only.
How to use LoRA's in auto1111:
Update webui (use
git pulllike here or redownload it)Copy the file to
stable-diffusion-webui/models/loraSelect your LoRA like in this video
Make sure to change the weight (by default it's
:1which is usually too high)
Description
FAQ
Comments (18)
very interesting
This LORA is hands down one of the best!! Thank you so much for that!
Few notes -
I tested this using XYZ plotting and basic S/R prompts to get different styles. It is hard to get few pokemons correctly.
I tested on first 151 these are few things I noticed -
Kakuna is ver very hard to create, almost impossible.
Clefable, Electrode, ghastly are same like that, but sometimes u get close resemblance outputs.
For Vileplume I keep getting results of trees and shrubs.
I would love to help in any way possible to improve this lora. Do let me know and cheers!
I'm amazed you managed to reproduce some of them. Consider that there is basically 1 example per pokemon, so this is not aiming at recreating specific pokemon exactly, but should get the general "concept" from the name and allow to mix them. It's a style lora and a fakemon lora. Wow, good job. Please post some of them.
Also I have already decided I'll do a 4th version trained on captions. But it will need a lot of coding to scrape good pokemon descriptions.
@Lykon I am willing to contribute to the dataset in any way possible. I can tag, caption, and create batches, maybe we can start with each generation to test it out?
@Psiclone99 how could you possibly caption 1000+ pokemon by hand? :D
@Lykon I can definitely do Generation by Generation, like can start off with the first 151 originals, I can get the data from Bulbapedia, so it might hardly take a day to create that set.
@Psiclone99 I wanted to make a script to scrape data directly from bulbapedia, then instruct chatgpt to shorten the descriptions to 75 tokens. No need to do it manually
@Lykon Awesome, Awaiting an update and looking forward!
@Lykon i am amazed by the professionality, will be awesome to see what you come up with!
Been trying to get some pokemon-themed trainers with no pokemon/creatures in the image.
Anyone have any success with this? EX: trying to get a pikachu-themed trainer without any pikachu-likes running around
this is not a pikachu lora :\
works good for fakemon, would be nice to also have an actual real pokedex but i understand that gotta be long work
that would probably require like 1000 loras to be accurate :D
I'm working to make one that's accurate with descriptions in English instead of tags.
@Lykon ahah like one lora for every pokemon? damn
@Yoshi_ yeah, new version has all descriptions for every pokemon and got trained for 10k steps. While it understands pokemon names a bit, it doesn't do them exactly.
@Lykon i see, you are kinda trying to convert a style to characters not easy
@Yoshi_ more into a tool to create fakemons
Details
Files
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.