Jim Lee (DC Comics / Marvel) style LoRA
Making models can be expensive. Do you like what I do? Consider supporting me on Patreon š æļø or feel free to buy me a coffee ā
Been wanting to try this for a long time now. I think this came out pretty nicely and captures the style of Jim Lee quite well. I tested it on AnyLoRA and AbyssOrangeMix2 with my negative embeddings and it's pretty convincing at weight=1 for the offset version and 0.65 for the original (check "original" tab for a comparison). Use the prompt jim lee for better accuracy.
Keep in mind that, while this can make Superman, Batman and Wonder Woman pretty well, this is not a character LoRa but a Style LoRA. Have fun making your own superheroes!
NOTE: I used ControlNet on some of the examples. Check the full generation data.
2nd pic courtesy of sadxzero from the reviews.
How to use LoRA's in auto1111:
Update webui (use
git pulllike here or redownload it)Copy the file to
stable-diffusion-webui/models/loraSelect your LoRA like in this video
Make sure to change the weight according to the instructions of the model (by default it's
:1)
Description
FAQ
Comments (53)
Nice !!!š
Your work is outstanding!! Could you please do "Shunya Yamashita" Style? He has good artbooks:
https://ia903209.us.archive.org/view_archive.php?archive=/33/items/One_Voice_Shunya_Yamashita_anthology_III_Artbook/One_Voice_Shunya_Yamashita_anthology_III_Artbook_jp2.zip
easy, added to the backlog
@LykonĀ You should do a guide. I think by far you're the best LORA maker here.
@Lykon thank you very much!! Some heroes don't wear capes but hoodies ;)
Amazing, cannot wait to try it!
Well, another Lykon file drops and my Lora folder gets even bigger, great work, will you consider doing a Jack Kirby or Curt Swan Lora next?
Curt Swan would be amazing! Or Neal Adams!
I'd love it. Those datatsets however have to be done manually (like this Jim Lee one), can't be scraped. It takes time
I am so happy to see this. Would love to see more from western comic artists like Todd McFarlane or J Scott Campbell.
Yeah, but they're not easy to make as I don't know of any good automated way to scrape, so I had to make the dataset on my own manually.
š„š„š„š„š„
Hoping somebody can help get better results here! I've downloaded the model, selected it in the lora tab, copied some of the prompts here exactly for testing, including the seed, cfg data and all that, but fail to get anything like the results I see on the page! I'm using SD 1.5.
You need to use the base model I indicated in the description. Or any nooru model. SD1.5 is terrible for anything now
Same here. I must be doing something wrong.
@DanAI9000Ā try post some generation data. Impossible to help you otherwise
@LykonĀ Oh. The description indicated SD 1.5 - so I used that. I tried searching for 'nooru' model now on the site and got no results. Can you name specific models? Thanks!
@righter831Ā that's just the stable diffusion version, not the base model on which to apply the lora. It's basically to differentiate the 1.5 architecture and the 2.1 architecture.
@LykonĀ Ah! Okay! So what do you mean by Nooru models? Can you elaborate or give examples so I can search them out?
@righter831Ā booru. It's models that use booru style tags instead of captions. See danbooru.com or chan.sankakucomples.com
@LykonĀ So, in other words, it doesnt work with the things listed on here, this site, but rather some wack additional thing. Thanks.
@BramagolaĀ no. What are you talking about? This is a lora trained on nai based anime models. You can use any anime model you want. Abyss Orange Mix 2 gives great results.
@LykonĀ Got it working and got very similar results using Abyss Orange Mix 2 :) One last question, and I hope you don't mind - but I see that your prompts (both positive and negative) are very elaborate, far more than anything I've ever input. Are you writing those manually or using some kind of tool to build them?
@righter831Ā neg one is standard, positive is usually from wd tagger or danbooru
I know this is an old post, but I've gotten very good results from this Lora using multiple models (including the suggested one). perceptron_v20 is a model that's given me some really good results. I also found a lot of great models by looking at the generation data on the images of other users who have used this lora. Hope that helps!
@LykonĀ Hi. Thanks for great job. Wanna say it`s NOT obvious that Abyss Orange Mix 2is need to be used or smthn, I thounght it`s SD1.5 as well.
@cbccbd666Ā that's why I always write ckpt models in the lora description.
Hi guys, this doesn't work for me I try everything that I know. The model displays a very ugly picture of a lady as a super hero no the comic style that shows the model, can someone help?
Impossible to help you like this. Provide the generation data at least
Where do I share the result image?
@spadron81678Ā just copy and paste the pnginfo. My guess is you're either using a wrong base model and/or you didn't load the lora properly
@LykonĀ Hi, ai upload the model into the lora folder of my automatic 1111 Iām using the colab version for regular ckpt models I put it in the model folder / stable difussion and works perfect. What base model do you recommend? or how do you think that Iām doing wrong?
@spadron81678Ā I usually outline ny recommended base model in the description.
Please do one LORA from "the invincible" comics drawing style.
All my results seem to be super blurry. Anyone know how to fix?
upload one on imgur so I can try to understand the issue. Or contact me on discord.
@lionbro32473Ā uhm what base model are you using, with what prompt? I suggest you use abyssorangemix 2 as base
@LykonĀ Alright, I was using abyss orange nsfw iāll try the mix 2.
Hi, just wanted to say thanks for this LoRA. Also wanted to ask if you could add other characters from the Justice League (Green Lantern, Martian Manhunter, Flash, Hawkgirl) it would be really very nice to try them. Thanks.
Thank you. This is not a character lora. It picked up characters that Jim Lee draws a lot, but that's all. In order to achieve other characters you'll have to make a character lora or embedding and use it in conjunction with this style one
Absolutely incredible job on this, dude! I'm a professional artist that uses ai art tools to generate my own reference material inspired by creators/styles that I'm already influenced by, and this lora is incredible for that purpose. Also, this is super versatile - it's worked really well for a lot of different model types. Seriously: great work. You and folks like you are such an important part of this community, and I feel like all of y'all get more complaints than compliments. Keep it up! Kenneth Rocafort is an artist that I'd love to see you tackle - just figured I'd throw it out there lol.
I don't know what I'm doing wrong but using the Offset Lora brings out some good looking outfits and such but it cannot seem to make any good faces, they turn into smudges. I've tried AnyLora, AbyssOrangeMix2, AbyssOrangeMix3 and YesMix and it keeps giving me non-faces! What am I missing?
If it helps settings I've been using are: Sampling steps 40, CFG 12, DPM++ 2m Karras, clip skip 2, SD VAE is KL-f8-anime2.
Update: I seem to have slightly improved the faces by making the prompts not include legs or background descriptions, making it only be upper body. The faces are not as bad but still smudged to a degree, but even so i don't udnerstand why they would do that anyway.
active Hires. fix with 2x upscale with Swinl_4x (it will take more time to render but it worth it) and you will get good quality on everything. it also depends on what checkpoint you use. but this method works on everything.
@2kmerchĀ damn then theres a problem there, I can't use Hires fix at all x-x I have tried using it multiple times and it will start to generate the images only to immediately cancel everything once thery are done and says 'CUDA out of memory. Tried to allocate 2.25 GiB (GPU 0; 6.00 GiB total capacity; 4.59 GiB already allocated; 0 bytes free; 4.77 GiB reserved in total by PyTorch)' when it gets to the hires fix (I have a 1660Ti). I've tried it without any custom launch paramteres, with Xformers and with that and MedVram but it still does not work for some reason.
Am I missing something or will that feature just not work? If it helps I run all image generations at 768x768 no matter what with 2 batches of 4 images.
Update: So it seems that the only way to get it to work is to reduce the size of the images, I had to go as low as 256x256 (max of 320x320) in order for it to do more than 1 image per batch, which is real small lol. The struggles of having a "lesser" gfx card :p
Full body images will always make faces kind of aproximated depending on your base model. The easy solution is highres fix and inpainting "masked only" on the face. It's what I usually do.
@HybridwolfĀ You could try running your Stable Diffusion in "Low VRAM Mode" but even then, if you have less than 8GB of VRAM it's gonna be a struggle.
@sylentwulf81Ā Forgot to update here about this lol. So I managed to fix the problem somewhat, the problem was fixed by adding "--precision full" and "--no-half" to the launch parameters and only doing 1 image per batch, I used to do 2 batches of 4 images now i do 8 batches of 1 image. Now I can run images at 512x512 with hires, apparently running it below that pretty much does nothing/very little. They look much better now, but I will look into the Inpainting suggestion from Lykon too.
The other setting I had was "Xformers" and "MedVram" as without MedVram it started to have issues. I also believe I have about 6gb with my 1660Ti, unfortunately can't afford an better card atm x-x
This model is really great, thank you ļ¼
My mind is just blown...this is amazing. I wish I could triple like this one.
This is one of the most stable western comic LoRAs around, it flattens everything out perfectly but doesn't burn or force styles until super high weights - looking at the json, I can't see the training settings as they seem to have been discarded during the offset process - is there any chance you could post the basics like lr_scheduler, network_alpha, network_dim, steps, etc?
Works great for adding shading detail to outputs.
I'm trying to get something close to the Wonder Woman/Superman/Batman images, but I keeping getting more anime faces. I believe I am using the recommended Sampling Method, as well as the correct checkpoint. I'm new to this, so there is probably something simple I'm overlooking in the settings. The pictures come out great, just more anime than Jim Lee. :)