@ZyloO I should learn how to read then! thank you 🙏

@ZyloO Hi, is the VAE baked in on the inpaint model? I only ask as it is 1 of 2 of all the models listed that doesn't have "VAE" in the title.

@ZyloO  Awesome! I appreciate you getting back to me

And if i use it as Checkpoint it doesnt load at all.

Errot code: Loading weights [059934ff58] from C:\Users\Gebruiker\Desktop\sd.webui\webui\models\Stable-diffusion\ponyRealism_v21MainVAE.safetensors

Creating model from config: C:\Users\G\Desktop\sd.webui\webui\repositories\generative-models\configs\inference\sd_xl_base.yaml

Loading VAE weights found near the checkpoint: C:\Users\G\Desktop\sd.webui\webui\models\VAE\ponyRealism_v21MainVAE.safetensors

changing setting sd_model_checkpoint to ponyRealism_v21MainVAE.safetensors [059934ff58]: RuntimeError

Traceback

@SkaffaMan This is not a VAE, it's a checkpoint. load it as a checkpoint, VAE is backed in.

I run this model as a checkpoint for XL in Fooocus and it works fine with great results.

https://civitai.com/models/257749/pony-diffusion-v6-xl

Pony was apparently intended to be an SDXL checkpoint trained on a lot of anime pictures, but the dataset used for training turned out to be so big that the model "detached" from SDXL and is incompatible with SDXL loras. Hence a new base model was born.

Pony is as 3Deity posted, a specially trained checkpoint branch based on SDXL. You can also download for free and try it out. It has some prompting specialties, and allows you to access sorted sources of images inside the pony corpus.

Pony uses quality classifiers like 'score_7_up' or 'score_3' to allow users to add quality filters to the classifiers of the sampling process by prompting the scores in positives or negatives. It also adds a very specific nomenclature for classifiers (somewhere about danboruu?) so it is having a rather high responsiveness when you use the right prompts. PLUS it is based on SDXL, so it understands longer and more complex prompts.

Add to that a rigorously harvested and selected corpus for training the whole thing, including not just NSFW but also things like furry, homosexuality and a lot of fetish stuff, which makes it a lot easier to create specialist LORAs for even more weird shit or just the normal shit or just normal pictures, as a lot of anatomy helps to teach a checkpoint to get the legs, arms, fingers and tails and tentacles right. And you can specifically call up realistic, anime or even furry images as specific image sources in a prompt.

@qpdo Thanks for your time, so it is supposed to be used as a check point in SDXL? Does it works with Forge or A1111?

@Dlzeze Yes, Pony Models are similar to other checkpoints you can download and use in Forge or A1111 or ComfyUI. There are two major differences: the prompting and how they react to other SDXL LORAs that are not pony. There are also specialist checkpoints of pony already, like Pony Realism.

@Dlzeze Basically, just remember: Loras where the table to the right says "Base Model: SDXL 1.0" are only compatible to Checkpoints where the table says "Base Model: SDXL 1.0", Loras where the table says "Base Model: Pony" are only compatible to Checkpoints where the table says "Base Model: Pony". That's why whenever I download a Pony model I put "(Pony)" at the end of the file name, similar to how I put "(XL)" at the end of the filename of XL models.

@ZyloO Thanks for the quick response, but I'm using it locally through Swarm UI, It's amazing and very flexible, it's just the grain-ish filter that I'm trying to figure out a way around.

I am also getting alot of artifacts in the background when I use the recommended samplers on my local machine. I have forge installed and results aren't very good with recommended samplers. However, it works somewhat okay with Eulers... except, with lots of noise in the background.

ddim and restart work tho

@ZyloO you are a boss 👍

@ZyloO please, do it, we are neet it, realy, man, you are create a best model of ever! please, give us inpainting version!!

It's going to be monstrous

@ZyloO Nice work

I'm using locally too with automatic1111.

Sampler: euler a - karras or automatic

cfg: 7.5 -8.5

clip skip: 2-5

sampling step: 20-25

resolution: 800x800

For better results put these in positive prompt: score_9, score_8_up, score_7_up, source_photo, source_irl, photo-real

In the negative prompt: score_1_up, score_2_up, score_3_up, source_pony, source_anime, source_cartoon

Faces are not the best, you have to use hires.fix

settings:

denoising strenght: 0.20-0.35

hires steps: 0

upscale by: 2

For upscaler I'm using 4x_NMKD-Siax_200k, this is the best upscaler I worked with.

Some prompts can be hate eachother. That means, everything is fine and you add a new prompt and its ruining everything. Find a synonime for the promt wich ruins everything, the synonime maybe not gonna ruin the end resulst.

@TheReal_Gabe YOU ARE A TRUE GENTLEMEN SIR! Thank you so much for your help. I actually got good results with you method. Thank you once again!

@TheReal_Gabe Did you know the parameter settings in ComfyUI?

@Supamania89 I'm glad I could help!

@xiaoshuisheng24248 Sorry, I'm not using ComfyUI!

Your prompting is probably trying to pull from the less realistic sources or something

@ZyloO That's great, it's a fine model. When I got it to work, I loved the results.

just as confused, I'm thinking prompt strength, so like 35 year old woman:0.8, wearing a bright dress:1.0 or summat.....

@ZyloO I see, so it's a different prompt language than standard SD which takes either (()) weighting or 0-1.0 weightings. Looking forward to messing about with PonyDiff for more control over my characters in fashion and cinematic scenes. Some people say IF YOU DOWNLOAD PONYDIFFUSION ITS A SIN I say fuck it, its a checkpoint with relatively impeccable prompt ability especially with the character control I've seen. Now I wonder how well I can get people in fancy clothing and harnesses....

@StrapsKinks if you dont want to type all of that use this embeding https://civitai.com/models/332646/pony-pdxl-negative-embeddings

@ZyloO Is there at least some approximate time frame?

I wrote a quick article on how to do some very effective inpainting.
https://civitai.com/articles/5957/excellent-xl-inpainting-results-using-two-models

I just shared some inpainting results in my "Posts"

@altoiddealer thanks but it doesn't work well for me, plus the workflow is a bit long but thanks anyway

@rachidlamouchi13790 Yeah it's not practical for just quickly inpainting things. However, if you take the time to make nice masks using Segment Anything, or manually drawing them in Photoshop or other software, with this setup you can process your image endlessly without halo effect (loopback script / upscaling / simple img2img / etc). The method I described is only limited by the quality of the mask images - very high quality masking will yield very high quality inpainting results.

If you consistently get total abstract-painting smears remove one or more LoRAs entirely and re-Generate; see what happens. Remove another. Some LoRAs will conflict with others and it's not worth even trying to puzzle out with them all in there some magic combination--you might find that you don't even need one for the result you want. "Less is more" quite often. In img2img if it's "grainy" I don't have a quick answer, but if you get grainy results in txt2img, two things to try:

1. Just try a different sampler.

2. (Very fucking odd) with this model for some crazy reason if I split the prompt across multiple lines--just press Enter, no need for BREAK text, it just... makes the grain vanish. This should not be. 20+ other models don't do this and tokenization in SD XL does 75 token "chunking" for you. Maybe try that in img2img along with just Euler a, for the sampler.

3. Not related to 1 & 2 nor your problem, but I'm an evangelist: Ditch the cargo cult voodoo bullshit in the negative prompt. Start with nothing except maybe the single word, "watermark." Nothing else. The only reason you'll see strings of nonsense tokens that probabilistically were never originally in the image dataset tags in my work is to reproduce something where someone else copy-pasted that crap. Do voodoo magic copypasta words in the neg. "work?" Well sometimes of course they do, you see it--but sometimes they make no difference or make things worse, so why start with something that might? Thank you for coming to my TED Talk.

i'm unscientifically convinced that each Pony model is a unique snowflake. my first troubleshooting steps are always testing clip skip and eliminating 'natural language' or anything i don't think a model trained on booru tags will recognize. sometimes a simple phrase confuses the model and gives me those color blotches. plus, civitai is a terrible way to learn from others - i'm also 99% sure they've adjusted the API to create good imagery with bad Pony-specific methodology.

then we have all these refined models now that frankenstein these disparate methods and 'rules' together. it's a fun mess, but it's a mess imo

@ZyloO You're likely 100% right about it, more than just opinion. My current install of webui is so mangled with all the extensions and tweaks to settings that some things that historically have worked perfectly, and that I understand well, now crash and burn with esoteric python errors that defy logic. Works well enough that I'm not compelled to reinstall from scratch... yet. Cheers.

@Narz It's a beautiful chaotic mess; like Cthulhu in a pretty floral dress; dare to gaze--you'll then obsess... oh, such a beautiful beautiful mess!

I expprence the bloches yesterday as well for the first time on my 3rd day of using, noticed it may be Lora combination. maybe also attention wieght

@ZyloO I've noticed it on a couple of other PonyXLs too, I think @nofilterman is right about it being LoRA related, with possible additional issues due to samplers. I switched over to your new Alternate VAE that you uploaded and haven't had any issues with splotches yet, though I haven't intentionally tried to get them by cranking strengths or LoRAs way up.

Anyway, that issue aside, this model makes some really good images. I know PonyXL can be hard to push in the realistic direction so I appreciate the effort you've put into this.

@ZyloO you mean attention weight? I did run into mush when I had mean to many promts using weights?
if that's what your mean

You could try my realistic Pony model: https://civitai.com/models/515034/jib-mix-pony-realistic
I haven't noticed any of those issues on mine, but it could be setting or Lora's you ae using?

@J1B Do you just hang out in the comment sections of other (better) models to peddle your own frankenmerge? holy lmao