Are you using DPM++ 2M Karras sampler by any chance? Pony Realism is very picky and won't work with most samplers. On CivitAI you can use Euler A (cheaper, but adds some blur and less detail) or DPM2 A (more expensive, but better details). Locally you should use one of SDE samplers as recommended in the model description.

Also put ‘blurry, blurry face,’

In negatives

@ZyloO I did ZyloO. I mean no disrespect. I just can't get the results I'm seeking. I'll try again. The only thing I didn't understand was [Use Danbooru tags]. No idea what Danbooru tags are..

@BigSad11 Probably means use tags that are common on the website Danbooru

@Polraudio Thanks! Didn't know about that website.

This kind of grainy is also in https://civitai.com/models/139562/realvisxl-v40?modelVersionId=361593

Thanks for all the replies! I figured it out. It was the sampler and my prompting. I was still getting grainy on DPM++ samplers when I wrote this which is why I was confused. But, after altering my negative prompt, it went away. Errors and artifacts in negative prompt seem to do the trick.

this model looks like a painting on smaller resolution too, sometimes, even in higher resolution

@ZyloO alright ima test it out again, I have tons of prompts, maybe that's why it's acting weird with "white shirt" and somehow confusing it with white skin.

Danbooru tags I believe are just the tags that the r34 image hosting sites go by, We should be using those in our prompts I believe

It was the wrong VAE file. Swapped out for sdxl and it works.

wondering the same thing. version 2.1 wont show up in my checkpoint loader :(

@ZyloO yes

single prompt legnth and prompt strength. not too long or high

Inpaint works and match with bodies but it gives me blurry results in fooocus, I don't know why :C Using Forced Overwrite of Refiner Switch Step with a value of 2 fixes the blurring issue but the painted parts don't match with the body and and it gives me bad results.

@michifuzote Yeah, that's definitely an issue. Try turning the "Inpaint Denoising Strength" to between 0.75 to 0.95. Leaving it at 1.0 sometimes works for me, but a lot of the time it causes misalignment. Also, try disabling the Inpaint Engine in the "Inpaint" tab (personally I leave it enabled, but it may help depending on the image).

The other settings I use are: Guidance Scale: 7.5 // Image Sharpness: 2 // Sampler: dpmpp_2m_sde_gpu // Scheduler: karras // Sampling Steps: 32-40 // Loras: None // Refiners: None // Styles: None // Inpaint Respective Field: 0.618 (increasing this setting could also help misalignment)

I get really good results with these settings but YMMV. If these settings don't work try playing around with the other settings. Eventually you'll probably find the ones that work for you.

this looks fantastic! any other fun cartoony tags you found work well?

Easy. Just copy the checkpoint to Fooocus/models/checkpoints and you are good to go. Use default or ponyv6 and uncheck all styles. Change model to PonyRealism. Uncheck loras. With CFG 7 I get the best results. Don't use FooocusV2-Style. It is AI based and works not well with Pony models.

Same here. Pretty much the same face always, only slight variations in age. The only way I've been able to figure out to get different faces is using character loras. Doesn't seem to care about any kind of descriptions at all.

It is a Pony model. You have to describe what you want. I get 1000+ different faces without any lora. If you do not vary your prompt, the faces will be similar.

@Ananda can you please share some advice on how to get different faces? I am very new and having the same problem as the other commenters, and have tried many different things that I thought should work, such as ethnicity, nationality, age, describing specific facial features..... but I am always getting small variations of the same face

@johnsonshardware I use Fooocus local on my PC. This has many styles to choose from. Each one gives me different faces. Also ethnicity, nationality, young, old, teen, mature, aged, hair style, hair color, body, species, country, dress, clothing, punk, hippie, asian, european, african, anime, kawaii, fantasy, futuristic, tribal, native, modern, fairy tale, manga, elf, adorable, sweet, ugly, all kinds of danbooru tags, character tags, etc etc. You have to experiment. Pony models (if based on Pony V6) are very powerful. If you know the tags, you are unlimited. Prompting is different from SDXL or 1.5 models. Avoid complex negative prompts. Keep them as simple and short as you can. They can produce conflicts and ruin your variations.

It is a Pony model. Pony models are based on XL. There is no Pony SD 1.5 model.

I never had a problem with ears. I leave them untouched and they are looking good. I mainly use 1024x1024 and no adetailer. If I get bad faces/eyes/ears due to far away characters, I use at the end of the prompt "close face" which fixes it. If you use "ear" or anything related in the neg. prompt, you create a conflict, and the picture will look worse in most cases. Long neg. prompts tend to ruin every picture, so try to avoid them and keep them as short as possible.

I've noticed this issue as well, but only when using iterative upscaling from the ImpactPack in ComfyUI. This happens both with the image and the latent versions of the upscaler. I don't think it's an issue with this model since it doesn't happen when I chain individual KSamplers in the same configuration with the same number of steps.

@ZyloO gotcha in Karras as always?

I had some trouble outside of euler a.
Here is an example I saw from someone else with the same texturing all over.
https://civitai.com/images/23178383 euler a
https://civitai.com/images/23178274 DPM++ 2M Karras

ive been having the same issue. you could try uploading the model into your gdrive and link to that in colab.

@GreedyDoy i tried it, but it is still giving me error, can you point me to the right way if you have done it before ?

If we're lucky he'll do a remake for Pony 7. heres hoping

@ZyloO Very interesting. Maybe that's something that could be better communicated on the model's page? Because to me Vixon was just making a bunch of anime styles which don't interest me, so I would never have thought to even look for and then mix something from that page with a realism model.

@ZyloO For sure most don't, but people who want to get in-depth and really use the model for more involved things (I am currently using it to do a rough 3D storyboard because Pony Realism actually has the ability to convey facial expressions and body poses) are probably the 1% that will come up with the most interesting gens and also the most likely to need a guide. Anyway... something for you to think about. But this is not criticisms or anything, so keep up the amazing work, it's appreciated.

I used to get this randomly after a gaming session and trying to render a batch of images that was to large, Example, I can do about roughly 20 images or so (Across a few batches) with out errors, but if i tell it to create 30 or so images, sometime into the process the CUDA error will pop up, also telling i to generate a different seed for every image in a batch can increase this greatly, My guess is you get this when your buffer for your vram is full.

@Zoophilian Thank you very much. I’ll follow your advice and reduce number of images. Would it mean that I should restart PC before rendering?

@ZyloO Where I can find it?

Now it makes slightly different images but still I don't see a reason why to use it. It's almost 99% the same as inpainting with the 'ponyRealism_v21VAE' model. So either 'sd_xl_base_1.0_inpainting_0.1.safetensors' is not working with pony or the merge failed.

Thanks for a quick and thorough reply. I'm relatively new to all this (been learning about AI generative images, LORAs, fooocus then A1111, etc. for about 9 months) and was only recently learning about textual inversions and embeddings. And apparently one of them is a small file that teaches a concept to an existing model (if I'm remembering correctly). Would a 'deep penetration' embedding or textual inversion help with this issue?

What I meant about with cock size is that in certain poses it will change, so the pov blowjob example you gave works, but when changing to a penetrative position, it's hit and miss, but usually shrinks compared to what it was before, even if I prompt for 'huge cock' when previously 'big' was suitable during bj. I just wonder whether it might scale based off the size of the recipient, or whether it has to do with the 'source' prompt e.g. previously I had been using 'source_cartoon' as that's what another user who's prompts I based my images off used, but then I removed it more recently to try and get more realistic appearance, and now I'm wondering whether it's trying to keep things more realistically proportioned (I'll have to experiment).

I'm in the middle of reading through your Pony Realism Compendium, but I've had other things needing my attention, so sorry if you've mentioned all this in there.

@ZyloO oh crap! About the 'mouth closed' thing - I just checked the danbooru site and it doesn't have an entry for 'closed mouth', so the official tag is 'mouth closed'. I must have picked that tag up from somewhere else, but noticed it closes the eyes too. But in fairness, I didn't actually say I had used the danbooru tag in that instance in my original post. Sorry for the mix up!

I didn't realise you'd only just released this guide. I've been playing around with this model for weeks now and came back to see what other's had been using for prompts to iron out these issues I'd mentioned, and by chance was in the right place at the right time. Like I said, I'm new to this PonyXL stuff after learning all manner of other stuff over the past months, so I don't know if there's other info out there regards 'source_cartoon' etc, in Pony base model info stuff, but that would be useful in the guide, maybe?

And also the 'score_9, score_8_up, score_7_up, score_6_up' stuff, as I see different users using different amounts in both prompt fields. I'm guessing the training images had been given quality ratings and the user is keeping higher ratings, and eliminating lower ranked ones in the negative prompt? (This info may be out there somewhere and/or common knowledge to long time users. I just haven't got round to finding it all out yet. So don't write a massive guide just on my part.)

@ZyloO Thanks for the link to that guide, and your help answering my questions.

Hi, the age references do generally not work very well, but there are a few LORA's out there that will do the trick. Just search for "Age Slider" and pick one that works with the Pony base model. You can then adjust the age by a factor that you apply to the Lora. With that you can very easily convert any character into a 30 year, 40 year, or 80 year old person. There are quite a few age sliders out there, some even specifically designed for old people.

Try something like:
mature, grayish hair, GILF, (55 year old), (age spots:0.3), (skin discoloration:0.2), wrinkles, body wrinkles, arm wrinkles

Then, in A1111, must I select No VAE? I am getting very grainy pictures...

@tono4 Wondering if it's resolution that's playing a part in that? The VAE option I think is blank or No for me too and tends to work. AI generation often breaks when the resolution is an odd one.

@ZyloO On automatic1111? or a setting that is literally 'automatic'? I use ED, so I'm not sure if other settings exist in that. I tried to install a1111, but it refuses to admit that I have pytorch installed, even after several tries and making sure I was using the version it was asking for.

@ZyloO why? that sampler is recomendet, I'm just trying to understand, and don't repeat mistakes

@ZyloO I use comfyui, very grateful for the advice

just download the main21vae version, the vae included in it

@ZyloO Sorry, I'm not at all experienced, wouldn't training a LoRA involve having already generated many realistic images of that character? I'm struggling to make one. I'd use cosplay, but the overwhelming majority of cosplayers are Asian. So far found out that adding freckles tends to make the character look more European.

Have you tried recreating an image using someone else's prompt and settings?
The biggest impact seems to be in sampling method; it seems like the majority of the successful ones here use Euler a, even though the recommendations suggest DPM samplers.

@oddlittlepixel From what I understand, Euler a is recommended for on site generation, if you generate on your computer, then it's better to use DPM.

It was VAE in general A1111 settings. I also set it to None and it works perfect!

Thank you for your answers

@tono4 Good ol' Vae grains.

yes it is

Due to its nature, Pony-based models need certain controlnet models - heavier ones. Most don't work properly. I suggest controlnet-union-sdxl. It's essentially all-in-one model. I tested it extensively and it works well enough.

@Miragenerate thanks, I'll try it

@ParadiseGoddess Has it worked?

I guess it's because the AI thinks of "legs" as bare legs, not legs covered by socks.

i have trouble with from outside view balcony, very hard

So any Pony Lora, and SOME XL Loras will work?

@Deerobouros  Pony base Lora, yes. XL based, depends, concept shoulbe be ok, characters may not work as expected. Embeddings may not work at all.

@ZyloO So then you know when extension parameters ask you to go by certain size based on the model 512, 768, 1024 which size does this go by?

agree with both observations, great model, backgrounds are a challenge

я думаю люди делают это по инерции, никто не читал детально что там создатель пишет, все просто копируют какой-то промт который все это начал))) ну а потом это зашло так далеко, что некоторые создатели моделей следуют этой идее

@ZyloO It worked. Thank you very much!

@ZyloO May I also ask you, what prompt words can make the picture get rid of the CG picture texture and get closer to the feeling of real photography?

@ZyloO But the description says

Recommended Parameters for Local Generation

Samplers:

Recommended:

DPM++ SDE

DPM++ 2S a Karras

DPM++ SDE Karras

DPM++ 2M SDE Karras

DPM++ 2M SDE Exponential

DPM2 a

DPM++ 2S a

DPM++ 3M *

@ZyloO But it happens with all DPM I tested all of them after I wrote it

@ZyloO Yeah I checked the VAE, tried with none and the other VAEs. I don't have that problem on realistic models such as Everclear or Pianomix but these don't give a better quality than Pony Realism with Euler a. Thing is that with DPM++ Pony Realism produces really detailed backgrounds, plenty of leaves on bushes and trees and so on where other models produce quite simple backgrounds. However the whole image turns out to get these artifacts. I saw some examples in the gallery here that have the same problem. If you look for examples using DPM.

Some SFW examples https://civitai.com/images/24179787 https://civitai.com/images/27683546 https://civitai.com/images/27676639 https://civitai.com/images/27539199
I find these images much better in details but it seems the model has an affinity to create particles, oversaturated/contrast colors and oilcolor texture.

@ZyloO  I have the same issues on local generation, default VAE, and the samplers you have listed.

From time to time, I manage to get really good result (it's extremely rare), but most of the time, the backgrounds are overdetailed which results in a blurry mess, and the characters have weird artifact on their skin.

I also use the recommended resolution (1280x768), recommended steps (30 max),etc...

Edit: and of course, after writing this, the generation managed to create only good images without artifacts, I don't know what is causing this.

@lnomsim It seems the model just doesn't work well with DPM++ 2M and similar samplers. Which is a pity, because they generally produce better detail than the Eulers. It is not the only realism Pony model with this issue tho. My workaround now is to run first with DPM++ 2M and then HiresFix with Euler a. Preserves some detail, need to experiment with denoiser.

You probably have to stick to 1.5 with that rig.

Increase virtual memory to 64GB or more

Youll need to enable low or med vram in the command line arguments if your using auto1111, it will run but it will take about a minuite or 2 per generation.

brother in christ you dont need 40 steps

dpm2 is very slow. it can give good results at after less than 10 steps. author recommends to use SDE-samplers. like with "dpm++ 3m sde exponential" at 55 steps picture renders 30 sec, and loras don't influence that time

@raphielle I started using comfyui and the time reduced a lot, even with 40 steps I get the same result at 1920x1920 resolution in around 5 to 6 minutes with a workflow where the image is generated at 1280x1280 and enlarged with img2img to 1.5x.

I managed to solve the speed problem hahaha :D But thanks for the help :D

@oddKoodoo I started using comfyui and the time reduced a lot, even with 40 steps I get the same result at 1920x1920 resolution in around 5 to 6 minutes with a workflow where the image is generated at 1280x1280 and enlarged with img2img to 1.5x.

I managed to solve the speed problem hahaha :D But thanks for the help :D

@TheCAL With Automatic1111 I was having problems with speed, today I'm using Comfy UI with a workflow that generates the image in 1280x1280 with 40 steps, 7 cfg, DPM Karras and then enlarges the image 1.5x to 1920x1920 with the same configuration but with 0.5 denoise, and the whole process usually takes 2 minutes.

My problem now is being able to place more than one character per image, right now I'm trying to place Anna and Elsa from Frozen with different clothes, but without success... At least not with the clothes I want, they are mixing everything... I haven't been able to use BREAK yet. Although my prompt is quite complicated. If you can help me, I can share my discord.

@Belethian there's extention for automatic1111 called "tiled duffusion", very popular. for comfyUI also should be a version of that. it helps with regional positioning of elements

@TheCAL experiment with everything, for automatic1111 there's "script" tool, that allows to draw comparative grids of images generated on same seed with some different parameters - like sampler or scheduler. it's better to choose different sampler for different model and object on picture

@oddKoodoo Thanks!!! Now I just need to learn how to use it hahaha I'll look for a tutorial ^^

@Belethian Hey what are your pc speccs and how did you fix your speed issues? My generation time switches alot, sometimes im done in 20 seconds sometimes it takes minutes to generate at 512x512 using euler a but the results are usually nice.
Ty in advance

@jaggy700 I currently have an I7 9700Kf, 32gb ram and a 12gb RTX 3060. About what I did to solve the speed problem was just swapping the a1111 for comfyui. Right now I have a queue with 3 images, not simultaneously, but using comfyui I can set up a workflow with 3 different prompts. Two of them with 1080x1280 images and one with 1280x1280, but all 3 are enlarged by 1.5x, in the final result I have 2 images of 1616x1920 and one of 1920x1920.

But in the process, the image is created, then it is enlarged by 1.5x and as soon as the result comes out, you go through an img2img process with the same prompt but denoise of 0.5 to correct some errors and add more details. A refinement. The entire process usually takes approximately 8 minutes. Ah, all stages of the image go through 40 steps, both after enlargement and initial creation

@Belethian well, that's a nice workflow, but I really don't believe that comfyUI gives any benefits in speed. I think if you really compare a1111 and comfyUI at same settings them would be equal

@oddKoodoo On the a1111 I was using it without lora, any lora I added would take the process from 2 minutes to 18. And on the A1111 I couldn't use hiresfix, for the same reason as lora, the process took much longer. I had to generate the image at a lower resolution, send it to img2img and then enlarge it, but not much.

The whole process, in the end, took about 30 minutes per image.

I don't know what changed, but the settings are the same, dpm2 to karras, cfg 7, 40 steps, 1280x1280 or 1080x1280. The difference is that I can make 3 images with a single click ^^ each one with a different theme and prompt.

I don't understand programming, on this subject I'm a real noobie. I don't know why on comfyui I can do it faster and still use the PC without many problems, I watch videos, I can even broadcast the screen on discord without many problems...

@Belethian wow, the timing you've described for a1111 is really insane. for me it happens much faster, and I find a111 just more comfortable and detailed. mb you were using some wrong settings somehow or very early and raw version of a1111... however - if you're satisfied with your current comfyUI setup all is good <3

@Belethian thats the thing i dont get aswell. While using a1111 i get very big swings in time. Sometimes it takes me 5 seconds to generate 512x512 image and the next time it takes me 5 minutes without changing anything.

Yesterday i played around abit with token merging and my times are more consistent with around 20 seconds for a 1024 x768 image.

But upscaling still takes very long.

Do you have any tips for speeding up upscaling?

Im pretty noobie aswell when it comes to programming

@jaggy700 I think a1111 strangely works with VRAM alocation, it helps to keep less applications lauched

@oddKoodoo it just doesnt make sense to me. My taskmanager shows nothing is at max capacity and it still takes very long sometimes.
Are you using A1111?
Which upscaler do you use?

@jaggy700 upscaler-model by itself isn't important, matters the method a1111 uses to upscale, and it's standard. there's common problem with memory allocation. like - an app needs to operate another 250mb of mem, and it's reserving for itself about 4gb of mem - just in case, you know, why not? and on that moment it can crush with error "out of memory" or just work MUCH slower. relaunch of app helps, and just don't use simultaneously several memory-hungry apps

@jaggy700 and try alternative upscale-methods - "tiled diffusion" extension, controlNet tile and others alike

@oddKoodoo will try ty for the tips :)

What UI do you use to generate images? A1111 is super slow (>10 minutes?) on my 8GB 3070 laptop, while Forge or ComfyUI can generate an image in 20-90 seconds depending on the sampler you use and LoRA usage.

Forget LCM.

use Hyper SD lora, start with 8 steps for sdxl with 0.5 weight and CFG 1,worked flawlessly on pony

https://huggingface.co/ByteDance/Hyper-SD/tree/main

Ditch A1111 those thing is fill with OOM error,lol.

Use sdwebui Forge or SwarmUI.

try putting "plain background" in negatives

I know that you said you don't want blurry backgrounds, but have you tried including "(bokeh:0.4)" in your prompt? You might be able to scale that up or down as needed depending on how much blur you want of the background. As I understand it, shallow depth is pretty much the oppose of bokeh. Unless you're talking about the concept where parallel lines converge like looking down a long hallway.

the face issue is common. I use (beautiful Asian female:1.8) but have to increase the weight as i add other details.

Check my comment. This is due to using the wrong sampler. I had the same issue. Now the faces/eyes are literally perfect.

Same issue happening to me

Thank you. I was getting horrible results till I read the description and your comment.

by refiner, do you mean upscaler? Where would i find refiner 5? Thank you

@Colero apologies that is for local renders. This is the most used refiner for SDXL/Pony - https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0 when the generation process gets X amount through it switches to a different checkpoint to refine details and slightly alter the output. This can be defined even if you are using a cloud Stable Diffusion Forge or ComfyUI instance :)

@wodz30 Hello, if you use ComfyUI, could you please share your workflow for reference?

@QHvI7vwtWD sure thing! I have upload the json here for anyone to use - https://file.io/VgbeIc2qAyUn

@QHvI7vwtWD Here is my img2img workflow as well, I keep these separate - https://file.io/xA2874jH9Byx

@wodz30 Thanks, it looks like the KSampler node you're using doesn't load for me, and it doesn't show up as missing in the Manager. Apparently most people don't use the refiner anymore? I'm kinda curious but not sure how to set it up

@QHvI7vwtWD I believe it might be the ComfyUI_Fooocus_KSampler the refiner seems to make a better difference for pony rendering compared to latent upscaling

@wodz30 could you reup pls

Was going mad about it, tried to replicate settings as in the description and still not getting the results I was looking for. Even passing the images through my sd1.5 detailing and upscaling mad such a difference in making them pop! Will give your flow a shot later. I see the flows are deleted by now 😢

text2img workflow @Kadse https://easyupload.io/db0h9i link is good for 30 days :)

I've reupload Wodz30 text2img workflow here - https://drive.google.com/file/d/1lxUle8SyU0L2Et8WsLm64BaNjd7_J8VB/view?usp=sharing

Happens when you're denoise is too high. But extra belly button is actually really easy to fix in post processing. gimp/PS

@ZyloO a new version of pony realism would hit hard🤪

Yo, the new model is definitely a step up overall, no doubt. But honestly, some of the little things that got changed kinda threw me off. There were a few details in the old version that I was vibing with more, and now it just feels a bit off in spots. Don’t get me wrong, it’s solid, but I miss those touches from the last version.

@Cheemz99 which new model?

@SavgsCurv I’m not talking about ponyrealism, I've messed with other models before.