Introducing RunDiffusion XL Beta!
RunDiffusion.com has done it again! One of the worlds first SDXL Models!
Join our 15k Member Discord where we help you with your projects, talk about best practices, post exciting updates and news, and share the amazing art we generate.
Why? Isn't SDXL already amazing?
Yes! We feel this is a step up! SDXL has an issue with people still looking plastic, eyes, hands, and extra limbs. We couldn't solve all the problem (hence the beta), but we're close! We tested hundreds of SDXL prompts straight from Civitai.com (using ComfyUI) to make sure the pipelines were identical and found that this model did produce better images! See for yourself here.
The model tends to favor closeups or "headshots" more than full body unless you explicitly prompt for that. Not sure if that is a desired feature. Open for feedback on that.
Strengths
More photorealistic for animals, cars, and people.
Different Model, different generations. Try it with your favorite prompt!
Better portraits. SDXL likes to put the subject way far out. This brings things closer.
Eyes and hands are a bit better (from our testing)
Better prompt control. Make sure to use commas and break apart subjects and styles!
Weaknesses
Still a few concepts that are "plastic" (Food, but please test your prompts and let us know!)
Can sometimes be too "person bias"
Creativity is a little bit hindered. This could be considered a "Photorealistic" model. But again, TEST!
Bias towards "sports cars" lol
Notice the RDXL follows the prompt a bit better. When asking for "upper body" it tends to follow that.
How to Use
Prompt as usual. Use commas to separate subjects and styles. If you aren't getting what you want, use the token again. e.g.
iron man made of lava, suit of armor infused with molten lava, ...etcif you don't get Iron Man in your generation, then put that token somewhere else, `iron man made of lava, suit of armor infused with molten lava, marvel iron man super hero ...etc`Use the standard refiner or the 1.0-0.9 refiner. https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/blob/main/sd_xl_refiner_1.0_0.9vae.safetensors
Use ComfyUI
Our Process
We used a small dataset of 700 images and trained for 14k steps at 20 epochs. Manually captioning everything to be as descriptive as possible. It turns out that SDXL is pretty receptive to training! The changes we were able to make prove to be extremely exciting. The future is bright for SDXL!
Future Plans
We want to make generations easy, so we're bringing ComfyUI to the platform so you can simply drop in images with metadata to generate with one click. This is the future of workflow sharing and we're very grateful to /u/comfyanonymous for his help with getting this working on our platform. Also a special thanks to /u/Searge for his amazing work on SeargeSDXL. We love that workflow! (Could we work on getting the generation data to work on Civitai.com?)
We will continue to work on this model from the feedback we receive. Get ready for a full release in the coming weeks.
Description
FAQ
Comments (51)
That's for the breakdown in the description people often don't understand how helpful these things can be.
Guys, please tag your model with "SDXL" so it appears in the search and filter properly. Cheers
You can Also turn on "1.0" in the top right filter .
Following your logic, we should also rename the bajillion SD1.5 models so that they have "SD1.5" in the name ?
XL is the PROPER way to refer to the new SD version (1.5 > 2 > 2.1 ... > XL)
SDXL is only the base Model : Stable Diffusion XL.
Putting SDXL in all the models to come is : nonsensical. When a model is made out of 25 merges, you don't put in the name a list of the recipe...
SDXL is ONE model only. Published by OpenAI.
All merges and fine tuned models to come based on that base model only "need" XL eventually in their name.
What is the process for running this in A1111 please? Do we need a refiner or are we good to go?
No need for refiner. If your Auto is updated you should be good to go.
Can I ask also is there a planned inpainting version?
No plans yet, try inpainting with it though! It's not bad, much better than default 1.5 inpainting.
Im using A1111. The results are great, but when i run the refiner, either 1 or 0.9, i notice almost no difference or upgrade. im using denoise 0.1. am I doing something wrong?
You can go all the way up to 0.3 before it starts to ruin the image instead of enhance it.
The refiner is kind of weird imo - it's trained for the base model. I've been experimenting with it but haven't got anything consistent. I think the devs of Auto, SD.Next, etc are still figuring out good workflows for the refiner. Some good experiments happening with Comfy but not sure anything has been decided upon as the "best" method.
Your mileage may vary but I mainly use A1111 and I stopped using the refiner with custom models. It really isn't needed IMO and cuts down on generation time. It's useful for the base model but I also don't use that anymore.
@wideload Personally I'm using RDXL for the base and Hephaistos XL6 for the refiner. Works pretty well imo. but I'm on ComfyUI so I've got a bit more freedom with how I set it up.
@Triple_Headed_Monkey Yes, I've heard of people doing similar things and plan to try it myself. I've been switching between a couple of main models, Angra and Copax mainly but I d'loaded RDXL earlier, just haven't got around to trying it yet.
@wideload Hephaistos in particular is a merge between the Refiner and Base model that's been fine tuned and trained a bit on top. It's pretty decent imo.
There are models in 1.0 that no longer use a refiner and that should be a trend for the future. It must be a single model without depending on a refiner
@marcoescobedo If anything the trend is going to be more models. Sorry to disappoint.
The majority of users have moved away from Automatic1111 in recent weeks towards UI's like ComfyUI. And because of this, both base and refiner models can be run in a single click by most people.
There simply isn't a real justifiable reason to remove the refiner step from the process for most people, because even on the custom models like RDXL, it still enhances the image. And it only takes a few extra seconds.
This model is the best SDXL one so far! Using it with ComfyUI with their "example" setup for SDXL with my own minor changes. GJ!!!
where do you find their "example" setup for SDXL? i am new to comfyUI
@infolk https://comfyanonymous.github.io/ComfyUI_examples/sdxl/
Just Download the Bottle Image and drag it into the Workflow Area it'll load in the entire Node Setup.
Yeah, this is good. Keep up the great work.
My favourite SDXL model so far, great job!
sorry for the silly question - this file should go into Checkpoints folder right? Not sure why it's not detected in ComfyUI. only my base & refiner checkpoints get detected.
Standard Checkpoint. Just put it into your models folder
I'm feeling that refiner makes it worse, don't know if I'm using it right
ComfyUI with SeargeSDXL is our go to. Try that!
I feel like the refiner always makes it worse, only sdxl base benefits somewhat from it :(
Awesome, I get insane results easily with the shared prompts. I'm really new to AI and I only tried some SDXL models, but this one is my favorite by far. Thank you so much !
You're so welcome!
I see that your finetuning has very good results, can I ask how long did you train on what GPU
most realistic model ever, even better than midjourney sometimes!!
~THANK YOU~
This model is great, I do have some questions though. How’s does one prompt for a full body scene all my gens are close up portraits? I’ve tried a negative prompt for portrait and that’s not working along with weighting (full body:1) and nothing all close in. Last question is there a prompt for controlling breasts sizes?
Thanks for the work folks.
This model has a portrait bias and we’re fixing that in v1
Is there a workflow json file to run this model in comfyUI and if so can this be shared here or in any place so we can do some experiments based on what was achieved already. Thanks for your feedback and wonderful model work
Sure thing! https://rundiffusion.com/comfyui-workflows
Just switch the model in this workflow.
I get an error when using XL controlnet in A1111 with this model, though not with other models, including the base.
Does this model not support controlnet?
ControlNet isn’t model specific. We use this model with ControlNet all the time.
Does this work with AUTOMATIC1111/stable-diffusion-webui version ?
Of course! Demo it on our platform if you'd like.
This is my Go-To SDXL model! Love it!
I'm having problems today using this model, out of nowhere I get an error :c
The model really wants to generate asian people for me!?
Also, always adds weird red freckles/splatters to faces.
Eyes are anime, even when adding "realistic", and "photograph". Adding "smiling" creates fat crazy lips. Help?
Even when I put "european" in pos prompt and "asian" in neg prompt, they always end up asian?
Hello all, Newbie here!
I have 4090 GPU (24gb) with 32GB x2 memory, NVME 2TB SSD. (supposedly the set up is good to go for running SDXL no?) but I get the following message running this model.... what seems to be the problem? anyone?
"OutOfMemoryError: CUDA out of memory. Tried to allocate 6.45 GiB (GPU 0; 23.99 GiB total capacity; 20.24 GiB already allocated; 0 bytes free; 21.34 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF"
Adjust your settings. Start with 1024x1024 20 steps CFG of 7 and generate.
use confy ui or forge ui
Use a Fooocus https://github.com/lllyasviel/Fooocus WEb UI
I wish you'd make the Rundiffusion Photo V2 model public for private use :(
It's a really awesome model isn't it! It's available on our platform for now! That could change though. Thanks for the feedback.
@RunDiffusion It's indeed a nice model but not "I pay $5 for additional server time"-nice. There are enough models here that offer similar realism for free. I think your business model would also work if you make the model available, bcs the people who pay are mostly the ones who don't have a good graphics card.
First of all, thank you for the great model and I would like to know if I can use the images created through this model for commercial use.