DreamShaper XL - Now Turbo!

People really expect that they would get best out of model right from release? It took months from 1.5 release to get fine tuned models into what it is today. 1.5 was milked out from all angles and right now the model choise is basically subjective. Yet, again it didn't take days. It took months to reach this stage. Same thing is going to happen with SDXL.

Right but it is a base model. I imagine in the near future the fine tuned XL models will outperform their 1.5 counterparts.

Now just imagine a fine-tuned XL

@squirz The way it was getting hyped up, yes.

It just launched. We have no fine tuned models yet. Let them cook a minute lol

Honestly, I'm getting better results with base SDXL 1.0 than I am with most custom checkpoints I've tried. Maybe it's not a great upgrade for people who just make nothing but endless pictures of girls, but I don't really do that. Getting great results with art styles, artists, various subjects, animals, etc. This is what SD 2.0 should have been.

@captaindingo920 I agree with you!

you need to compare this with 1.5 base and 2.0 base if you want to give it a fair review

if you want to compare the best of the best finetunes of 1.5 then just give it time and wait for some sweet finetunes to come out eventually :)

People are making good points about XL eventually getting more fine-tuned to look better but to me compared to 1.5 XL is disappointing because it's way slower to run on 8GBs and requires swapping models, a whole new 1-2 steps, even this one doesn't require refiner which is cool but still requires highres fix, which I never use because it's also way slower, I don't know 100% how it works but I hope these are things that can be improved, including the model file size.

@darkceptor44 try it with ComfyUI. With that interface I managed to use the base model with refiner on my laptop with only 4gb vram (I had to use 768x1024 resolution though) whereas it took me forever to even load the base model in a1111 and it crashed eventually.

@darkceptor44 it's hard to get it to work with A1111 tbh however it does work with ComfyUI Sebastian Kamph put up a recent YouTube explaining how to install it and effectively link it to your existing A1111 folder so it can access your models there. Simple installation and once you have it installed just go to the ComfyUI Github here and download the 2 images and then load one to generate the SDXL UI. You can then tweak and adjust the prompt to your liking.

same, the model won't load :(

@BilboTaggins loads for me, I just get a cuda memory error when trying to use

on settings stable diffusion in A1111 webui the first two i set it to 5gb each. that solved my cuda out of memory issue and i have a 12gb vram

@vitokeorlini225 Thanks, not sure which setting you're referring to though. I've got an 8gb card myself

Not a fan of generation sites personally and would rather manage things locally. Unfortunately seems I won't be able to try out any XL based models myself for the meantime.

@Lykon Yes I know you were, and that may be helpful to a lot of others. Google colab would be the same scenario outside of an existing workflow for me so not for me 👍

@Ocean3 I finally got it working after installing VS Code, Cuda, xformers, and updating drivers. didn't realize Cuda and xformers were pre-reqs for XL. Big change of pace going from 15/20 seconds a gen to 4 minutes though....

for me it was composable loras, which broke 0.9 already. after disbaling it ran perfectly fine

@Lykon Yeah but you have to change the prompt while changing the seed only is insufficient. For example, adding ethnies to the prompt works well to get different faces.

@Lykon ahhh,ok,take your time 😅

"Also it's too early now. XL needs good base models first."

What about your XL base model here? :)

Don't chase the dragon!

haters and jealous people are always around :)

if you have <12gb vram, youll need to use medvram command

@Lykon nice bro

I am using a RTX 4070 12 GB and Comfy no problems except I use 1.5 model as refiner and try to upscale to 4k...running out of memory because 36GB is requested but not existing

a1111 with rtx 3060 12gb here. works only with medvram for me yet, but it works

3070 6GB and Im able to use it --medvram in parameters though and --xformers too

Not ready for primetime. (SDXL as a whole) rtx 3060 12GB as well. Not in A111 anyway. They'll get it optimized and A1111 will learn to use these things better eventually.

@Lykon I read through your description twice just to be sure but I don't think I see a model name for the upscaler you are using to upscale to 2x,

He's using 8x_NMKD-Superscale_150000_G, you can see in the comfyui workflow.

@xperia256 Thanks! Mine currently says null, but that's probably because didn't have any upscalers in the folder.

where is the comfyui workflow?

wow, XL for a clip looks very promising!

@Lykon oh, ok, I guess I spent too long checking the base SDXL models and I assumed all were fp32 (since it's marked fp32, even though it's the same size... I guess the info was added wrong on that model? or is it the extra training data, I guess?).

I had a similar problem. The solution for me was to update my automatic1111 instalation. Using the cmd in the folder where the "webui-user.bat" file is located I ran the command "git pull"

I had a similar problem, my fix was to change my upscaler from 4x ultrasharp to latent and for some reason that seemed to work. Its important to note that SDXL 1.0 itself seemed to not have a problem with that upscaler so its specific to this checkpoint.

Textual Embeddings from 1.5 will not work with SDXL

directly in the preview image. click on one of them

No, it's a single-stage model. I had no problems rendering things from scratch.

@SeriouslyMike so, we do not need to use refiner model again also ?

@happynguyen91 no, you can use a simple high-res fix instead. I had good results generating in 768px on the shorter side and running a high-res fix in 1024px.

SDXL is meant to be used always with the refiner that comes with the model make sure you are using that.

@Artificial_Excellence asking because lykon said the refiner is mostly not needed with finetunes even here in this model description. And I heard it's not really a good idea to use the refiner on a finetuned SDXL model anyway. (tried it myself and no real improvement, at least not with the skin texture)

you can 'refine' it on 1.5 model 'img2img' :)

@brokolies then it changes a lot of the original picture details especially if I use a denoise of 0.4 or more. I already tried Lykon's workflow in comfyui with a 1.5 model used as the refiner (DS7 or Absolute reality), that's why I am asking all this because it didn't help much. All I am asking here is if SDXL will be much better in that regard of its own or not in the future and if the current limitation is caused by the lack of enough datasets.

@xperia256 Wait for the community to refine the model and it will eventually surpass 1.5, for now I'm using both together.

i guss don't support old lora

You are trying to use lora that are not trained with SDXL model

@acknowledgement Yeah thanks a lot

@banlg thanks!

try ComfyUI ,

Try ComfyUI and possibly pagefile memory. I have a 1070ti (8GB) and it runs flawlessly now and I can get generations with this with the upscale in around 4-5 minutes

Higher resolutions often make extra or unwanted subjects. If you are using higher resolutions try generating the initial image at a lower resolution (around 512x512) and upscale it after generation

@sproingo125743 don't generate 512x512, sdxl are trained on minimum 1024x1024 image.

@acknowledgement so how would I fix multiple subjects in the image?

@dgohio86 You can try adjusting aspect ratio so image has less width and more height thus fitting one person. This will cause it to bias towards a single person when you do 1girl, 1guy, or whatever else. Caveats being this obviously has some limitations in the images you create.

I doubt this is the problem, but high prompt weighting sometimes has a tendency to cause models to just add more instances of something. Might be worth trying to reduce the weight for the offending image element, and see if it makes a difference.

@acknowledgement Thanks, I've just got back into stable diffusion after a few month. Progress is crazy fast

You are not doing it wrong, the model does, it doesn't follow prompt at all :( It just take the part it can do and ignore everything else, it is now much worse than 1.5, sad

@dgohio86 You can add multiple people to your negative prompt, you can also up-weight your positive 1girl/1guy prompt. If anything, I have trouble getting multiple people to show up.

I'm pretty sure we are going to have to wait till CN models are trained on XL data.. dont think u can mix and match model version ie, 1.5 to 2.0 or XL..

@DreamingComputers You're totally correct. Thanks!

the model has more parameters and its name is xl, thus slower performance is expected.

I was having that issue yes. I was using --xformers, but apparently I also needed --medvram and/or --no-half-vae in the webui-user.bat file (assuming Automatic 1111). Like this:

set COMMANDLINE_ARGS= --xformers --no-half-vae --medvram

See this video from Monzon: https://youtu.be/gguLtMM4g_Q

Edit: for info, It took me about 4-5 minutes for 20 steps before adding the 2 parameters to the command line, using a RTX 3060 12GB, and after the change I now get about 1.2 to 1.4 iteration per second for 1024x1024 images.

What’s your gpu memory like when loading it? Generally I get reports like that when I max out vram.

I'm getting this when I try to load it:
Failed to create model quickly; will retry using slow method.

changing setting sd_model_checkpoint to dreamshaperXL10_alpha2Xl10.safetensors [0f1b80cfe8]: AssertionError

Traceback (most recent call last):

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\modules\shared.py", line 633, in set

self.data_labels[key].onchange()

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\modules\call_queue.py", line 14, in f

res = func(*args, **kwargs)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\webui.py", line 238, in <lambda>

shared.opts.onchange("sd_model_checkpoint", wrap_queued_call(lambda: modules.sd_models.reload_model_weights()), call=False)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\modules\sd_models.py", line 578, in reload_model_weights

load_model(checkpoint_info, already_loaded_state_dict=state_dict)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\modules\sd_models.py", line 504, in load_model

sd_model = instantiate_from_config(sd_config.model)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\stable-diffusion-stability-ai\ldm\util.py", line 89, in instantiate_from_config

return get_obj_from_str(config["target"])(**config.get("params", dict()))

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\generative-models\sgm\models\diffusion.py", line 65, in init

self._init_first_stage(first_stage_config)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\generative-models\sgm\models\diffusion.py", line 106, in initfirst_stage

model = instantiate_from_config(config).eval()

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\generative-models\sgm\util.py", line 175, in instantiate_from_config

return get_obj_from_str(config["target"])(**config.get("params", dict()))

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\generative-models\sgm\models\autoencoder.py", line 295, in init

self.encoder = Encoder(**ddconfig)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\generative-models\sgm\modules\diffusionmodules\model.py", line 553, in init

self.mid.attn_1 = make_attn(block_in, attn_type=attn_type)

File "C:\Work\StableDiffusionAutomatic1111- Working V2\stable-diffusion-webui\repositories\generative-models\sgm\modules\diffusionmodules\model.py", line 286, in make_attn

assert XFORMERS_IS_AVAILABLE, (

AssertionError: We do not support vanilla attention in 1.13.1+cu117 anymore, as it is too expensive. Please install xformers via e.g. 'pip install xformers==0.0.16'

There's more than a few people with this problem, I think they should take down the models, till they get a more compatible model and stop wasting our time !

I just went through this yesterday. Had the same error with my months-old A1111 trying to use SDXL. You need the latest A1111 to even use SDXL properly, and doing an "in-place" upgrade via "git pull" will leave broken dependencies. You pretty much need a fresh A1111 copy and just let it reinstall all of its dependencies, then you can copy back over whatever models you were using into the new copy.

these sdxl is definitely not easy to use. I had to do a lot of messing around and swapping the pytorch versions to get it to run. And when it did run, the girls it generate have horseface (maybe a little bit too realistic? lol). Either way he say it is in alpha, so hopefully the issues will fixed later.

@midnight_stories People who don't know how to update their automatic1111 properly. You need the latest version of automatic1111 to run sdxl model. It's not rocket science.

@MrOhyao  You just need to use --medvram if A1111 is slow for you on SDXL and it will be as fast as Comfy, tried myself on GTX 1070 8GB and A1111 was even 5 seconds faster than Comfy with --medvram (120 seconds on comfy vs 115 seconds on A1111 generating 1024x1024 image), without --medvram A1111 took ~12 mins to generate the same image. I hope A1111 will release an update that can auto detect cards with low VRAM and apply med or low vram argument automatically like Comfy does.

I think there must be something corrupted with my original install A1111 had to total get it off my system including user files. And just install the latest, I used the zipped install and that seamed to get it working, even the refiner addon is working now. Defiantly hard going for non coders like my self !

I got it working by doing a fresh install and adding to following to webui-user.bat
set COMMANDLINE_ARGS= --xformers --no-half-vae
(I'm using a 4090. For older cards you might not be able to use xformers)

@blugail don't use xformers it's slower. Most people don't use it now. If you are using Torch 2.0.1 you can use --opt-sdp-attention instead. Really improved the speed of sdxl on my 4090 so it's about the same speed as ComfyUI

@S4f3tyMarc Thanks! It's faster. I still think xformers is useful if you run into vram limitations, as it uses less vram than --opt-sdp-attention --medvram and is about the same speed.

1.5 embeddings can only be used with 1.5 models, etc. You see the embeddings in their sample images not because they can be used with SDXL, but because they are using 1.5 models as the upscalers for their SDXL generations. The generation settings in the samples are from the 1.5 refining step, the entire process used is in the "Workflow: (n) nodes" gen info.

SDXL was released officially a week ago - there is a lot of catch-up to do with SD 1.5. The base SDXL model and even trained checkpoints are going to struggle with nudity until it's "crowd trained" ie: from folks on civitai merging Loras and checkpoints together. It will get there, but it will take some time. The good news is that base SDXL knows what human anatomy is, it just isn't well-trained.

@Lykon good thing I haven't downloaded sdxl yet. just dreamshaperxl so far. surprisingly its generating 1024x1024 images in 30-40s on my rtx3060 with highres fix. will download the lora thanks. will have to wait at least 2 months before fully diving into sdxl

@MachineMinded unpopular opinion but don't it seem like sdxl is just sd 2.1 trained on 1024 pxls?

@vitokeorlini225 No, I spot checked some celebs, and it can generate them again.

@vitokeorlini225 By the way, SDXL and DreamshaperXL native resolution is 1024x1024. You should not use highres fix to generate 1024x1024 images. Using a lower resolution or highres fix will distort the image like as if you were generating 256x256 images in SD 1.5.

Wrong VAE

@thorgal what vae should we use?

@editorpabs this one should work stabilityai/sdxl-vae at main (huggingface.co)

or the one baked in the very checkpoint (set default in A1111)

I had the same problem, and the VAE was the root cause. Don't use VAE for SDXL.

@pawelzny "Don't use VAE for SDXL."

Wrong. Use that for SDXL models that don't have the vae baked :

https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main

There's a thread on reddit saying you need to remove any old lora's in your prompt.

Don't use LORA's or embeddings not trained on SDXL.

But that is a very interesting output, what If I would write a node that stretches the the size to be the same dimension?

How much minimum ram required to run this model?

did u try: "option+command+esc" lets you force close apps (in most cases)

64g memory minimal + 16g video mem

@gausssidorov928 I use it with 32g RAM and it works fine

I tried on my 16gb machine, it took like 10 minutes to load the model but then it worked fine.

Well, I'm gonna try it on my uh, lower end machine. It has 16 megs of ram and 4 megs of video ram. I'll let you know how it goes.
UPDATE: Ya says I need another 26 gigs of system ram.

I've been using it without a separate VAE just fine :)

@timothy860 do you using any anime or realistic models?

@timothy860  thanks. i think i use Anime VAE

You can get extremely good images, you just need to use LORAs, inpainting, img2img, and play around with different numbers a lot. See my LORA for examples (and these aren't even that good comapred to some of the newer LORAs I'm experimenting with)

@jacob42 Thanks for the comment, but I mean I was hoping XL to be better than sd1.5 (I mean, since it takes more vram and inference time, so it seems meaningless to use it if it is not getting better results than sd1.5). Not to mention the efforts you mentioned ("LORAs, inpainting, img2img, and play around with different numbers a lot") can also work on sd1.5, so I still don't see a good reason to use this instead of sd1.5

@seer_2022 All of the newer base SD models are highly censored. 1.5 is still the go to, for NSFW.

1.5 is generally the way to go for everything but maybe landscapes. XL is atrocious, objectively. Compare and contrast the quality of:
- 512x512 hi-res fixed to 1024x1024 1.5 output
- 1024x1024 (default) SDXL output

The community should go back to 1.5 en masse and stop working on this censored nothingburger.

I'm using none and I find the colors very natural, if you want them more saturated you can try the SDXL VAE

im getting the exact same error which doesnt allow me to load the checkpoint

You usually don't need upscaler for SDXL. But if for some reason you do want images larger than 1024x1024 -> you can use ESRGAN ultrasharp 4x upscaler.

Yep, I noticed that it works like complete shit in lower resolutions, I ran the same prompt at 1024x544 with the intention of GAN upscaling the result and in 1920x1080 straight, the latter looked infinitely better. Re-running the output through a refiner prompt set to the same model straightens out a couple of weirdnesses usually, but I always decode the first step output anyway.

@SeriouslyMike I need to research more, I didn't understand the "decode the first step output" is that something related to setting the seed? Anyway, XL is quite the upgrade, it will be interesting to explore more.

@Murdo no, in ComfyUI refining and highres fix can use the raw latent data instead of img2img, but I still decode the initial render to a human-viewable form so I can cancel the highres fix/refining stage if the base image is too distorted.

Look in the Console-Output, the file may be corrupted and you have to redownload it.

I had same issue and for me it was the webUI that wasn't updated to use SDXL. I changed one file to have Karras V2 sampler and because of that unfinished commit, WebUI didn't update for some time, had to stash the changes for git. After that I got to version 1.6 of gradio it worked fine

Lykon has decided to take a break from SD related activities: https://www.reddit.com/r/StableDiffusion/comments/15iq4l3/new_absolutereality_taking_a_break/

Also, this model is unfriendly with us horses too... how are we as horse folks be able to use it without having oposable thumbs in the model?

That seems exceedingly unlikely, given how early the model was uploaded. This was literally the first custom SDXL 1.0 model.

@steamrick If you use SD scripts, you will find that the checkpoint is merged from a LoRA. I simply unmerged it here for ease-of-use.

How much VRAM do you have? SDXL needs at least 8 GB to run at 1024x1024

Use a program called ISLC (Intelligent Standby List Cleaner) since Windows loves putting shit in Standby memory. I have 32GB and I still use this.

My settings are:

The list size is at least: 500MB

And

Free memory is lower than 18,000MB

ISLC polling Rate: 1000ms

Start

Start ISLC minimized and auto-start monitoring

Launch ISLC on user logon

Both checked