This is merge model for:
1. 100% stable-diffusion-xl-base-1.0 and 100% stable-diffusion-xl-refine-1.0
https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0
https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0
2. sdxl-vae-fp16-fix
https://huggingface.co/madebyollin/sdxl-vae-fp16-fix
you can use this directly or finetune.
same license on stable-diffusion-xl-base-1.0
same vae license on sdxl-vae-fp16-fix
SDXL-VAE-FP16-Fix
SDXL-VAE-FP16-Fix is the SDXL VAE, but modified to run in fp16 precision without generating NaNs.
VAEDecoding in float32 / bfloat16 precisionDecoding in float16 precisionSDXL-VAE✅
⚠️
SDXL-VAE-FP16-Fix✅
✅
Details
SDXL-VAE generates NaNs in fp16 because the internal activation values are too big:
SDXL-VAE-FP16-Fix was created by finetuning the SDXL-VAE to:
1. keep the final output the same, but
2. make the internal activation values smaller, by
3. scaling down weights and biases within the network
There are slight discrepancies between the output of SDXL-VAE-FP16-Fix and SDXL-VAE, but the decoded images should be close enough for most purposes.
Benchmark from here:by Kubuxu
https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7
Evaluation on COCO val-2017, 256x256, RandomCrop with padding
Metrics:
LPIPS: https://github.com/richzhang/PerceptualSimilarity/ (lower better) and structural similarity index measure via skimage.metrics (higher better)
Metrics given as: mean [79% credibility interval]
Description
fixedvae_fp16
FAQ
Comments (18)
青龙大佬，那用这个修复的模型，跑图的时候还需要外挂原生1.0的VAE吗？
Thanks for putting the time in to make this and release it to us! 🙏🏻💙
So basically you baked in the 16fp VAE into the full checkpoint?
I don't think so, bc isn't the full XL base 13gb?
@bleuropa329 the new SDXL1.0 base is 6gb,this is the version remove the watermark
Majin @buuuuuuuuu On a1111 toolkit , it says the vae is broken and the original checkpoint had 6gb of junk in it (probably because of the original vae being fubar) this version should have fixed that. Mad week at work so not had time to test it , hopefully this weekend
yeah,but fp16_vae had fixed other thing,such as upsample problem in finetune.
@GG_System 6GB of junk in a 13.5GB model? What the heck? Are devs at Stability AI ok?
@Nekos4Lyfe it was the 6.3gb model that had the broken vae in it causing the junk data.
"no watermark" does this mean this removes the watermark from SDXL?
i wish i can used sdxl, always i got an error,