This is merge model for:
1. 100% stable-diffusion-xl-base-1.0 and 100% stable-diffusion-xl-refine-1.0
https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0
https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0
2. sdxl-vae-fp16-fix
https://huggingface.co/madebyollin/sdxl-vae-fp16-fix
you can use this directly or finetune.
same license on stable-diffusion-xl-base-1.0
same vae license on sdxl-vae-fp16-fix
SDXL-VAE-FP16-Fix
SDXL-VAE-FP16-Fix is the SDXL VAE, but modified to run in fp16 precision without generating NaNs.
VAEDecoding in float32 / bfloat16 precisionDecoding in float16 precisionSDXL-VAE✅
⚠️
SDXL-VAE-FP16-Fix✅
✅
Details
SDXL-VAE generates NaNs in fp16 because the internal activation values are too big:
SDXL-VAE-FP16-Fix was created by finetuning the SDXL-VAE to:
1. keep the final output the same, but
2. make the internal activation values smaller, by
3. scaling down weights and biases within the network
There are slight discrepancies between the output of SDXL-VAE-FP16-Fix and SDXL-VAE, but the decoded images should be close enough for most purposes.
Benchmark from here:by Kubuxu
https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7
Evaluation on COCO val-2017, 256x256, RandomCrop with padding
Metrics:
LPIPS: https://github.com/richzhang/PerceptualSimilarity/ (lower better) and structural similarity index measure via skimage.metrics (higher better)
Metrics given as: mean [79% credibility interval]
Description
Improved decoder weights
* Further-reduced risk of NaNs
* Further-reduced discrepancies with original SDXL-VAE (0.9) decoder
Encoder weights are unchanged.
FAQ
Comments (7)
Can you confirm that the V2 corresponds to this commit: 6d10734?
yes,same
目前怎么看SDXL的效果都丑到爆，不过毕竟是开始，静观其发展~
不好说，毕竟这个还和模型作者的审美有关
毕竟只是基础模型，等各种微调模型吧
我看前几天Stability更新了水印权重。
并且貌似可以直接禁用水印系统？不知道是不是一个东西
稳定的扩散XL (huggingface.co)
我们建议为稳定扩散XL生成的图像添加不可见水印，这有助于识别图像是否为下游应用程序进行机器合成。为此，请安装 不可见水印库通过以下方式：
复制
pip install invisible-watermark>=0.2.0
如果安装了库，则默认情况下将使用水印器。invisible-watermark
如果您有其他安全生成或部署图像的规定，您可以按如下方式禁用水印器：
复制
pipe = StableDiffusionXLPipeline.from_pretrained(..., add_watermarker=False)
只改了vae部分。