This is merge model for:1. 100% stable-diffusion-xl-base-1.0 and 100% stable-diffusion-xl-refine-1.0<a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0">https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0</a><a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0">https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0</a>2. sdxl-vae-fp16-fix<a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix">https://huggingface.co/madebyollin/sdxl-vae-fp16-fix</a>you can use this directly or finetune.same license on <a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0">stable-diffusion-xl-base-1.0</a>same vae license on <a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix">sdxl-vae-fp16-fix</a><h1 id="heading-7">SDXL-VAE-FP16-Fix</h1>SDXL-VAE-FP16-Fix is the <a target="_blank" rel="ugc" href="https://huggingface.co/stabilityai/sdxl-vae">SDXL VAE</a>, but modified to run in fp16 precision without generating NaNs.VAEDecoding in <code>float32</code> / <code>bfloat16</code> precisionDecoding in <code>float16</code> precisionSDXL-VAE✅<a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/blob/main/images/orig-fp32.png"><img src="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/images/orig-fp32.png" alt /></a>⚠️<a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/blob/main/images/orig-fp16.png"><img src="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/images/orig-fp16.png" alt /></a>SDXL-VAE-FP16-Fix✅<a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/blob/main/images/fix-fp32.png"><img src="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/images/fix-fp32.png" alt /></a>✅<a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/blob/main/images/fix-fp16.png"><img src="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/images/fix-fp16.png" alt /></a><h2 id="heading-8">Details</h2>SDXL-VAE generates NaNs in fp16 because the internal activation values are too big:<img src="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/resolve/main/images/activation-magnitudes.jpg" />SDXL-VAE-FP16-Fix was created by finetuning the SDXL-VAE to:1. keep the final output the same, but2. make the internal activation values smaller, by3. scaling down weights and biases within the networkThere are slight discrepancies between the output of SDXL-VAE-FP16-Fix and SDXL-VAE, but the decoded images should be close enough for most purposes.<h1 id="heading-2">Benchmark from <a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7">here</a>:by <a target="_blank" rel="ugc" href="https://huggingface.co/Kubuxu">Kubuxu</a></h1><a target="_blank" rel="ugc" href="https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7">https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/discussions/7</a>Evaluation on COCO val-2017, 256x256, RandomCrop with padding Metrics: LPIPS: <a target="_blank" rel="ugc" href="https://github.com/richzhang/PerceptualSimilarity/">https://github.com/richzhang/PerceptualSimilarity/</a> (lower better) and structural similarity index measure via skimage.metrics (higher better) Metrics given as: mean [79% credibility interval]<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/16a4a92c-4764-4baf-8c30-b52f7158ff54/width=525/16a4a92c-4764-4baf-8c30-b52f7158ff54.jpeg" />

Files with SHA256: 9296259af7ae0baa53c724757e34e0a793a2825f842b3c5cf535304b4966e2e9