Warning: Although these quants work perfectly with ComfyUI - I couldn't get them to work with Forge UI yet. Let me know if this changes. The original non-k quants can be found HERE which are verified working with Forge UI.
[Note: Unzip the download to get the GGUF. Civit doesn't support it natively, hence this workaround]
GGUF version of FluxUnchained by socalguitarist . Credit goes to him for tuning this model. More specifically these are the K (_M) quants for this model. K quants usually lead to better quality for the same model size. Also, the inference can be slightly faster. The non-K quants can be found here.
It can be used in ComfyUI with this custom node. But I couldn't get these to work with Forge UI.
See https://github.com/lllyasviel/stable-diffusion-webui-forge/discussions/1050 to learn more about Forge UI GGUF support and also where to download the VAE, clip_l and t5xxl models.
Which model should I download?
[Current situation: Using the updated Comfy UI (GGUF node) I can run all of these quants on my 11GB 1080ti.]
Download the one that fits in your VRAM. The additional inference cost is quite small if the model fits in the GPU. Size order is Q2 < Q3 < Q4 < Q5 < Q6. However, I wouldn't recommend using Q2 and Q3 unless you absolutely have to, because they tend to mess up text and finer details.
All the license terms associated with Flux.1 Dev apply.
Description
FAQ
Comments (22)
Thanks for Q3 and Q2 version for low vRAM users. Will let you know if they work in my forge setup.
Still not working with Forge. Thanks anyway though.
@JoanFrances Try comfy for now.
What is the difference of being K_M?
Thx.
marginally bigger than K_S but should be better quality
I get this error when i run this model in ForgeUI, RuntimeError: mat1 and mat2 shapes cannot be multiplied (4032x64 and 256x768)
mat1 and mat2 shapes cannot be multiplied (4032x64 and 256x768)
I used your node setup and comfyUI as well, same error. All other models work.
@Ceylon_Ai Try updating the GGUF node and also Comfy
@Ceylon_Ai Did you find a fix?
same error on my sd forge but works fine on comfyui
i get this error on both forge and comfyui which has latest updates for nodes too
you will upload a km q8 one ? i wait for it , or i just pick the q6 one ?
q8 does not have a k quant. If you wanna save some memory go for Q6_K . But Q8_0 will probably give you slightly better quality.
@nakif0968 thanks , i'm waiting for well improved gguf q8 unet .
@amazingbeauty I have already uploaded the q8 quant here https://civitai.com/models/662112?modelVersionId=748232
@nakif0968 i forgot to add that i wait it to be a merge flux s , that capable for 4 steps only
@amazingbeauty You mean this one ? https://civitai.com/models/666145?modelVersionId=748390
@nakif0968 yes maybe , would say might be but if it improved at nsfw without losing any other capabilities
Are the preview photos actually generated by each version of the model? They look identical.
No, all of them are generated by Q4_0.
Any plans to release the SVDQuants models for Nunchaku. It produces results much faster with lower VRAM requirement.
Sorry, I don't have any plans for now. Maybe you can also request the original author (SocalGuitarist).