NF4 Version of FluxUnchained (by SocialGuitarist)
All credit goes to SocialGuitarist for tuning this model,
and lllyasvielfor for providing the conversion script.
I converted this model since my 3080 10g has trouble running FP8 or FP16, and Q4 and Q5 are considerably slower than NF4 while giving similar result.
Not sure how NF4 can be used in ComfyUI, but I have been getting good results using it in Forge with good inference speed.
There are 2 versions, one with text encoder baked in and one with only the unet (to save disk space).
To use the unet, select/add the following encoders in the Text Encoder Setting of Forge.
All the license terms associated with Flux.1 Dev apply.
Description
v1.1 - Based on the v1.1 version of FluxUnchained (t5_16xfp8_e4m3fn)
This version contains the Unet only without Text Encoder baked in. To use it, add the following encoders in the Text Encoder.
clip_l.safetensors
t5xxl_fp16.safetensors or t5xxl_fp8_e4m3fn.safetensors
ae.safetensors
FAQ
Comments (46)
Hi, do we need to use the "t5xxl_fp8_e4m3fn.safetensors" CLIP model? Or is it all baked into this model?
Everything is baked in the model.
I was sure Unet work for NF4, but if it does, I will probably upload it alongisde to reduce the size for people who already have the encoder.
@sissyslave907507 Thanks! What sampler and scheduler do you use?
@brsci9000 Sampler: Simple. Schedular: Simple
I tried to experiment with other samplers, but for a lot of them I was getting a black or noisy/blurry image so I stick with what llyasviel suggested.
Is it just displaying a different model size because of the way civitai counts bits and bytes compared to the OS? It says your NF4 model is 15,8GB while my local original Flux.1-Dev_BNB_NF4_v2 is only 11,7gb.
@lost_moon I am not sure what is the reason for the file size increase, but the displayed sizes are correct.
@sissyslave907507 I'm asking because I also have a RTX 3080 10GB edition but only 32GB RAM and I already sometimes hit a bottle neck in Forge where all my VRAM and all my RAM is full or very near full. I'm not sure if a 16GB model would even work. Do you happen to have 64GB system RAM?
@lost_moon I also got 32gb DDR4 ram with 3080 10g. It take around 30-45 seconds for an 896x1152 image. Running async.
The only time I hit a bottleneck is when using lora, and even that has been improved in the current version of forge. Btw maybe reduce your VRAM, I allocate it around 8.8gb to ensure it doesn't overflow or result in OOM.
sampler and sheduler
with at least 12 steps better 20 only Euler (not a) with:
Simple, Normal, SGM Uniform or DDIM -> last one quite ok
at least 50 better 100 steps with:
also other shedulers (and you get realy good pics)
more than 100 "restarts" seems ok, but for my hardware 10min for one image is too long ;)
hope it helps
When I try the model in Forge, I get the error: AssertionError: You do not have CLIP state dict!
What am I doing wrong?
@lazyengineer050693 You need to add the clip_l.safetensors text encoder
what's ae.safetensors? and do clip and t5xxl affect performance in forge UI?
Not sure, but all 3 are needed as per Forge dev for generating images with GGUF or if you are using unet.
You can also use the baked version which has them baked in, if you don't have the encoder (links are there in description)
@sissyslave907507 ok, thank you, you wouldn't have ae.safetensor I tried downloading it but it says it's not downloadable
ae.safetensors is the flux VAE
@stygianwizard42 You would need a huggingface account to download.
@sissyslave907507 still no luck it says its "file wasn't available on site" and wouldn't download, it also claims to be a txt file
@stygianwizard42 The link will work if you have a huggingface account. The error happens because you're required to accept some type of rule in that website, to be able to download it
For some reason this model takes longer than normal flux.dev fp8. Other NF4 Flux.dev models are much faster. I think this one is for some reason too large?
Just tested with the following settings (on the latest commit of Forge without LORA)
GPU Weight: 8945mb, Swap Method: Async, Swap Location: CPU
fluxunchainedArtfulNSFW-bnb-nf4.safetensors Time taken: 48.5 sec.
flux1-dev-bnb-nf4-v2.safetensors Time taken: 43.1 sec.
For me, the generation times are similar though the model likely takes more time to load initially. But for subsequent generations, the time are similar in nature.
Though I do find the prompt adherence of this model to be noticeably worse than Flux Dev NF4.
@sissyslave907507 I do get more SD3 looking deformities with this as well.
Can I ask some questions. How big was the dataset and how did yo tag it?
This is just a conversion, you should ask the original creator
https://civitai.com/models/645943?modelVersionId=722828
How long does it take to create an image with your 3080? I have a 3070.
Somewhere around 30-45 seconds for 896x1152 image.
1 min 12 seconds in 3070ti. what about you A1988?
@AlienPleasures I haven't tried it yet, but since it's too slow for my hardware I'm not going to download it.
It is not supporting Lora in SwarmUI.
I can confirm that the model is working in ComfyUI using the Custom CheckpointLoaderNF4 Node.
For installation and setup, you can follow the instructions of the original nf4 flux release.
https://civitai.com/models/638572/nf4-flux1
it was a pain to install bitsandbytes :) but it all works now
Are you using the one with the text encoder baked in, or the other? I've been trying the other and it keeps throwing a "Could not detect model type", something which doesn't happen with the standard NF4 model.
With this model you trade off generation speed for boobies.
That's a sacrifice I'm willing to make xD
For the prompts, how is it you're making use of compel's weighting in forge? Did you implement it yourself?
Sorry but I simply converted the existing model, don't even have an idea what "compel's weighting" means
where to put the encoders for forge?
webui\models\VAE
@sissyslave907507 really? My version of forge has a folder called Text_Encoders.... I may have made that myself when I originally downloaded them and wasn't sure where to put them.. but idk. I'll have to copy all those and put them in the VAE folder too now.
hey how did you convert it to nf4??
is there a tutorial for it??
HI, I'm getting a "ValueError: Failed to recognize model type!" in Forge on Collab. I have all the clip/vae/ae installed. I've looked elsewhere for help with this error but nothing works. Any ideas?
Only unet model working on Forge.
UPD.
Only 1 generation possible on my GTX3080 16Gb, after that it crushing.
"Only unet model working on Forge."
I'm on Forge. Using nf4 model. Working perfectly fine. Don't state things you don't know.
It's not because your system cannot handle the head that suddenly your reality becomes a truth for the rest of the world.
nf4 works fine on forge - no unet needed
FREE HEM TIDDIES!
Can you tell me which is better to use in Forge?
Sampling method -- ?
Schedule type --?
thank you =)