I have 12GB VRAM too (4070) and can run Fp8 models just fine in ComfyUI, Fooocus and Invoke. The only problem is it takes a long time to load when fired up for the first time but after that its smooth sailing.

I can make you f8 scaled, if you want, not sure if it would run in Forge, but it will in ComfyUI.

As for NF4, Im working on creating something like NF4v3, but it might take ages, so..

Anyway I think you can make NF4v2 relatively easy in Forge.

Cyberdelia I'll test it and let you know, forge is what I use, I know I tried fp8 before and had issues, I believe the issue isnt running it, its that the generation time is awful and it ruins the experience, like 2minutes for 1 image or something like that, whereas I get like 30-40s an image with nf4v2 and GGUF 4, I'm going off memory I havent messed with flux in a good few months, but I'll get back to you on more precise comparisons between running fp8 on my rig and gguf/nf4v2 at the very least I can say the experience so far hasn't been ideal

Back after testing, firstly I gotta say, incredible work cyber! undoubtedly the best flux model I've ever worked with, prompting is very intuitive, I was using a mix of booru tag based prompting and natural language and the model got it right 80-90% of the time, the NSFW in particular (which is mostly all I tested for now dont judge me lol) is outstanding, I dont think I have ever used a flux model (even if it is labeled NSFW) that didnt need a little help from loras to get anatomy right, but cyberreal accomplishes that like a boss, 10/10 as far as appearance, details, and prompt comprehension. Sadly for me (and it very well could be something Ive done wrong with flux idk) each single image took 3:39s to generate at 30 steps, 1cfg, it's a testament to this model that I still cranked out like 20 images at that speed, cause it was just too good, it was like trying a dab for the first time when youve only ever smoked weed, very exciting, it definitely marks a new age of Flux models imo, next level shit, but 3:39s really does dampen the experience, so while fp8 worked and I love the model, I'm still really hoping a GGUF or nf4 version pops up in the near future, cause those are the only models I can get a reasonable speed with for whatever reason.

Mescalamba Sounds ambitious I wish you luck, I would love to experience something like an nf4v3, thus far for speed, best Ive experienced is nf4v2 hyper, I tend to use that more than anything else cause its only like 30sec an image which my adhd can handle just fine. Thanks for the offer for scaled, I never even heard of fp8 scaled Ill have to inform myself on it better, it would have to work in forge cause Im really not a fan of comfy, that UI setup irks me idk why, it just does

Urameshi fp8 scaled is basically another way to quantize and store models, think of it as a lot improved fp8. Its not like Q8, but should be something like Q6. Also in case someone has 40xx or newer GPU, it runs very fast.

Mescalamba well it sounds worth trying at least, as it stands the regular fp8 I just cant really get in there and have fun with cause generations are just so long

Cyberdelia In general, Schnell is easier to train, de-distilled Schnell is actually probably even easier. But its also rather complex and requires a lot of final tweaks to get something out of it.

Not sure how hard to train is Chroma, but people already did some LoRAs and it didnt seem that hard. Plus it should be fairly easy to train further (it kinda needs it anyway still). I find it fairly difficult to prompt, so maybe its not that good choice for generic public. But it can make porn so..

Cyberdelia Ok Cool I'll give it a looksee. ;)

Cyberdelia I would say that text focused one is still best. Mostly cause it doesnt give just "sharp text", but sharp everything. Also works on everything from SD15 till .. well even future stuff that uses CLIP-L.

https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main?show_file_info=ViT-L-14-TEXT-detail-improved-hiT-GmP-HF.safetensors

Cyberdelia Yea I did, yesterday. Still believe that one I linked is best. Altho I dont do much FLUX, so it might be different for that.

In case of FLUX, CLIP doesnt have that much impact, only to some degree on scene composition and I would say final details (it basically ties image for FLUX together). Its not really needed as FLUX can be used even without CLIP, but since it was trained with it, its better used with that.

T5-XXL is more important, like GNER or Alpaca tuned T5-XXLs. Altho FLAN is probably best for most.

Cyberdelia Forgot I could link those too, but Im sure you can find them on HF anyway.

Wouldnt mind if you talked/answered in chat tho, but dont want to bother you too much. Also can try to help with my limited options, if needed/wanted.

Power to the worthy ones! 💪

Cyberdelia i use a1111 is it not supported? thank you

You just need 24 Gb VRAM to fit model with clip models unless you wanna wait 10 minutes while generation done from shared memory.

But this fits my 12 gb 3060 no issues.

Around 3s/it on my 3060 Ti 8GB with less than 2 mins for a 1024x1024 in Forge; FP8 and quantized Flux models should almost always work on 8GB cards and up

Why not stop complaining and appreciate the incredible work that Cyber has put in here, I'm running it on a H100 SVX, this model is a revelation! Mind = Blown!

ronikush H100:))))))

Cyberdelia Yes, of course its the minimum.

ronikush Complaining? I was just asking and gave reason why I was asking. About appreciating work. Yeah appreciated. I was actually going to suggest to Cyberdelia to check out the model my team works on called Chroma. lodestones/Chroma · Hugging Face

S1LV3RC01N Dunno why they were so mad lol. I had the same question. Appreciate your work on Chroma, by the way. Solid model itself, although I still have much to learn about using it to its full potential.

erickj92790 Join the discord and check out the chroma-gens channel as well as the workflows-tips-n-tricks threads. Might help a bit.

Cyberdelia Thank you very much, thank you very much. We love you

Great shout! THanks that fixed Lora issue for me!

I had similar issue and using this clip model it improved things a lot, but nothing is perfect.
https://civitai.com/models/1805024/hidream-clip-distilled-for-flux-sdxl-sd

Also, str of 1.4 may be needed,

increased CFG may be needed, 3.5 misses on some but at 6 it works.

Test with 1 character lora, 1024x1024, Euler simple, 20 steps, 3.5 CFG and prompt for a basic close up face portrait of the character to see if you can find them and then go from there.