fp8 quantized Newbie-image for ComfyUI. And Gemma 3 4b.
All credit belongs to the original model author. License is the same as the original model.
Note: Images from bf16 and fp8 models are identical, like this. If image from fp8 model changed drastically, your ComfyUI somehow enabled fp16 mode. Newbie doesn't not support fp16, and you will get deformed image.
Versions:
Exp0.1 EP7:
Scaled fp8 + Mixed precision.
Exp0.1 base tcfp8:
Scaled fp8 + Mixed precision + Hardware (tensor core) fp8 support
Exp0.1 base:
Scaled fp8 + Mixed precision.
Gemma 3 4b:
Scaled fp8 + Mixed precision.
Note:
Where is Jina clip v2 ?
Jina clip v2 is very small (~1gb), seems not necessary.
Hardware (tensor core) fp8 support:
TLDR: The file contains calibration metadata. On supported GPU with hw fp8, ComfyUI will automatically do calculations in FP8, instead of dequantizing then BF16.
torch.compile is recommended, if you can get it up and running. Might be 80% faster.
More info about tcfp8: https://civarchive.com/models/2172944
If your GPU does not support fp8, then this version is the same as the normal one, because weights are the same.
Description
FAQ
Comments (16)
So does this work now? Or are we still waiting for comfyui to accept the pull.
Some example generations would be nice
for anyone wanting to use this you need no custom nodes just need to use this specific comfyui branch:
https://github.com/woct0rdho/ComfyUI/tree/98b25d43c24eea9e21062c38f7165ecabf94f621
I'll attach an image with workflow
it appears to not like the attached gemma, model does work.
No, the fp8 gemma 3 works.
There is a bug in the dual clip loader, if you use fp8 gemma 3 in it you will get noise image.
Single clip loader (Load CLIP) works fine.
@reakaakasky ah, I c
also, main stream comfyui already supported newbie, just jina clip not supported. but newbie model can be used without clip because gemma 3 is the main text encoder.
And personally I think jina clip might be unnecessary. Because gemma 3 is powerful enough, the model might just learn to ignore the weak embedding model. Just like Flux.1, Chroma decided to just drop the clip because it's almost useless.
Hi, I've fixed fp8 Gemma in NewBie dual clip. It's already merged into ComfyUI.
May I ask what method you used for creating this TensorCoreFP8Layout model?
https://github.com/comfyanonymous/ComfyUI/blob/master/QUANTIZATION.md#calibration-for-activation-quantization
Is Gemma3 4b the actual gemma text encoder as fp8?
?? what else could it be?
@reakaakasky idk, the file name is just confusing it says the name of the newbie model and gemma so i thought it might be some type of merge?
file name is "gemma_3_4b_it_fp8_scaled_mixed_precision.safetensors"
you probably downloaded other model.
@reakaakasky Yes, that might be the file name from your end, but civitai automatically changes the name (most of the time) to the name of the model page and the version
this is the file name newbieImageFp8_gemma34b.safetensors i get when i download the gemma3 4b text encoder from you
@Cass_ I see... they squeezed the model name + version name together...