IMPORTANT
INT5 model requires my custom node - https://github.com/bakapotatolord/ComfyUI-PotatoForge-INTx
Clone it in ComfyUI/custom_nodes folder or it won't work. INT8 Mixed model should work as is, no need for custom node.
Quantization of Krea 2 Turbo primarily meant for low VRAM/RAM usage, expect degradation in quality compared to INT8 Convrot only quants or BF16
INT5 is highly experimental, my goal was to find a middle ground between INT4 and my very own INT6. All block layers are in int5 convrot, it's not mixed.
Once again, used a simple T2I workflow, no second stage or upscaling done as I'm too lazy for that.
Tested on:
NVIDIA GTX 1660 Super 6 GB
32 GB System RAM
Test settings:
8 steps
CFG 1
Euler / Simple
696 × 1048 resolution (0.7 on resolution selector)
Approximately 9 s/it on a GTX 1660 Super
Quantized using my toolkit
Description
FAQ
Comments (10)
okay this is quite interesting
Glad you think it is
The image changes more than even compared to w4a8, but the speed is 1.5 times faster, and the picture isn’t worse — it’s just slighty different. 3.5s per iteration on a 3050 6GB is awesome.
P.S.: sorry mistype 3.5s not 1.5
Yep, the images will change wildly across the same model but different quants even with the same seed.
But wow, I'm surprised, 1.5 s/it on 3050?! Damn, I get like 9 s/it on my GTX 1660S and they aren't too far off from each other in performance. Guess those tensor cores are contributing a ton. Which resolution are you running it at?
@BakaPotatoLord Sorry mistype 3.5s/it not 1.5s/it at standart 1M (1024x1024 or different proportion but 1 megapizel total) with default pytorch attention, fisrt iteration 8-12 s and i have pcie 3.0 (cpu 4600g) and models on sata SSD but 32GB of RAM (--cache-none disabled because sometimes there are problems with large models when switching). Total time with 8 steps 35-45s with TE and VAE, loras add ~0.7-0.8s/it for each with you model.
@wingbrother Ah yeah, I forgot the 30xx and others have sage attention and so on, makes sense.
@BakaPotatoLord default pytorch attention, torch 2.13+cu130 python 3.12 ( sage I didn't find the precompiled wheels with sage-attention, and I didn't bother to assemble them myself).
@wingbrother My bad, guess I misread that. Man, I wish I had waited and gotten a 3050 instead lol. Just making do with my GTX 1660S for now.
Interesting. Have you run any qualitative tests on INT5/INT6 versus INT8 Convrot? In my experience, INT4 loses a little too much in the pursuit of speed, at least with Krea.
There is quality loss for sure, the reason I am having these INT5/6 and what not is to have steps in between INT4 and INT8 so there's more possibilities on how layers can be allocated.
So really sensitive layers can be bumped from INT4 to INT5/6 and less sensitive from INT8 to INT5/6. I'm working on mathematical ways to allocates these layers depending on numerical errors.
But really, I have observed no speedup between the various INT methods because ultimately Comfy uses INT8 GEMM in the end. It's more of a storage thing more than anything, plus reducing peak RAM usage.
This INT5 only model is more of a proof of concept that it works.