Quantization of Krea 2 Turbo primarily meant for low VRAM/RAM usage, expect degradation in quality compared to INT8 Convrot only quants or BF16, saves ~64.3% in storage relative to BF16.
Layer count
139 layers in convrot_w4a4_mse (it's just convrot_w4a4 but with additional logic for selecting better scales, reduces error quite a lot numerically but I am unsure about visually)
78 layers in int8_convrot
1 layer in convrot_w4a4
Once again, used a simple T2I workflow, no second stage or upscaling done as I'm too lazy for that.
Tested on:
NVIDIA GTX 1660 Super 6 GB
32 GB System RAM
Test settings:
8 steps
CFG 1
Euler / Simple
696 × 1048 resolution (0.7 on resolution selector)
Approximately 9 s/it on a GTX 1660 Super
Quantized using my toolkit
Description
Comments (5)
okay this is quite interesting
Glad you think it is
The image changes more than even compared to w4a8, but the speed is 1.5 times faster, and the picture isn’t worse — it’s just slighty different. 3.5s per iteration on a 3050 6GB is awesome.
P.S.: sorry mistype 3.5s not 1.5
Yep, the images will change wildly across the same model but different quants even with the same seed.
But wow, I'm surprised, 1.5 s/it on 3050?! Damn, I get like 9 s/it on my GTX 1660S and they aren't too far off from each other in performance. Guess those tensor cores are contributing a ton. Which resolution are you running it at?
@BakaPotatoLord Sorry mistype 3.5s/it not 1.5s/it at standart 1M (1024x1024 or different proportion but 1 megapizel total) with default pytorch attention, fisrt iteration 8-12 s and i have pcie 3.0 (cpu 4600g) and models on sata SSD but 32GB of RAM (--cache-none disabled because sometimes there are problems with large models when switching). Total time with 8 steps 35-45s with TE and VAE, loras add ~0.7-0.8s/it for each with you model.