Please re-download int8 version. I have uploaded much faster, proper version.
Versions
int8: recommended. Fast, accurate, compatible with almost any GPU.
mxfp8: added for comparison. In theory (and according to nVidia PR) should be more accurate than int8, but in practice I was not able to spot any definitive advantages. A bit slower than int8, but still faster than original bf16. Compatible only with RTX 50xx series (Blackwell).
Performance on my setup
original bf16 (baseline): 2.20 it/s +0%
int8: 3.23 it/s +46%
int8 + torch compile (comfy core): 3.59 it/s +63%
mxfp8: 2.58 it/s +17%
This is high quality int8 quantized version of WAI-ANIMA model. It retains ~90% of original model quality, but uses about 50% less VRAM and also runs faster on almost any nVidia GPU (AMD not tested). Nice trade-off, especially for low-end GPUs.
Can be used as a drop-in replacement for original model in latest ComfyUI, no custom nodes required. If you have troubles running the model make sure that you updated both ComfyUI itself and its dependencies (e.g.pip install -U -r requirements.txt on manual linux install).
Converted to int8 / mxfp8 using convert_to_quant script.
Description
int8, ConvRot with dynamic group size, rowwise, SVD
Potentially better quality than ConvRot with fixed group size, but reported to be slower by some users.
FAQ
Comments (17)
A proper int8 convrot quant will retain nearly 100% quality. If you're seeing a 10% quality drop that's significant and points to a bad process.
Well, the quality is maximum supported by ComfyUI: convrot, row-wise, learned rounding. Imho it holds very well, but there are still small differences at the same seed, check comparison images. You can't expect exactly same result with less precision.
Post comparison images!!!!
Is this better than using https://github.com/Haoming02/sd-forge-compressor to convert it to fp8-scaled? For my use case only the VRAM and quality matter, my GPU isn't new enough to support INT8 acceleration.
I quickly checked source code of that extension and it appears that it performs only basic fp8 scaling. So this int8 version should be better, because it also applies convrot, learned rounding etc (different methods to improve quant precision). Best way is to compare side by side to see how much real impact it has; fine details like text rendering, hands etc
sd-forge-compressor does use the comfy_kitchen package to do the fp8_scaled / int8_convrot conversion~
hi, thanks for the int8 version! idk if this is a problem with my environment or what... because i realized it's actually slower than the native fp16/bf16 model.
@jancok Please re-download int8 version. I have uploaded much faster, proper version.
Is this compatible with Forge Neo?
Yes, but only in the recent GitHub updates. If you download it directly from the releases, it should be available in 2.27.
just get comfyui. there's zero reason not to these days. you're missing a huge performance boost with how badly optimized forge neo is. i shaved off thirty whole seconds of gen time jumping to comfy.
Nice, maybe I'll be able to use Anima in my old laptop after all.
Please re-download int8 version. I have uploaded much faster, proper version.
Could you please post your arguments for the script to get the new (more accurate) int8? Thank you in advance.
@everylight Here it is
ctq --anima --input ~/apps/ComfyUI/models/diffusion_models/anima/merges/waiANIMA_v10Base10.safetensors --manual_seed 42 --comfy_quant --save-quant-metadata --int8 --convrot --scaling_mode row --verbose VERBOSE --full_matrix --scale_optimization dualround --calib_samples 6144
@somedoby Thank you greatly! I have a number of private Anima merges that I wanted to apply this. Cheers and please keep up the good work!
@everylight You are welcome. I am just experimenting with this stuff and glad to share the knowledge.