Diffucore vs ComfyUI on an RTX 5090: Anima Speed Test
Update (October 11): this article has been rewritten. It replaces the original version I published on October 10, whose ComfyUI numbers were wrong. A friend on Discord told me my ComfyUI --fast numbers looked weird, and they were right. That pod had PyTorch built for CUDA 12.8, and ComfyUI turns off comfy-kitchen's fast CUDA kernels on anything below cu130. It even prints a warning about it, which I missed. So every ComfyUI number in the first version of this article was ComfyUI running without its fused kernels. My bad, and thanks for catching it.
So I reran everything properly. And since ComfyUI's INT8 models came out ahead in the rerun, I spent the day adding two things to Diffucore: INT8 models, and int8 attn for RTX 30, 40 and 50 series. Here's the whole thing again.
The setup
Model: anima-base-v1.0, qwen_3_06b_base text encoder, qwen_image_vae
30 steps, Euler, simple scheduler (on Diffucore that's "flow" with shift 3, same schedule), CFG 4.5 on every step
1024x1024, batch 1
One warmup image, then 3 timed images with different seeds. The numbers below are the median.
The time is the whole image: from hitting generate until the PNG is saved, VAE decode included.
RTX 5090 on Runpod Secure Cloud. ComfyUI ran on PyTorch 2.12 with CUDA 13.0, and I checked its log every time to make sure comfy-kitchen's CUDA backend was on.
I did two rounds, each on its own pod. Pods aren't identical (the one from the first version of this article was a lot slower for both apps), so only compare numbers inside the same round.
Round 1: ComfyUI with comfy-kitchen working
ComfyUI default (bf16): 6.62 s (5.16 it/s)
ComfyUI
--fast: 6.02 s (5.57 it/s)ComfyUI
--fast --fp16-unet --use-ck-attention: 5.48 s (6.34 it/s)ComfyUI fp8_e4m3fn_fast: 6.43 s (5.05 it/s)
ComfyUI INT8 (anima-base-v1.0 ConvRot INT8 from Hugging Face) +
--fast --use-ck-attention: 3.94 s (8.49 it/s)Diffucore, no chips: 7.55 s (4.18 it/s)
Diffucore, fp16 acc + fused glue + fp16 vae: 5.13 s (6.16 it/s)
With its kernels on, ComfyUI got a lot faster, and --fast now helps instead of making things slower. Diffucore's chips were still a bit faster than ComfyUI's best fp16 setup (5.13 vs 5.48 s), but ComfyUI's INT8 model was about 1.3x faster than anything Diffucore could do. That's what I went to fix.
What's new in Diffucore
INT8 models. Diffucore now loads ComfyUI's INT8 checkpoints (int8_tensorwise, with or without ConvRot) straight from the DiT dropdown. There's also a new int8 chip that turns a normal Anima model into INT8 when you load it, so you don't need a special file. Both the weights and the activations run in INT8, on the GPU's int8 tensor cores. LoRAs work on INT8 models too.
int8 attn now runs on RTX 30, 40 and 50 series. Before, it was an RTX 20 series kernel only. The new kernel does the same trick on newer cards: Q times K on INT8 tensor cores, P times V with fp16 accumulation. On the 5090 a single attention call is about 1.85x faster than PyTorch's attention, with the same accuracy as the RTX 20 series version.
Round 2: Diffucore with INT8 and int8 attn
Everything here ran on one pod, back to back:
ComfyUI fp16 +
--fast --fp16-unet --use-ck-attention: 5.33 s (6.35 it/s)ComfyUI INT8 +
--fast --use-ck-attention: 3.91 s (8.53 it/s)Diffucore int8 + fp16 acc + fused glue + fp16 vae: 4.27 s (7.77 it/s)
Diffucore int8 + int8 attn + fp16 acc + fused glue + fp16 vae: 3.63 s (9.32 it/s)
So with both new chips, Diffucore makes a 1024x1024 Anima image in 3.63 s vs ComfyUI's best 3.91 s, about 1.08x faster, or 9.32 vs 8.53 it/s. I ran the Diffucore config again at the end to check the GPU hadn't drifted: 3.66 s.
And the images still look like the normal fp16 ones. Same seed, same prompt:
Left: fp16 + PyTorch attention, Right: int8 + int8 attn. The small differences are the kind you also get from int8 attn on an RTX 20 series card: a slightly different sample, not a worse one.
What about my 2060?
On my RTX 2060 12GB, with my usual chips (fp16 acc, fused glue, int8 attn, fp16 vae), the int8 chip takes a 1024x1024, 30 step image from 32.7 s to 28.5 s, and the DiT uses 1.6 GB less VRAM. The speedup there comes mostly from the MLP layers.
How to use it
Both new chips are in Diffucore UI v0.1.22. Update, then turn these on when you load an Anima model:
fp16 acc: any NVIDIA GPU, all model families.
fused glue: Anima only, Linux only for now (it needs Triton).
fp16 vae: any NVIDIA GPU. If the decode ever breaks in fp16, it falls back to fp32 by itself.
int8: Anima, RTX 20 series or newer, needs Triton. Or just load an INT8 checkpoint, which works without the chip.
int8 attn: RTX 20 series or newer. The first load builds the kernel (about a minute) and needs the CUDA toolkit (nvcc) installed.
I only had a 5090 to test this time, so if you're on a 30 or 40 series card, I'd love to see your numbers.
Try Diffucore UI and let me know how it runs on your card.