Quantized fp8 circlestone-labs/Anima for ComfyUI.

It contains calibrated metadata for hardware fp8 linear. If you GPU supports it, ComfyUI will use hardware fp8 automatically. More about hardware fp8 and hardware requirement, see ComfyUI TensorCoreFP8Layout.

All credit belongs to the original model author. License is the same as the original model.

Just ignore ComfyUI log warnings about tons of keys not loaded. Its a small bug in ComfyUI, it checked wrong keys. Those keys are metadata and they are loaded.

fp16 patch: A plugin/patch for ComfyUI to let you run anima with fp16 on old gpus.

Surprisedly, only one line code overflowed. Seems they (Nvidia) capped all activations for linears and attentions, for better hardware fp8 etc. The range of those activations are beautiful. Except one simple matrix multiplication at corner...

So the patch is also simple. Run 99% of the model in fp16, and that one line in fp32. I guess ComfyUI will also have its own patch soon (?).

Because of fp16_accumulation (?), fp16 is little bit (~10%) faster on my 4xxx card than bf16.