What Are FP16, BF16, FP8 and Quantization?
If you work with local AI models, you have probably come across terms such as FP16, BF16, FP8, INT8 and quantization.
You might see different versions of the same model available for download, each with a different file size. Some are labelled BF16, others FP8 or INT8. And when you load a model in ComfyUI, you may discover that one version fits into your GPU memory while another runs out of VRAM.
So what do these terms actually mean?
The good news is that you don't need to be a mathematician to understand them. Once you know the basics, choosing the right model for your hardware becomes much easier.
What Does Precision Mean in AI?
AI models contain large numbers of learned values called weights. These values help the model determine how to process information and generate an output.
Each value needs to be stored in memory.
Precision describes how numbers are represented and how much numerical detail that representation can preserve.
Higher-precision formats can represent values with greater numerical accuracy, but they generally require more memory. Lower-precision formats can reduce memory usage and sometimes improve performance, although they introduce different trade-offs.
This is particularly important for local AI, where your GPU has a limited amount of VRAM.
The most common formats you will encounter include:
FP16: 16-bit floating point
BF16: 16-bit brain floating point
FP8: 8-bit floating point
INT8: 8-bit integer representation
4-bit quantization: a compressed representation using approximately four bits per quantized value
Let's look at what makes them different.
What Is FP16?
FP16 stands for 16-bit floating point, also known as half precision.
It stores each number using 16 bits, or two bytes.
FP16 has been widely used in deep learning because it can significantly reduce memory usage compared with FP32, the 32-bit floating-point format.
For AI image and video generation, FP16 can be useful for:
Running diffusion models
Generating images in ComfyUI
Training or fine-tuning certain models
Reducing model memory requirements
Taking advantage of GPU acceleration for half-precision operations
FP16 provides more precision in the significant digits than BF16, but it has a smaller range of representable values.
That difference can matter during training, where numerical stability is especially important.
For inference, FP16 is a common option when a model and its software implementation support it.
What Is BF16?
BF16 stands for brain floating point 16.
Like FP16, it uses 16 bits per value. That means the raw storage requirement for model weights is essentially the same.
The difference is how those bits are allocated.
BF16 uses more bits for the exponent and fewer for the fraction, also called the mantissa. This gives it a much wider numerical range than FP16, but less precision between nearby values.
Why is that useful?
Deep learning calculations can involve very small or very large numbers. BF16's wider range can make numerical behaviour more manageable, particularly during training.
BF16 is therefore widely used in modern machine learning and generative AI.
FP16 vs. BF16
FeatureFP16BF16Storage per value16 bits16 bitsRaw weight memorySimilarSimilarNumerical rangeSmallerMuch widerPrecision between nearby valuesHigherLowerCommon useTraining and inferenceTraining and inference
Neither format is automatically better in every situation. The model, hardware and implementation determine which one makes sense.
One important detail for local AI creators: a model saved in BF16 doesn't necessarily run faster than the same model in FP16. Both formats use the same number of bits, and actual speed depends on your GPU and the operations being performed.
What Is FP8?
FP8 stands for 8-bit floating point.
It stores each value using eight bits, or one byte. Compared with FP16, that halves the raw storage required for values represented in FP8.
FP8 is increasingly relevant in modern AI systems because it can reduce memory requirements and improve throughput on supported hardware.
However, FP8 is not one single numerical format. Common variants include E4M3 and E5M2, which allocate their bits differently between the exponent and the mantissa.
In simple terms, different FP8 formats balance numerical range and precision in different ways.
FP8 can be useful for:
Reducing the memory footprint of model weights
Increasing throughput on compatible GPUs
Running large models more efficiently
Supporting optimized training and inference workflows
But there is a catch.
FP8 requires suitable hardware and software support to deliver its intended benefits.
A model stored in FP8 may still need to be converted or processed in another precision for certain operations. Not every GPU or application supports every FP8 workflow equally well.
This is why checking the model's recommended format and implementation is important before downloading it.
What Is Quantization?
Quantization is a broader technique for representing model values using a more compact numerical format.
Instead of storing every weight in its original precision, a quantized model stores values in a lower-bit representation, often using scales and other information to map the compressed values back to useful numerical ranges.
You will commonly see quantized models labelled:
INT8
8-bit
6-bit
5-bit
4-bit
3-bit
2-bit
The lower the bit count, the less memory the quantized values generally require.
However, quantization is not simply deleting unnecessary data. It changes how the model's numerical values are represented, which can affect the final result.
Different quantization methods preserve information differently. Two models with the same nominal bit width can therefore have different quality, memory requirements and performance.
Why Quantization Matters
Imagine a model that is too large to fit comfortably into your GPU memory.
A quantized version might fit, allowing you to run a model that would otherwise be impractical on your hardware.
This is particularly valuable for local AI users who want to experiment without upgrading their GPU.
Quantization can also reduce memory bandwidth requirements and improve performance, depending on the hardware and software implementation.
However, lower-bit models can introduce quality loss, and some formats may require additional processing that reduces their speed advantage.
How Much VRAM Can You Save?
Let's use a hypothetical model with seven billion parameters.
The table below estimates the memory needed for its weights alone, assuming every parameter is stored using the indicated number of bits.
<box border radius="lg" padding={3} gap={3}> <text color="secondary" size="sm">Estimated weight storage for a 7B-parameter model</text> {#each [{name:"FP32",bits:32,mem:"28 GB"},{name:"FP16 / BF16",bits:16,mem:"14 GB"},{name:"FP8 / INT8",bits:8,mem:"7 GB"},{name:"4-bit",bits:4,mem:"3.5 GB"}] as item} <box gap={1}> <row align="center" justify="between"> <text weight="medium">{item.name}</text> <title size="md" tabularNums>{item.mem}</title> </row> <box background="surface-tertiary" radius="md" height="8px" clip> <box background="surface-inverted" radius="md" height="8px" width={`${item.bits/32*100}%`} /> </box> </box> {/each} <text color="secondary" size="xs">Approximate decimal GB, based on raw weight storage only. Real files and runtime memory will differ.</text> </box>
These numbers show why quantization is so useful.
A seven-billion-parameter model stored in FP16 requires approximately 14 GB for its raw weights alone. A four-bit representation needs roughly 3.5 GB for the same number of parameters before quantization metadata and other overhead are included.
But these figures do not tell you the total VRAM required to run the model.
Additional memory may be needed for:
Text encoders
Image encoders or other model components
Activations and intermediate tensors
Attention calculations
Latent images or video frames
Temporary buffers
The application itself
For video generation, the number of frames and the resolution can have a substantial effect on memory usage.
That is why a model advertised as fitting into a particular amount of VRAM may still run out of memory in a demanding workflow.
FP8 vs. INT8: Are They the Same?
No. Both use eight bits per value, but they represent numbers differently.
FP8 is a floating-point format. INT8 uses integer values, usually combined with scaling information to represent a model's original weights.
The distinction matters because the two formats have different numerical properties and hardware support.
In practice, you should not assume that an FP8 checkpoint and an INT8 quantized model are interchangeable just because both use eight bits.
They may require different model loaders, kernels or conversion procedures.
What About GGUF?
If you explore local language models, you will probably encounter GGUF files.
GGUF is a model file format commonly associated with llama.cpp and compatible inference tools. It can store model weights in a range of quantized formats, as well as other information required to load the model.
Names such as Q4_K_M and Q5_K_M indicate particular quantization schemes.
GGUF is not itself a precision level. It is a container format that can hold weights represented using different numerical formats.
It is also important to understand that GGUF support for language models does not automatically mean that every image or video model can be loaded through the same workflow.
Diffusion models and multimodal systems often need model-specific support.
Which Format Should You Choose?
There is no single format that is best for everyone.
Your choice depends on what you want to do and what your hardware supports.
Choose FP16 or BF16 when...
You want to use a model in its native or recommended half-precision format, have enough memory, and your workflow supports it.
These formats are particularly useful when you want to avoid the extra compromises that can come with aggressive quantization.
Choose FP8 when...
The model provides an FP8 checkpoint, and your GPU and software support the required operations efficiently.
FP8 can offer a useful balance between reduced storage and numerical representation, but compatibility matters.
Choose 8-bit or 4-bit quantization when...
Memory is your main limitation and you want to run a model that would otherwise be too large for your system.
A good quantized model can be a practical solution for local inference, although quality and speed depend on the particular quantization method and implementation.
Check the model's recommendations first
Don't download a random quantized checkpoint just because it has the smallest file size.
Check which format the model author recommends, whether your software supports it, and how much memory the complete workflow requires.
A smaller file is useful only if you can load it correctly and get the results you need.
What Does This Mean for ComfyUI?
In ComfyUI, you may see different checkpoints or workflow instructions referring to FP16, BF16, FP8 or quantized weights.
These formats can influence memory usage, compatibility and generation speed.
However, the model's architecture and the workflow also matter.
For example, a video model can be demanding because it processes a sequence of frames. A quantized checkpoint might reduce the memory required for its weights, but the video workflow still needs memory for intermediate calculations.
Likewise, a model may support BF16 but require a particular loader or configuration to run correctly.
If a workflow fails with an out-of-memory error, changing precision or using a supported quantized version can sometimes help. Other options include reducing resolution, generating fewer frames or using memory-saving settings.
It is best to change one variable at a time so you can identify what actually improves the result.
Final Thoughts
FP16, BF16, FP8 and quantization all help determine how AI models store and process numerical values.
The important differences are straightforward:
FP16 and BF16 use 16 bits per value but allocate their bits differently.
FP8 uses an eight-bit floating-point representation.
Quantization is a broader approach to representing model values with fewer bits, often reducing memory requirements.
GGUF is a file format that can contain weights stored using different quantization schemes.
For local AI creators, understanding these terms makes it easier to choose the right model for their hardware.
You don't always need the largest or highest-precision version of a model. Sometimes a smaller, quantized checkpoint gives you the best practical balance between memory, speed and quality.
The goal isn't simply to use the smallest model. It's to find the format that lets you create what you want, on the hardware you have.