Smaller, native-ComfyUI quantizations of Meta's SAM 3.1 Multiplex checkpoint. Both variants use ConvRot weight and activation quantization for selected transformer linears while keeping CLIP and quality-sensitive layers in FP16. They load through ComfyUI's standard checkpoint loader—no custom quantization node or loader is required.<ul><li>INT8 W8A8: the conservative default; 1.19 GiB and very close to FP16 in our spot checks.</li><li>INT4 W4A4: the smallest release; 0.98 GiB, with surprisingly similar masks in our testing.</li><li>FP16 CLIP: included in both files for stock-ComfyUI compatibility.</li><li>Selective quantization: embeddings, normalization, convolutions, geometry and mask heads, final projections, and other sensitive or low-value operations remain FP16.</li></ul>SAM 3.1's headline change is Object Multiplex for faster multi-object video tracking. Meta's release notes focus on optimized tracking and do not claim an image-detector accuracy upgrade, so this should not be read as “SAM 3.0, but smarter at still-image segmentation.” See the <a rel="ugc" href="https://github.com/facebookresearch/sam3/blob/main/RELEASE_SAM3p1.md">official SAM 3.1 release notes</a>.<h3 id="downloads">Downloads</h3><ul><li>INT8 ConvRot W8A8 | <code>sam3.1_multiplex_int8_convrot_selective_fp16clip_fp32source.safetensors</code> | 1.19 GiB | 26.60% smaller than FP16</li><li>INT4 ConvRot W4A4 | <code>sam3.1_multiplex_w4a4_convrot_selective_fp16clip_fp32source.safetensors</code> | 0.98 GiB | 39.95% smaller than FP16</li><li>Official Comfy-Org FP16 reference | <code>sam3.1_multiplex_fp16.safetensors</code> | 1.63 GiB</li><li>Official Meta FP32 source | <code>sam3.1_multiplex.pt</code> | 3.26 GiB</li></ul>Relative to Meta's FP32 source, the INT8 file is 63.42% smaller and the INT4 file is 70.08% smaller. INT4 is 18.19% smaller than INT8.ComfyUI v0.27.0 introduced <a rel="ugc" href="https://github.com/Comfy-Org/ComfyUI/releases/tag/v0.27.0">native INT8 ConvRot support</a>. ComfyUI v0.28.0 added <a rel="ugc" href="https://github.com/Comfy-Org/ComfyUI/releases/tag/v0.28.0">native INT4 ConvRot support</a>. INT4 matrix multiplication is used on supported GPUs; ComfyUI may fall back to INT8 multiplication on unsupported hardware. Keep ComfyUI and <code>comfy-kitchen</code> current, especially on Turing or AMD hardware where backend fixes have continued to land.<h3 id="installation-and-native-workflow">Installation and native workflow</h3>Put either <code>.safetensors</code> file in:ComfyUI/models/checkpoints/Then use the normal native chain:Load Checkpoint → CLIP Text Encode → SAM3 DetectThe files may also live in any checkpoint directory configured through <code>extra_model_paths.yaml</code>. Select the checkpoint with the standard Load Checkpoint node. Do not use a legacy INT8 custom loader.These releases were validated on the native image-segmentation path. The checkpoint also contains SAM 3.1 Multiplex tracking weights, but video tracking has not received the same breadth of release testing yet.<h3 id="quantization-recipe">Quantization recipe</h3>The conversion starts from Meta's official FP32 <code>sam3.1_multiplex.pt</code>, not from the FP16 repack. Its 3,502,755,717-byte size and SHA-256 match <a rel="ugc" href="https://huggingface.co/facebook/sam3.1">the file served by </a><code>facebook/sam3.1</code>:<code>0567DEBEEC80BA4AC6369540C6C248025283CB3FF2B92827509E57E2B3541CB6</code>For each of 230 eligible block-linear weights, the converter:1. upcasts the source weight to FP32;2. applies a block-Hadamard ConvRot transform;3. chooses an MSE-optimized clipping scale;4. stores per-row quantization scales and native ComfyUI quantization metadata.The INT8 file uses W8A8 operations: 8-bit weights with online 8-bit activation quantization at those linears. The INT4 file similarly uses W4A4 operations. Unselected floating-point tensors are stored as FP16. This is dynamic per-operation activation quantization, so no calibration image set is embedded in the files.<h3 id="performance">Performance</h3>Quantization improves file size reliably. Speed is more workload- and GPU-dependent because SAM's preprocessing, mask refinement, convolutions, CPU/GPU synchronization, and relatively small matrix multiplications can dominate the quantized linears.Controlled warm benchmark on an RTX 3090, PyTorch 2.11 + CUDA 13.0, <code>comfy-kitchen</code> 0.2.22, 1280×720 input, six detected people, threshold 0.3, and two native refinement passes. Values are the median of four model-resident runs:<ul><li>FP16 | 1.933 s</li><li>INT8 | 1.647 s | 14.8% faster</li><li>INT4 | 1.396 s | 27.8% faster</li></ul>With refinement disabled, a CUDA operator profile showed INT8 reducing total self GPU time by approximately 17.5%, while time in the main eligible linear operations fell by approximately 32%. In ordinary single-object workflows, warm FP16 and INT8 measurements were both commonly around 0.6–0.7 seconds; a measurable end-to-end speedup is therefore not guaranteed. Image size, detection count, refinement count, GPU generation, offloading, and model warm-up all matter.<h3 id="quality-spot-checks">Quality spot checks</h3>On the pictured official Meta <code>truck.jpg</code> demo image, using the prompt <code>truck</code>, threshold 0.3, and two refinement passes:<ul><li>Comparison | Binary-mask IoU</li><li>FP16 vs. FP32 source | 0.99999</li><li>INT8 vs. FP16 | 0.99765</li><li>INT4 vs. FP16 | 0.99652</li><li>INT4 vs. INT8 | 0.99716</li></ul>On a separate six-person test with refinement disabled, union-mask IoU versus FP16 was 0.99882 for INT8 and 0.98498 for INT4. These are useful regression and visual checks, not a substitute for evaluation across a segmentation benchmark. Some prompts or difficult edges may regress even when these examples look nearly identical.<h3 id="why-these-two-variants">Why these two variants?</h3>The release candidates were deliberately narrowed to selective quantization with FP16 CLIP:<ul><li>Quantized CLIP is not yet broadly native-compatible. It saves another approximately 226 MB, but ComfyUI's SAM 3.1 text-encoder state-dict conversion must also remap the quantization companion tensors. The experimental files work with a local ComfyUI patch, not an unmodified public release, so distributing them now would create a fragile support burden.</li><li>Aggressive targeting offers almost nothing here. In our candidates it saved only about 14 MB beyond selective quantization, produced no repeatable speed gain, and increases the chance that a sensitive projection or short-matrix operation causes a quality regression.</li><li>FP16 CLIP is cheap during repeated segmentation. Text encoding is a small portion of an image run and can be reused, while native compatibility is much more valuable.</li><li>Selective INT4 already held up well. A mixed INT4/INT8 checkpoint remains possible, but initial tests did not justify its added complexity or larger file size.</li></ul><h3 id="future-improvements">Future improvements</h3><ul><li>Upstream-compatible quantized CLIP: extend ComfyUI's SAM text-encoder key conversion to include ConvRot configuration and scale tensors, add round-trip tests, and then repeat prompt-quality evaluation. This is the clearest remaining checkpoint-size reduction.</li><li>Batched and cached refinements: the current native image node refines detections serially and can recompute the 1008-pixel vision path for the same crop across iterations. Batching crops and caching image features should improve real workflows more than quantizing another handful of small layers. This requires native-node code changes, not a new checkpoint.</li><li>SAM-specific fusion or kernels: ConvRot already runs through specialized <code>comfy-kitchen</code> kernels. Larger gains would likely require fusing SAM-specific preprocessing, detector, and refinement operations, reducing dispatch and synchronization, or compiling stable subgraphs. That is substantial, hardware-sensitive engineering rather than a straightforward weight conversion.</li><li>Broader activation quantization: activations are already A8/A4 at quantized linears. Extending low-precision activations into convolutions, geometry processing, or mask heads would require representative calibration, backend support, and careful accuracy analysis; the initial release avoids that risk.</li><li>Video validation: benchmark long clips, object-count scaling, memory use, and tracking stability across FP16, INT8, and INT4. SAM 3.1's largest upstream gains are in this path.</li><li>Wider hardware coverage: publish reproducible results for Turing, Ampere, Ada, Blackwell, AMD/ROCm, and low-VRAM offload configurations.</li></ul><h3 id="checksums">Checksums</h3>D4AF12DF7DB2881ED44476DEBF555A5ACE15234572E6C25EBAA3AF3183AD2133 sam3.1_multiplex_int8_convrot_selective_fp16clip_fp32source.safetensorsA75D0559F508C30FC418FD060A5E3F330F1B7085ED2636F5FD67E3384A2EBBBD sam3.1_multiplex_w4a4_convrot_selective_fp16clip_fp32source.safetensors<h3 id="license-and-credits">License and credits</h3>The original checkpoint is from <a rel="ugc" href="https://huggingface.co/facebook/sam3.1">Meta's gated SAM 3.1 repository</a>. The gate controls access to the upstream download; it does not replace the license. The included weights are derivatives of the SAM Materials and remain subject to the <a rel="ugc" href="https://huggingface.co/facebook/sam3.1/blob/main/LICENSE">SAM License</a>, which permits derivative works and redistribution only under that same agreement and requires a copy of the agreement to accompany the files. <code>SAM_LICENSE.txt</code> is included for that purpose. Review the full terms before uploading or using the checkpoints.The <a rel="ugc" href="https://huggingface.co/Comfy-Org/sam3.1">Comfy-Org SAM 3.1 repository</a> uses the same SAM License for its repackaged FP16 weights. ComfyUI's GPL-3.0 software license applies to ComfyUI code; it does not relicense Meta's model weights.Credits:<ul><li><a rel="ugc" href="https://github.com/facebookresearch/sam3">Meta SAM 3 / SAM 3.1</a> for the model, source weights, and demo image.</li><li><a rel="ugc" href="https://github.com/Comfy-Org/ComfyUI">Comfy-Org/ComfyUI</a> and <code>comfy-kitchen</code> for native SAM 3.1 and ConvRot execution support.</li><li>The comparison and promo use <a rel="ugc" href="https://github.com/facebookresearch/sam3/blob/main/assets/images/truck.jpg">Meta's official </a><code>assets/images/truck.jpg</code>. The promo's abstract background was generated specifically for this release.</li></ul>

Files with SHA256: d4af12df7db2881ed44476debf555a5ace15234572e6c25ebaa3af3183ad2133