<h1 id="moss-image-sft-segmentation-control-lora">MOSS-Image-SFT — Segmentation Control LoRA</h1><h2 id="short-description">Short description</h2><p>A segmentation Control LoRA for <strong>MOSS-Image-SFT</strong>. Feed a flat-color region map. The model follows the region boundaries, and the prompt decides what each region becomes.</p><p>This is a ControlNet-style <strong>edit LoRA</strong>, not the official MOSS color/depth concat adapter. Load it with a LoRA loader and pass one control image through <strong>Text Encode MOSS-Image Edit</strong>.</p><p>Selected checkpoint: training step <strong>4500</strong>.</p><h2 id="files">Files</h2><ul><li><p><strong>Weights:</strong> <a target="_blank" rel="ugc" href="https://huggingface.co/thedeoxen/MOSS-Image-segmentation-controlnet/blob/main/moss_image_sft_segmentation_control_lora.safetensors">moss_image_sft_segmentation_control_lora.safetensors</a></p></li><li><p><strong>ComfyUI workflow:</strong> <a target="_blank" rel="ugc" href="https://huggingface.co/thedeoxen/MOSS-Image-segmentation-controlnet/blob/main/moss_image_segmentation_control.json">moss_image_segmentation_control.json</a></p></li><li><p><strong>Repo:</strong> <a target="_blank" rel="ugc" href="https://huggingface.co/thedeoxen/MOSS-Image-segmentation-controlnet">thedeoxen/MOSS-Image-segmentation-controlnet</a></p></li></ul><h2 id="examples">Examples</h2><p>Each preview is one combined image, left to right: <strong>control</strong>, then three prompts.</p><h2 id="extended-description">Extended description</h2><p>Follows flat color regions. Boundaries and the big masses stay, while the prompt fills them with a new subject and style. Appearance, materials, clothing and the world come from the text prompt.</p><p>The LoRA is rank 64 on linear layers and rank 16 on convolutions. It was trained as an edit adapter: the control image is the reference, and the prompt describes the picture to generate.</p><h2 id="how-to-use">How to use</h2><ol><li><p>Load <strong>MOSS-Image-SFT</strong>: UNet <code>moss_image_sft_int8_convrot.safetensors</code>, text encoder <code>qwen3.5_9b_w4a8_gptq.safetensors</code> with type <code>moss_image</code>, VAE <code>flux2-vae_small.safetensors</code>.</p></li><li><p>Load this LoRA on the UNet at strength <strong>1.0</strong>.</p></li><li><p>Build the control map and pass it as the single reference image in <strong>Text Encode MOSS-Image Edit</strong>.</p></li><li><p>Set <strong>edit format</strong> to <code>ai-toolkit</code> on both the positive and the negative encode. This LoRA was trained with kv cache on.</p></li><li><p>Sample with <strong>30 steps</strong>, <strong>CFG 4</strong>, <code>euler</code> + <code>simple</code>.</p></li></ol><p>No trigger word. The control image sets the structure. The prompt sets identity, materials, clothing and the world.</p><h2 id="comfyui">ComfyUI</h2><p>The workflow in this repo loads a photo, builds the control map, encodes it with the prompt, and samples.</p><p>The included workflow uses the <strong>UniFormer</strong> semantic segmentor. The training maps are fully filled flat colors, with no large black unlabeled areas. The colors are region ids, not the colors you want in the result.</p><p>Also uses the built-in <strong>Text Encode MOSS-Image Edit</strong> node. Set the edit format to <code>ai-toolkit</code>.</p><h3 id="base-model">Base model</h3><p>Trained on and recommended with <a target="_blank" rel="ugc" href="https://huggingface.co/OpenMOSS-Team/MOSS-Image-SFT"><strong>OpenMOSS-Team/MOSS-Image-SFT</strong></a>.</p><p>Typical settings from the included workflow: <strong>30 steps</strong>, <strong>CFG 4</strong>, <code>euler</code> + <code>simple</code>, LoRA strength <strong>1.0</strong>.</p><p>Negative prompt used in training samples:</p><pre><code>nsfw, worst quality, bad anatomy, watermark, jpeg artifact</code></pre><h3 id="example-prompt">Example prompt</h3><pre><code>Full-body photograph of a circus ringmaster in scarlet tails, gold braid and a tall black hat, standing three-quarter view in front of a painted circus tent. One arm raised high, the other holding a slender black cane down at the side. Confetti, saturated color, entire body in frame.</code></pre><h2 id="what-it-does">What it does</h2><ul><li><p>Follows flat color regions. Boundaries and the big masses stay, while the prompt fills them with a new subject and style.</p></li><li><p>Lets the prompt define identity, materials, clothing and style.</p></li><li><p>Works as a single-image control adapter for MOSS-Image-SFT.</p></li></ul><h2 id="tips">Tips</h2><ul><li><p>Flat, solid colors. Gradients and textures in the control map confuse the regions.</p></li><li><p>Do not expect a region color to become that color in the picture. Say the material in the prompt.</p></li><li><p>UniFormer maps match these weights better than a segmentor that leaves the background black.</p></li><li><p>Lower the LoRA strength toward 0.7 if the control is too rigid. Strength 1.0 is the trained value.</p></li></ul><h2 id="use-cases">Use cases</h2><ul><li><p>Blocking a scene into regions, then restyling each mass from the prompt.</p></li><li><p>Keeping a layout of people, furniture and background while changing the render.</p></li><li><p>Color-script layouts where only the shapes are decided.</p></li></ul><p></p>

Files with SHA256: 1cce5ff3ee6732cf5f07bce6743a40ab91b565d1ae0a247ec7340a113e39d735