Regional LoRA QKV — multi-character LoRAs in ONE image.

🎭 Regional LoRA QKV — multi-character LoRAs in ONE image. We might have actually solved it.

Two (or more) trained character LoRAs, one frame, zero identity bleed — at strength 1.0. If you've ever put two character LoRAs in the same Z-Image render and watched them melt into the same face, here's the part nobody told you: on fused-attention models, regional engines have been silently throwing away the layers that carry the identity. This pack fixes that — and adds a router so you don't even have to draw the masks.

🔬 The actual problem (and the actual fix)

Z-Image / Lumina2 (NextDiT) fuses attention QKV into a single Linear(2560 → 11520). Character LoRAs are trained on split projections — separate to_q / to_k / to_v, each 2560 → 3840. Those shapes don't match the fused module, so a typical regional engine's dimension gate skips them: it applies ~120 attention-out + MLP layers per character and drops the entire QKV stack — where most of the likeness lives. That's why regional identity has felt weak, and why people crank strength to 1.3–1.5 and still get mush.

This engine composes each split LoRA into the correct output slice of the fused module — q into 0–3840, k into 3840–7680, v into 7680–11520 — as a mask-gated delta. The base forward is untouched, masking still gates tokens, and identity arrives at strength 1.0. With diagnostics on you can watch it happen: matched 120 full + 90 fused-QKV layers.

🧭 Zero-touch placement (Auto Region)

Tier 1 — box LLM: a local Ollama vision model (default qwen3-vl:8b) reads your prompt and returns true 2D boxes per character — big foreground face vs small figure down the hall, not just stripes.
Tier 2 — spatial language: no Ollama? "X on the left, Y on the right" is parsed deterministically, with depth handling and automatic axis choice.
Tier 3 — mention order: equal feathered bands as the last resort. Feather is a fraction of band width, so seams look identical at every aspect ratio.

Trigger words drive everything: characters whose trigger is missing from the prompt are skipped automatically, so you can batch-queue a prompt file and each prompt selects its own cast. One character active → full frame, no masking overhead.

📦 What's in the zip

ComfyUI-Regional-LoRA-QKV/ — the node pack: Define Region (one per character), Auto Region (optional router), Apply to Model. No pip installs, no extra wheels — plain Python on what ComfyUI already ships.
workflows/ZImage_Regional_QKV.json — a complete, grouped, titled two-character graph using core ComfyUI nodes only plus this pack.
README.md (full technical writeup) + INSTALL.md.

🛠 Complete install — everything you need, start to finish

ComfyUI: a current build with native Z-Image / Lumina2 support (the workflow uses only core loaders/samplers: UNETLoader, CLIPLoader in lumina2 mode, ModelSamplingAuraFlow, EmptySD3LatentImage, SamplerCustomAdvanced).
The pack: unzip ComfyUI-Regional-LoRA-QKV into ComfyUI/custom_nodes/, restart. Nodes appear under RIFT → Regional LoRA QKV.
Z-Image model → models/diffusion_models/. Any Z-Image Turbo build works; tested with fasciumzImageTurbo (lastMERGE) from Civitai.
Text encoder → models/text_encoders/: the Qwen3-4B encoder from the official Z-Image release (fp16 or an fp8 build — both fine). Load it with CLIPLoader, type lumina2 — the workflow has this preset.
VAE → models/vae/: the standard Z-Image 16-channel VAE; tested with UltraFlux VAE Improved v10.
Your character LoRAs → models/loras/. Z-Image character LoRAs trained on split q/k/v (the common trainer output) are exactly what the fused composition is for; LoRAs that already match full layers compose too.
Optional but recommended — Ollama for box placement: install Ollama, then ollama pull qwen3-vl:8b. The node calls http://127.0.0.1:11434 by default and yields VRAM back before sampling (keep_alive). Skip this entirely and Tier 2/3 take over — nothing breaks.
Open the bundled workflow, then touch exactly four things: Character A's LoRA + trigger word, Character B's LoRA + trigger word — and write a prompt that mentions both triggers. Queue.

✅ Verify it's working (first run)

Leave diagnostics ON. The console must print, per character:

matched ~120 full + <non-zero> fused-QKV layers
Fused QKV composition active: 30 fused module(s), equal 3-way split (out=11520 / slice=3840)

fused-QKV: 0 on a Z-Image LoRA means your trainer used q/k/v key names the matcher doesn't know yet — post the printed unresolved-target list in comments and the token rules get extended.

🎛 Settings that matter (the workflow ships with all of these)

patch_size 2 (MM-DiT: Z-Image / Lumina2 / Flux / Qwen-Image) · text_token_position: beginning · normalize_overlapping_masks: ON (overlaps blend instead of double-stacking)
strength 1.0 — the whole point; no more 1.3–1.5 compensation
feather 0.15 · sampling: euler / simple / 8 steps / shift 3.1 (the Turbo recipe)

➕ More characters, manual masks, other models

More people: chain more Define nodes — each character_pack output into the next node's input; the last one feeds Auto Region.
Manual placement: delete Auto Region, wire the last Define straight into Apply, give each Define its own real mask. (The SolidMask in the workflow is just a placeholder — Auto Region rewrites it per prompt.)
Other MM-DiT families (Flux, Qwen-Image): set patch_size accordingly; fused-QKV matching depends on that model's module naming.

⚠️ Honest limitations

Slice composition assumes equal contiguous q/k/v (standard MHA — true for Z-Image/Lumina2). GQA models are detected and logged, never mis-composed.
Masks are static per generation. Two characters is the tuned case; more works via chaining.

💜 Support

Everything here is free and stays free — the format spec, the nodes, the workflows, the cartridges, the LoRAs. If it saved you a night of debugging (it contains several hundred of mine), tips keep the 5090 warm:

☕ Buy me a coffee on Ko-fi
💛 Sponsor on GitHub
🔁 Liberapay (recurring)
⚡ Or right here: the Civitai tip button on this page sends Buzz directly.