A ready-to-run ComfyUI workflow for multi-shot talking-character video: one recurring character across completely different locations, with the same face and the same voice — audio generated together with the picture, not dubbed.
The showcase video is exactly what the workflow does: five shots, five locations, one woman, one voice. Nothing carries her between shots except a single identity sentence repeated byte-identically in every prompt, plus the cross-shot memory bank this graph wires up for you.
Before you dive in — a word on expectations. This is a community-built, bleeding-edge pipeline: a 22B audio+video model with cross-shot memory, running on consumer hardware. It is powerful, but it is not one-click — your first clean render will likely take some tuning to YOUR machine (VRAM, system RAM and pagefile, and which model build). No two rigs behave identically. INSTRUCTIONS.md in the zip covers install, per-VRAM settings, and every failure mode reported so far. If you get stuck, comment here or open a GitHub issue — I answer, I troubleshoot, and most of the fixes in this pack exist because someone reported something. You will not be left hanging.
Quick fixes — read this first
Nearly every problem reported with this pack is one of these.
Renders crawl; VRAM pinned at 100%
Why: Withoutfp8_scaled_mmthe DiT runs in bf16 — ~40 GB staged. On a 32 GB card that streams over PCIe every step.
Fix: Turnfp8_scaled_mmON (with a bf16 or INT8 checkpoint) andsequential_offloadOFF — ~22 GB resident, native fp8 matmul on RTX 40/50-series. It is off by default and is the single biggest speed setting in this pack.fp8_scaled_mmerrors — “needs the bf16 checkpoint as its source”
Why: The toggle downcasts the attention/FF linears itself, so it must start from bf16. An fp8 FILE would load every tensor as fp8 — norms, tables and adalns included — and crash the denoise pipeline.
Fix: Pointmodel_fileat the bf16 (or INT8) build. Do not pair the fp8 file with this toggle.The fp8 file didn't speed anything up
Why: With the toggles off, an fp8 checkpoint upcasts to bf16 at load. It saves download size, not memory or time.
Fix: Use bf16 or INT8 +fp8_scaled_mmfor the actual win.The INT8 file is slow
Why: You ran it withoutfp8_scaled_mm. INT8 reconstructs to bf16 at load, so on its own it is the bf16 path plus ~40 s of reconstruction.
Fix: Turnfp8_scaled_mmON. INT8 then behaves exactly like bf16 as a source — same ~22 GB resident — at roughly 60% of the download.Lip sync drifts apart ~10 s into every shot
Why: The wrapper's video RoPE clock was hardcoded to 24 fps while audio RoPE runs in true seconds — a 25 fps render drifts ~4%/s. Looks like a model limit; it is not.
Fix: Apply the multishot patch (Bug fix #0). No checkpoint can fix it. With the patch, 60–105 s multishot masters hold sync.memory_size=0every shot; a new face each shot
Why: Withenable_audio_memoryoff, the video memory-bank save was gated on the audio latent, so the bank never filled.
Fix: Fixed in the patch (Bug fix #1). Consolememory_size=should climb 0,1,2,… up to your cap.Quality degrades over a long run — waxy skin, smearing by the late shots
Why: The memory-bank trim was a no-op whenmemory_max_size <= num_fix_frames, so the bank grew unbounded.
Fix: Fixed in the patch (Bug fix #1b).memory_size=now freezes at your cap.A GGUF errors about VAEs
Why: A GGUF is DiT-only.
Fix: Keep a full bf16 checkpoint incheckpoint_path— it supplies the VAEs, vocoder and connectors.Burned-in subtitles or captions
Why: The DMD pipeline has no CFG, so a plain negative does nothing.
Fix: Usenegative_prompt_video/negative_scale_video(~0.5). Above ~0.8 it locks every shot to shot 1's composition.The voice comes out British
Why: LTX drifts British unless told otherwise.
Fix: Name it in the positive: “in a casual American accent”.
What's in the zip
The workflow (11 nodes, grouped and labeled: Prompt Source → Enhance & Encode → Models & Reference → Generate → Finish & Save)
Built-in note blocks explaining which widgets to set, which to leave alone, and what to change on bigger cards
A 2-shot example prompt file demonstrating the identity-sentence rule
A README with setup, prompt-writing guidance, and performance notes
What you need (not included)
Node pack: RealRebelAI's
ComfyUI_JoyAI_Echo_GGUF_Nodesplus my multishot patch (v1.2 or newer required)Checkpoint: the echoVid+ltxAud surgical merge — bf16 (43 GB) or the INT8 ConvRot build (27 GB)
Text encoder: any Gemma-3-12B-it, single-file safetensors or GGUF
Setup is two things
Pick your Gemma in the Model Loader's
gemma_filedropdown.Drop
example_multishot.jsonintoComfyUI/input/joyecho_prompts/and pick it in the Prompt Source node. Queue.
Everything else I've published
JoyAI-Echo multishot patch — nodes, releases, issues
LTX-2.3 Multishot Lite — the core-node workflow
Support this work
Everything I publish — workflows, merges, quants, patches — is free and stays free. The models are non-commercial by license and I have no plans to change that.
If something here saved you an evening:
GitHub Sponsors — recurring, no platform cut
Ko-fi — one-off, no account needed
Liberapay — recurring, open-source and non-profit
Bug reports are worth as much as money. Most of the fixes in these packages exist because somebody posted a log.
Description
**v1.2 — A/V sync fix, AutoFinish, encoder picker, ready-to-run workflow**
This listing goes from v1.0 straight to v1.2. v1.1 was the workflow release, which now has its own listing.
**Breaking:** JoyEcho Model Loader's widgets were reorganized — all inputs are now optional, with file pickers above their manual-path fallbacks. Saved graphs from v1.0 will load this node with shifted values: delete and re-add the node (one output wire to reconnect), or start from the bundled workflow.
**Fixed**
- **Multi-shot A/V sync drift.** Each shot's generated audio runs ~30 ms shorter than its video, and the final assembly compounded that — audio ran ~120 ms early by shot 5, so lip sync visibly degraded through the piece. Each shot's wav is now padded to its own video duration before concat; sync is exact by construction. If multi-shot renders drifted out of sync late in a piece, this was why.
- **LLM Enhance no longer demands an API key for local endpoints.** When base_url is a local address (localhost, 127.0.0.1, 192.168.x, .local), a placeholder is sent automatically — Ollama, LM Studio and llama.cpp ignore it anyway. Cloud providers still raise a clear error naming the endpoint. An empty field previously errored even against a local server, which was the most common first-run stumble.
- **Reference Picker loads scripted refs for non-character scenes.** A scene's JSON `refs` block (`{"kelpie": "shot.png"}`) is now honored directly, so creature and location references load without the folder name having to appear in the prompt prose. Previously only names written in the prose matched.
- Hires refine no longer crashes on resolution retarget (cached seqlen mismatch).
**New**
- **AutoFinish node + worker** (missing from v1.0 entirely): after the render it automatically upscales the per-shot masters and assembles the audio-synced final MP4 — no manual upscale/concat step.
- **Model Loader: `gemma_file` dropdown** — pick the text encoder from `models/text_encoders` or `models/clip` instead of typing a path. Single-file .safetensors or .gguf encoders supported (v1.0 required the HF directory).
- **Prompt Source: `manual_path`** — point at any .txt/.json prompt file anywhere on disk, no folder conventions needed.
- **Conditioning cache** — the slow Gemma encode is cached per (prompt, encoder); re-renders with unchanged text skip the encode entirely.
- **Reference Picker:** character dropdown that survives node refreshes; default refs folder is now `input/joyecho_refs/` (one subfolder per character).
- **Writer system prompts** gained two hard-won sections: FRAMING (use shot-type nouns — "framed from the waist up" gets ignored or mis-read; keep speaking shots at medium close-up or tighter, because the VAE packs 32 px into one latent token and a wide shot puts the mouth below one token) and HUMAN MOVEMENT (state gait/turn/contact mechanics; only describe body parts that are in frame). The short-story prompt file is now included alongside the long-story one.
Canonical source and updates: https://huggingface.co/joeygambino/joyai-echo-multishot-workflow
FAQ
Comments (10)
example workflow doesn't load and it's 3kb only, i don't think it's correct
My fault, I had uploaded the nodes patch only without the workflow. Workflow is now in the zip as well. Sorry, these are my first uploads on Civitai.
I've re-uploaded the zip with the example included, sorry!
@joeygambino I think I've set everything up, but now I can't see the nodes where to insert the source image.
@joeygambino In general, this is the only process as I understand it for the model and how it works is not clear at all. If you make a simple i2v process, it will be cool, but what we have now is not clear what and how it works.
@joeygambino As I understand it, you can only use 1 low in your process, which is very bad today. You need a unit that allows you to load several at once.
@Renessance The reference is optional, which is why there's no image node in the graph — but that clearly needed documenting. To wire one: right-click > Add Node > image > Load Image, then connect its IMAGE output to the reference_image input on the left edge of the JoyEcho_Generate node. It plants your image into the cross-shot memory bank so every shot is conditioned on that face. If you want shot 1 to open ON your image (classic i2v behavior), also set reference_zoom to 1.0. Just pushed v1.3.1 with a full INSTRUCTIONS.md covering this step-by-step.
@Renessance Fair criticism — the zip explained the settings but never the model of how this works. It's multi-shot text-to-video with joint audio: you write shots, a memory bank carries the character between them, and the one hard rule is repeating the character description word-for-word identical in every shot. A reference image is an optional identity lock on top, not the starting frame. v1.3.1's INSTRUCTIONS.md walks the whole flow from install to first render.
@Renessance Correct — the loader fuses one LoRA at load time. Today's workaround is merging your LoRAs into one file first (any merge tool; the fused result loads fine). Multi-LoRA loading is on the roadmap for a future patch.