A ready-to-run ComfyUI workflow for multi-shot talking-character video: one recurring character across completely different locations, with the same face and the same voice — audio generated together with the picture, not dubbed.
The showcase video is exactly what the workflow does: five shots, five locations, one woman, one voice. Nothing carries her between shots except a single identity sentence repeated byte-identically in every prompt, plus the cross-shot memory bank this graph wires up for you.
Before you dive in — a word on expectations. This is a community-built, bleeding-edge pipeline: a 22B audio+video model with cross-shot memory, running on consumer hardware. It is powerful, but it is not one-click — your first clean render will likely take some tuning to YOUR machine (VRAM, system RAM and pagefile, and which model build). No two rigs behave identically. INSTRUCTIONS.md in the zip covers install, per-VRAM settings, and every failure mode reported so far. If you get stuck, comment here or open a GitHub issue — I answer, I troubleshoot, and most of the fixes in this pack exist because someone reported something. You will not be left hanging.
Quick fixes — read this first
Nearly every problem reported with this pack is one of these.
Renders crawl; VRAM pinned at 100%
Why: Withoutfp8_scaled_mmthe DiT runs in bf16 — ~40 GB staged. On a 32 GB card that streams over PCIe every step.
Fix: Turnfp8_scaled_mmON (with a bf16 or INT8 checkpoint) andsequential_offloadOFF — ~22 GB resident, native fp8 matmul on RTX 40/50-series. It is off by default and is the single biggest speed setting in this pack.fp8_scaled_mmerrors — “needs the bf16 checkpoint as its source”
Why: The toggle downcasts the attention/FF linears itself, so it must start from bf16. An fp8 FILE would load every tensor as fp8 — norms, tables and adalns included — and crash the denoise pipeline.
Fix: Pointmodel_fileat the bf16 (or INT8) build. Do not pair the fp8 file with this toggle.The fp8 file didn't speed anything up
Why: With the toggles off, an fp8 checkpoint upcasts to bf16 at load. It saves download size, not memory or time.
Fix: Use bf16 or INT8 +fp8_scaled_mmfor the actual win.The INT8 file is slow
Why: You ran it withoutfp8_scaled_mm. INT8 reconstructs to bf16 at load, so on its own it is the bf16 path plus ~40 s of reconstruction.
Fix: Turnfp8_scaled_mmON. INT8 then behaves exactly like bf16 as a source — same ~22 GB resident — at roughly 60% of the download.Lip sync drifts apart ~10 s into every shot
Why: The wrapper's video RoPE clock was hardcoded to 24 fps while audio RoPE runs in true seconds — a 25 fps render drifts ~4%/s. Looks like a model limit; it is not.
Fix: Apply the multishot patch (Bug fix #0). No checkpoint can fix it. With the patch, 60–105 s multishot masters hold sync.memory_size=0every shot; a new face each shot
Why: Withenable_audio_memoryoff, the video memory-bank save was gated on the audio latent, so the bank never filled.
Fix: Fixed in the patch (Bug fix #1). Consolememory_size=should climb 0,1,2,… up to your cap.Quality degrades over a long run — waxy skin, smearing by the late shots
Why: The memory-bank trim was a no-op whenmemory_max_size <= num_fix_frames, so the bank grew unbounded.
Fix: Fixed in the patch (Bug fix #1b).memory_size=now freezes at your cap.A GGUF errors about VAEs
Why: A GGUF is DiT-only.
Fix: Keep a full bf16 checkpoint incheckpoint_path— it supplies the VAEs, vocoder and connectors.Burned-in subtitles or captions
Why: The DMD pipeline has no CFG, so a plain negative does nothing.
Fix: Usenegative_prompt_video/negative_scale_video(~0.5). Above ~0.8 it locks every shot to shot 1's composition.The voice comes out British
Why: LTX drifts British unless told otherwise.
Fix: Name it in the positive: “in a casual American accent”.
What's in the zip
The workflow (11 nodes, grouped and labeled: Prompt Source → Enhance & Encode → Models & Reference → Generate → Finish & Save)
Built-in note blocks explaining which widgets to set, which to leave alone, and what to change on bigger cards
A 2-shot example prompt file demonstrating the identity-sentence rule
A README with setup, prompt-writing guidance, and performance notes
What you need (not included)
Node pack: RealRebelAI's
ComfyUI_JoyAI_Echo_GGUF_Nodesplus my multishot patch (v1.2 or newer required)Checkpoint: the echoVid+ltxAud surgical merge — bf16 (43 GB) or the INT8 ConvRot build (27 GB)
Text encoder: any Gemma-3-12B-it, single-file safetensors or GGUF
Setup is two things
Pick your Gemma in the Model Loader's
gemma_filedropdown.Drop
example_multishot.jsonintoComfyUI/input/joyecho_prompts/and pick it in the Prompt Source node. Queue.
Everything else I've published
JoyAI-Echo multishot patch — nodes, releases, issues
LTX-2.3 Multishot Lite — the core-node workflow
Support this work
Everything I publish — workflows, merges, quants, patches — is free and stays free. The models are non-commercial by license and I have no plans to change that.
If something here saved you an evening:
GitHub Sponsors — recurring, no platform cut
Ko-fi — one-off, no account needed
Liberapay — recurring, open-source and non-profit
Bug reports are worth as much as money. Most of the fixes in these packages exist because somebody posted a log.
Description
**v1.3 is a COMPLETE pack** — nodes, workflow, and example prompts in one zip, with a README_FIRST.txt that says where each piece goes. If you downloaded v1.2 and couldn't find the workflow or the example file: they were in that zip too, at the root where nobody looks. Now they live in a `workflow/` folder and the README points at everything. Nothing extra to hunt down except the models.
**Fixed: the VHS shot transition now reaches the final upscaled master.** The transition was always applied to the in-graph preview stream, but the AutoFinish worker rebuilds the master from the per-shot files — which are captured *before* the transition — so upscaled masters came out with clean, unmarked cuts at every shot boundary. You mostly wouldn't notice when consecutive shots changed location (the content sells the cut), but in same-location pieces the boundaries read as one continuous take. The worker now applies the same effect — snow, tearing bands, dropout scanlines, plus the tape-static audio hit spanning the room-tone seam — and it configures itself: the Generate node writes its transition settings to a small sidecar file each run, and the worker follows them, including `cut` and `dissolve`. No new widgets.
**Corrected: the `v2a_grad_scale` note.** The v1.2 workflow note claimed changing it degrades sync. That was wrong — the parameter is training-only and has no effect at inference. The note in the workflow now says so. (`num_fix_frames` and `audio_memory_window_size` remain genuinely tuned values; the warning stands for those.)
**After updating: open your workflow once, check the last few widgets on the Generate node (reference_zoom / resident_blocks / hires), and save.** ComfyUI maps saved widget values to widgets by position, so a graph saved under an older version of the pack can load a few trailing widgets scrambled — silently, every time — until you re-save it once against the new layout. If you have ever found yourself re-entering the same two or three values after every restart, this is why, and one save ends it.
Requires RealRebelAI's ComfyUI_JoyAI_Echo_GGUF_Nodes as the base install, same as before. Models are on the linked model listing.FAQ
Comments (7)
Hello,
It seems that your code and instructions are mixed up—some nodes are actually included in your ZIP file but are listed as not being there, while others are different versions of the original nodes (RealRebelAI's ComfyUI_JoyAI_Echo_GGUF_Nodes). Additionally, regardless of the combinations with checkpoints or diffusion models, there are quantization errors. Your idea seems excellent, but it looks difficult for me to reproduce it for testing. Good luck with the rest of your work.
This pack is a patch add-on to RealRebelAI's node pack, so you must install his node pack first. This information is in the instructions.md file included with my pack.
You are also welcome to open discussions with more specific errors and requests at either my Github or HuggingFace repos:
jlucasmcrell/joyai-echo-multishot-patch: Multishot fixes + automation patch for the ComfyUI JoyAI-Echo GGUF node pack (LTX-2.3). Memory-bank trim fix, lip-sync fps fix, hires routing, AutoFinish masters.
joeygambino (Joey)
I am always happy to help, but I do need more information concerning what you're experiencing. Everyone's hardware and software versions are unique, so it's difficult to account for every minor difference.
@joeygambino Thanks for your reply.
However, I'm still getting the same errors whether I use gguf or safetensors with
I'm having trouble understanding the exceptions related to the model choice (should I choose gguf and rename it to safetensors?). Sorry... maybe I'm just too tired.
In any case, good luck with your project
@sdktertiaire2 You do not rename anything. I would have to see the specific errors you're receiving. It's all very much determined by your hardware limitations, ComfyUI version, etc.
If you can provide errors or screenshots in either of the repos mentioned by opening a discussion, I can be of much more help. There are already multiple discussions going on in those repos that may be of help as well.
@joeygambino ok i will try harder. I'll keep you posted. thanks
I was one of the people that really wanted this to work.Believe me. But unless i'm supposed to be doing something extravagant on top of a typical workflow, the facial consistency from the original picture for an image to video is not sustaining at all. The person's face morphs into the person fromYOUR videos and images. Not mine. I hope it gets better though!!
I'd honestly love to see how you accomplished that, as none of my images are embedded in the workflow. You can confirm that by doing a simple image search in the workflow folder. Feel free to open a discussion at HF or Github, I've talked several people through getting the workflow functioning properly.