MiniMax H3 Unified · Nova.2 — Lip-sync, Scene Prompts and Resumable Video

Workflow: MMH3_Unified.json

Workflow version: Nova.2

Guide updated: 9 November 2026

Create a performance video from an existing recording, or generate video and audio from text and images. Both paths use the same render loop, with a prompt for each scene, an accumulated preview, and saved runs you can continue or replace from a selected part.

The optional reference-video analysis finds scene boundaries and describes the movement between them. You can review and edit those prompts directly before rendering, or enter all timings and prompts manually. Optional YuE2 and MiniMax Music 3 groups can create a soundtrack; lyric extraction and animated captions are also included.

This guide uses lip-sync with a character image and an existing recording as its main example. Native H3 is an alternative path. Model names and numeric settings below are examples to adapt to your files and hardware; the workflow may open with different selections saved during testing.

Recommended first test: Use a short excerpt, leave music generation and captions off, and generate one part per queue. Keep Disk and Keep in output enabled so you can continue from the result.

What is new in Nova.2

Prompt Review shows the opening prompt and all later scene prompts together. Leave Pause off for automatic passthrough, or pause before rendering, edit in place and continue the same run. The dropdown and previous/next arrows follow the Eclipse Nova theme.
Part Prompt Editor lets you correct the combined prompt for an individual render part of a saved run, then apply it on Resume.
SmartLLM scene input accepts the scene sampler's disk frames wherever the selected task supports images. Together analyzes the ordered selection as one scene; ordinary image and MiniMax H3 tasks can feed the collector through Scene JSON.
Task and mode controls distinguish motion transfer from full scene descriptions, keep Thinking optional and off by default, and select Standard/NSFW instructions independently of the Training examples.

Choose what you want to make

Lip-sync: animate a reference image to an existing song, dialogue or other vocal recording. Use manual scene prompts or copy the editing rhythm and gestures from a reference video.
Native H3: generate from text, a starting image, first/last images, different images for different scenes, or visual references. Supply a soundtrack or let H3 generate the sound.
Resume: continue an unfinished saved run without generating the completed parts again.
Replace: keep the beginning of a saved run and generate a new version from a chosen part onward.
Load: reuse a saved run for captions or another export without sampling the video again.

The workflow is organized into groups and subgraphs. Open a subgraph when you need its additional controls; most everyday changes are on the main canvas. The groups share their values through Get/Set routing.

Install and open the workflow

Use a ComfyUI build with native MiniMax H3 support. The working setup for this guide uses ComfyUI 0.42.1. YuE2 and MiniMax Music 3 support is needed when enabling their optional groups.

Install or update these Nova packs, including their dependencies in the Python environment used by ComfyUI:

ComfyUI Eclipse Nova — input panels, scene analysis, prompt review/editing, planning, the render loop, saved runs, previews and captions.
ComfyUI Smart Model Loader Nova — H3/music loaders, loader presets, LoRA presets and Download Manager.
ComfyUI SmartLLM Nova — reference-frame descriptions and optional music/lyric prompting.

The optional patch groups contain nodes from ComfyUI-sol-attn, including Diff-Aid. Vocal separation uses Audio Separation Nodes. Install those packs to have every included node available; their processing groups can stay muted for the first test.

Restart the ComfyUI backend after installing or updating Python nodes, then refresh the browser and open the workflow. The render and analysis loops are provided by Eclipse Nova; Easy-Use is not required.

Replace the example image, video and audio selections with your own files. An unavailable example saved-run selection is harmless in New run mode. Resume and Load require a run that exists on your computer.

Models, presets and downloads

Switch the generation path and both presets

In START HERE, choose Lip-sync or Native H3. Then select the matching setup in both Smart Model Loader and LoRA Stack:

Lip-sync: MiniMaxH3_LipSync in the loader and the separate LoRA preset list.
Native H3: MiniMaxH3_TFL2VA in both lists.

These presets belong to Smart Model Loader Nova. Use its current template files, or save your own versions after choosing your local models. Loading the workflow preserves its embedded settings, but your local preset list is managed separately.

Check when switching: The generation-path buttons do not automatically load model or LoRA presets. Check all three selections before queueing. The analysis task and appearance option are separate choices too.

H3 files used by this setup

Both paths need an H3 diffusion model, the MiniMax H3 Qwen3-VL 32B text encoder, the H3 video VAE and the H3 audio VAE.

Lip-sync example model: minimax_h3_fl2va_pruned_int8_convrot.safetensors, available from Comfy-Org/MiniMax-H3. Store it in ComfyUI/models/diffusion_models/, optionally in a minimax subfolder.
Optional community native model: minimax/h3ErosMax_beta7_3329966.safetensors. Its exact Civitai AIR is:
urn:air:minimaxh3:diffusionmodel:civitai:2851079@3429412+3329966
H3 ErosMax — example model version
H3 text encoder: the workflow calls its local file qwen3-vl-32b-minimax-h3-int8-convrot.safetensors. The official INT8 ConvRot file uses underscores and has the same SHA-256. Download it into ComfyUI/models/text_encoders/ and select its actual filename in the loader.
Video VAE: minimax_h3_video_vae_fp16.safetensors.
Audio VAE: minimax_h3_audio_vae_fp32.safetensors.

Both VAEs are in the H3 VAE folder; place them in ComfyUI/models/vae/.

For native endpoint modes, choose an FL2VA-compatible model. For reference_images, choose a Ref2VA-compatible model; the official H3 repository also provides minimax_h3_ref2va_pruned_int8_convrot.safetensors. Check the model author's supported modes when using a community model.

Two different language-model roles: H3's 32B encoder is part of video generation. The smaller vision model in Smart LM describes reference frames. Installing one does not replace the other.

Use a saved AIR in Download Manager

An AIR identifies the selected Civitai resource. The saved AIRs here include the version and exact file, so you can find the intended download even when the workflow's local filename is missing.

In Smart Model Loader, open the missing resource's Source / integrity controls. For a LoRA, use the row's i button.
Keep the saved CivitAI source and AIR, or paste an AIR into the source field.
Click Open in Download Manager. Review the listed file and its destination: diffusion_models, text_encoders, vae or loras, as appropriate.
Select the file and click Add Selected to Queue.
Open Queue, select the ready job and click Start Selected.
After completion, the originating field receives the downloaded filename and verified source information. Save the workflow and use Save… if you want to update your preset too.

Keep the workflow open while downloading if you want automatic assignment to the original field. If you reload the browser, select the completed file manually afterward. Opening a workflow or the download panel does not start downloads automatically.

For files without a saved AIR, choose Hugging Face, enter the linked repository or file URL, and follow the same queue/start steps. Model access may require your provider account; configure it through the pack's supported settings. See the illustrated Download Manager guide.

LoRAs and sampling

The lip-sync example uses Euler / beta, CFG 1 and 8 steps, with minimax_h3_turbo_v4_step600_pruned_comfyui.safetensors enabled at strength 1. Download it from the publisher's file page.

An alternative row is minimax_h3_PDMD_DMAD_4step_r256_add_fro1_turbo_lora.safetensors, from Darksidewalker's MiniMaxH3 repository. That repository requires accepting its access conditions. Leave this row disabled for the example above; when choosing a different accelerator, use its matching sampling settings and save that combination as your own loader/LoRA presets.

Optional additions for a native H3 setup include:

MysticXXX_MMH3-V5.safetensors — AIR urn:air:minimaxh3:lora:civitai:2856467@3416478+3309543.
Minimax H3 Cinematic Look.safetensors — AIR urn:air:minimaxh3:lora:civitai:2908686@3289775+3174203.
minimax_h3_fl2v_turbo_8step_v1.0_comfyui_bf16.safetensors — available in the official H3 LoRA folder.

Place LoRAs in ComfyUI/models/loras/ and select the actual local filenames. Disabled rows do not need their files downloaded for execution. Check the enabled rows, sampler, scheduler and step count whenever you change presets; these settings depend on the selected checkpoint and LoRAs.

Models needed only for optional features

Reference analysis / automatic prompting: choose a supported vision-language model in Smart LM. One local ComfyUI example is qwen3-vl-8b-heretic-1.3.0-int8-convrot.safetensors. Its recorded file source is:

urn:air:qwen2:text_encoders:civitai:2955374@3347601+3234858

Use this AIR in the standalone Download Manager, download to models/text_encoders, then refresh and select the model in Smart LM. You can choose another supported vision model for scene analysis, including a configured Ollama model. Text-only lyric prompting does not require vision. Backend setup is covered in the SmartLLM README.

Lip-sync activity encoder: the included audio-encoder branch selects wav2vec2_large_english_fp16.safetensors. Put it in models/audio_encoders/; it is available in Comfy-Org's audio-encoder files. Native H3 generation does not use this separate encoder.

YuE2: select yue2_3b_int8_convrot.safetensors from Comfy-Org/YuE2 checkpoints, under models/checkpoints/. Its loader uses the checkpoint's included text encoder and VAE.

MiniMax Music 3: the saved local names are minimax/minimaxMusic_v30_fp32.safetensors, qwen3-8b-minimax-music-v30-3107729-bf16.safetensors and minimax_music3_dav.safetensors. For a setup from Comfy-Org/MiniMax-Music-3, select minimax_music3_dit_fp32.safetensors, minimax_music3_text_encoder_pruned_bf16.safetensors and minimax_music3_dav.safetensors in the corresponding diffusion-model, text-encoder and VAE folders. Select the downloaded files in the Music 3 loader rather than leaving unavailable example names.

Whisper and optional Qwen timing setup are described in the captions section below. Leave unused groups muted; their models are not needed for a basic video run.

Find the main controls

START HERE · run mode + opt inputs: choose the generation path, then enable Images, Audio override, Reference video, Live analysis and LS vocal isolation as needed.
Shared H3 loader + separate LoRA presets: choose the model setup for that path.
Canvas + output folder · 24 FPS: choose aspect ratio, resolution and delivery folder. Keep H3 at 24 FPS.
Optional images / Reference video / Optional audio override: load your source material and set any excerpt.
Global prompts / Manual cut times + first / next part prompts: write the overall direction and individual scenes.
Part source / MMH3 Reference Analysis: choose live or manual scene inputs and inspect analysis reports.
First + next scene prompts · review: use Prompt Review to display the selected opening and later prompts, optionally pause and edit before generation.
Run operation + saved-run selection: choose New, Resume or Load, storage, and the saved run.
Part Prompt Editor: read the selected saved run's prompts and correct individual render parts before resuming or replacing them.
Shared H3 render loop: set the render seed, part limit and native image mode; watch the assembled preview.
Shared result + save video: review part ranges and export, with optional lyric captions.

First run: lip-sync to your recording

Choose Lip-sync and load the matching loader and LoRA presets.
Turn Images on and load one image in Load Image Timeline. Choose the canvas size and check the crop.
Turn Audio override on and select your recording in Load Audio. For this first test, turn Reference video and Live analysis off and keep both music-generation groups muted.
Choose a short excerpt using Shared clip start (seconds) and the duration on H3 clip (shared caption trim). Duration 0 keeps the remaining recording, so enter a short duration deliberately.
Clear the example manual cut times and first/next prompts if you want one simple performance. Keep a short global direction such as: “The subject sings to the supplied audio with natural head movement and expressive eyes.”
Choose New run, Disk, and Keep in output. Set Parts per queue to 1 in Video loop · saved inputs + resume. Leave Prompt Review's Pause off for this automatic first test.
Queue and review the result. For reference-driven scene prompts, follow the analysis and prompt-review sections below. To continue a longer plan, select the saved run and follow the Resume steps.

The lip-sync example uses the planner's warmup_reset strategy, with a three-second warmup for later reset parts. Warmup gives the model time to settle into the next shot before those hidden frames are removed. The accumulated preview shows the retained result and its join with the previous part.

Opening pose: The first frame still comes from your reference image. If the prompt starts with hands beside the head but the image has hands at the waist, the model may first move them upward. For a very short opening scene, use an image already close to the intended pose.

First run: native H3 from text or images

Choose Native H3 and select the matching loader and LoRA presets.
For a manual text test, turn Live analysis, Reference video and Audio override off. Leave the music and caption groups muted.
Set reference_mode on Video loop · saved inputs + resume to text_only. Turn Images off too when you do not need them.
Write Native H3 global direction + sound, then the manual first-part prompt. Clear old next-part text and cut times for a single-scene test.
Set Duration in Total duration + maximum render window (native H3), for example 8 seconds. This duration is used when neither audio nor a reference video supplies the length.
Choose New run, persistent disk storage and one part per queue. Queue and review.

To use an image, turn Images on, select your file and change reference_mode to first_only. For endpoints, add two images and select first_and_last.

Native H3 builds longer videos from short renders. Max clip frames = 362, for example, is about 15 seconds at 24 FPS. Longer scenes split into additional renders while keeping their scene prompt and image assignment. Those renders are independent; a shared prompt or image does not guarantee continuous movement or generated sound at every join.

Generate, resume and replace parts

The operation dropdown has three choices: New run, Resume saved run and Load saved run. Replacement uses Resume saved run together with Restart part; it has no separate operation button.

Save a run that survives a restart

In Video Run Storage, enable Disk and Keep in output. With the output directory runs, working runs appear beneath ComfyUI/output/runs/. Each run gets its own folder. The final delivery video uses the separate Smart Folder destination, for example video/H3 with a date subfolder.

Disk + Keep in output: retains completed parts and the saved inputs needed to resume after restarting ComfyUI.
Disk + Temporary: reduces accumulated memory use, but the working files follow the temporary cache lifetime.
RAM: keeps decoded parts in CPU memory. It uses more RAM and is not persistent across a restart. Preview and final export still create video files.

Recommended: Use Disk + Keep in output while developing a video. Every completed part is saved, so a later interruption does not discard the earlier work.

New run

Select New run when starting a new video or changing its source material, prompts or timing plan. It resolves those inputs and begins a fresh run on each queue.

Parts per queue controls how much work happens before returning:

0 renders all remaining parts.
1 renders one new part for review.
A larger value renders that many new parts, or stops earlier when the plan is complete.

Important: Queueing New run again starts another run. To build on what you just generated, select its saved run and change the operation to Resume saved run.

Stop at a useful review point

Click Render loop · stop after current part → Stop after current part on the main canvas, below the render seed controls in Shared H3 render loop. It lets the current part finish, saves it, updates the accumulated preview and then stops the loop. You can use it without opening the subgraph or queueing again. The button inside Video Loop Start works too. For predictable step-by-step work, set Parts per queue to 1 before queueing instead.

The preview contains all retained parts so far, including the latest transition. In lip-sync mode, use it to judge the movement and audio around the warmup-trimmed join, rather than judging the new clip on its own.

An immediate ComfyUI cancellation can interrupt the current generation. Previously committed parts remain saved; the unfinished part must start again when resumed.

Resume an unfinished run

In Load Video Run, click Refresh runs and select the run you want to continue. Check Run progress + saved path or Saved run / part ranges if several runs look similar.
In Video Run Storage, select Resume saved run. Keep Disk + Keep in output enabled for the new result.
Keep the matching Lip-sync / Native H3 path, H3 model and LoRA setup selected.
Set Restart part = 0. This means continue after the last completed part.
Set Parts per queue to 1 for another review step, or 0 to finish the remaining plan.
Change Render seed · change before queueing if you want a new seed for the next generated parts, then queue.

Resume restores the saved prompts, timings, reference images, canvas and audio. It does not request another reference-analysis pass. Loaded model objects are not stored with the run, so the selected model, LoRAs and sampling settings still matter.

A completed run resumed with Restart part 0 returns the completed media without generating more parts.

Replace a bad part and the remaining ending

Suppose parts 1–6 look good, but part 7 has an unwanted gesture or appearance change:

Stop after the current part, or wait for the queue to finish.
Select that saved run and choose Resume saved run.
Set Restart part = 7. Part numbers start at 1; use the part-range report to find the correct number.
Change the Render seed before queueing. Leave Parts per queue = 1 to review the replacement first.
Queue. Parts 1–6 are retained as saved media; part 7 is generated again.
Review the accumulated preview, including the join from part 6 to the new part 7.
Select the newly created run, set Restart part back to 0, and resume to continue its ending. To try part 7 again, keep the restart value at 7 and choose another seed.

Persistent resume writes a new run folder and leaves the original run available for comparison. Before the next continuation, refresh the list and select the new revision you want to build on.

Replacement scope: Restart part N replaces N and everything after it. It does not splice one replacement into an otherwise preserved ending. A long scene can contain several render parts, so scene numbers and part numbers may differ.

Do not delete individual part files to force a restart. Use Restart part instead; deleting files can make the saved run incomplete or unloadable.

Correct the prompt for a specific part

If changing the seed keeps producing the same unwanted gesture, check the prompt itself. For example, conflicting hand positions can make a short scene difficult to generate.

Select the saved run in Load Video Run. The editor uses this same selection.
In Part Prompt Editor, click Read selected run. This reads the saved text and timing without loading a model or generating anything.
Choose the Render part by its number and time range. Edit the text, keeping one clear description of the pose and movement. For example: “Both hands already rest on the table. The subject nods once while singing to the supplied audio.”
Turn Apply edits on. Choose Resume saved run and set Restart part to the part you changed. Use the earliest changed part if you edited several completed parts. For upcoming parts in an unfinished run, leave Restart part at 0.
Set Parts per queue = 1, optionally change Render seed, and queue. Review the accumulated preview before continuing.

Full replacement: The editor shows the combined global and scene prompt for that render part. Your text replaces it completely; the conflicting original text is not added again. Keep the performance direction and any native H3 sound instructions you still want.

Saved corrections: The new revision keeps your edited prompts. Select that revision to continue, even with Apply edits off. The original run remains available for comparison.

The editor leaves timing, reference images, audio selection and lip-sync warmup as saved. It changes only the selected parts' prompts. Restart part still replaces that part and the remaining ending; the earlier parts are retained. Part numbers identify actual renders, so a long scene split into several parts can have just one of those prompts corrected.

Reset this part restores the text you read from the selected revision. Clear edits removes pending edits and turns Apply off; it does not change saved files. Reading an edited revision again removes pending edits already saved there. Clear drafts before reading a different run, or turn Apply off to resume that run without them.

Seeds and edits during resume

The Render seed is connected to generation independently of the upstream analysis and planning. Changing it before the next queue changes the newly rendered parts; it does not regenerate the retained beginning. Different seed per part varies the seed across part positions.

The current queue already has its settings. Editing the seed while it is running does not change the clips still being generated in that queue. Stop, adjust, and queue Resume again.

Saved-input rule: Resume restores the saved inputs and then applies enabled Part Prompt Editor changes. Editing the ordinary manual prompt boxes does not affect Resume. Use New run for a changed timing plan or new source material. Part Prompt Editor and seed changes take effect on the next Resume queue. Prompt Review's Continue applies edits during the waiting New run.

Load a run for captions or export

Select a run and choose Load saved run to reuse its media. This skips generation, analysis and fresh planning, so you can change caption styling or output settings and export again. The source audio loaders stay inactive; the saved run supplies its own recording.

Keep the whole run folder if you move or back it up. It contains the ordered parts, audio and input checkpoint. NumPy .npy files hold pixel or saved-input arrays; they are not independently playable videos. The manifest tells the loader how those files belong together.

To reuse a moved run, place its complete folder under ComfyUI's output directory and click Refresh runs. Older media-only runs can be loaded for export but need a render checkpoint to support Resume. Saved output runs remain until you remove them, so clean up unwanted revisions when you are finished.

Scene prompts and cut times

Each scene receives the global direction plus its own part prompt. A list of scene prompts is not sent as one long instruction to every clip.

Global prompt: the performance or visual direction that applies throughout.
Manual first part: the opening framing, pose and movement.
Manual next parts: one nonempty line per subsequent scene. Keep each description on one line; the workflow carries these as a list.
Manual cut times: comma-separated seconds from the start of the selected excerpt.

For a 40-second video, cut times 8, 23, 31 create scenes lasting 8, 15, 8 and 9 seconds. Use one first-part prompt and three next-part prompts.

In Native H3, blank cut times divide the duration equally between the supplied scene prompts. Explicit times, including measured reference-video times, take priority. For a single global performance, clear the cut times and all part-specific text.

Write compact generation directions. For example:

Global: The subject performs naturally to the supplied audio with synchronized lip movements.

First part: Medium close-up with a static camera. Both hands are already beside the temples; the index fingers make small circles while the eyes widen.

Next part: Tight close-up. The subject lowers their chin slightly, holds eye contact and gives a brief sideways smile.

There is no need to write clip timestamps, warmup instructions or audio-priority rules into these paragraphs. The workflow handles timing and audio selection. A gesture already in progress should be described that way, especially when the scene is short.

Review and edit prompts before generation

Prompt Review, titled First + next scene prompts · review, sits after the live/manual prompt selection. It receives the opening prompt separately and the remaining prompts as a list, so both sources can be reviewed in the same place. The global direction remains in its own controls; change it before queueing.

Pause is off by default. The node displays the received text and passes it through unchanged. Saved drafts do not apply in this mode. To edit the prompts before a New run starts rendering:

Choose your live or manual source, then enable ⏸ Pause on Prompt Review before queueing.
Queue New run. When the prompts arrive, execution waits at the review node.
Select First prompt in the dropdown to inspect the opening scene. Use the ◀ / ▶ arrows beside the dropdown to move through later prompts. Switching entries keeps your edits; the dropdown and buttons use the selected Eclipse Nova design theme.
Edit each scene in place. Keep one paragraph per prompt, because the H3 planner uses one line per scene when the list is joined. Correct the action, overall pose and framing; remove contradictory or unsupported details. Edit the existing item rather than deleting it, so scene order stays intact.
Press ▶ Continue once to pass all reviewed prompts to planning and generation in this same run. The analysis or manual source is not rerun, and no copy/paste into the manual fields is required.

Reset this prompt restores the selected incoming text. ■ Cancel run cancels the waiting run without passing partial edits downstream. ComfyUI's interrupt control also works. While paused, the queue remains occupied and already loaded models remain loaded.

Saving the workflow retains the editor text. With Pause on, drafts can prefill another review only when all incoming prompts and their order still match; you must press Continue again. Changed input starts a fresh review. Refresh status reconnects to a waiting review in the same browser session after reopening the saved workflow. A nonempty prompt cannot be cleared completely.

The workflow routes reviewed opening and later prompts into both the lip-sync and native H3 paths. It keeps cut times and global direction on their existing routes. Reviewed prompts become part of the resolved inputs saved with the new run; the earlier saved scene-analysis file still contains the original analysis. Raw analysis previews may therefore differ from the edited text in Prompt Review.

Which review control should I use?

ControlWhen to use itWhat happens nextPrompt Review → Pause / ContinueInspect scene prompts before a New run rendersContinue uses your edits in the waiting queue, without rerunning analysisRender loop → Stop after current partInspect generated video and its latest joinThe current part finishes and is saved; select its run and queue Resume to continuePart Prompt Editor → Apply editsCorrect a particular render part of a saved runQueue Resume; restart at or before an edited completed part

Resume and Load use saved inputs and skip the New-run prompt-review route. Prompt Review edits scene text before global direction is combined; Part Prompt Editor shows the complete combined prompt for each render part. A scene split into several render parts shares its reviewed scene description initially, but those saved parts can later be edited individually.

Use a reference video for cuts and movement

Enable Reference video, load the clip and choose the excerpt you want. Enable Live analysis to measure its scene boundaries and describe the action inside each scene.

For a lip-sync motion-transfer example, keep your character image and recording selected, choose Scene Motion Analysis in Smart LM and Exclude source appearance in Scene Analysis to H3. Enable Prompt Review's Pause if you want to correct the descriptions before rendering. This task choice describes motion; the Lip-sync / Native H3 switch still chooses the generation path separately.

The workflow uses two separate passes:

Scene Detector scans the source frames for visual scene changes on CPU. 128 px longest side is a starting point for detecting cuts.
Smart LM describes only the sampled frames from each scene or window. Its separate Image handling → Longest side controls description detail. Start with 512 px and inspect the result; a larger limit can preserve more facial or hand detail, with additional memory and processing cost.

An example analysis plan selects 8 frames per scene, a maximum of 8 per window, and 2 overlapping samples when more than one window is needed. Increase samples per scene when brief gestures are missing, while keeping the per-window limit manageable. Raising only the window limit does not select more frames. Together: On lets Smart LM compare each window's selected frames in chronological order and produce one scene description.

Inside MMH3 Reference Analysis, Scene Analysis Plan creates those windows. The loop sends each one through Scene Frame Sampler and Smart LM Loader, with the frame context carried by Smart LM Prompt Pipe. Scene Description Collector gathers the results in order, Save Scene Analysis stores them, and Scene Analysis to H3 produces cut times plus the separate first/next prompts. Prompt Review receives the selected prompts after that conversion.

Every Nth frame in Scene Analysis Plan samples original video frames. In Smart LM it subsamples the images it receives. Usually leave the LM stride at 1 if you already reduced the selection in the plan. Longest side 0 uses the loader's automatic model limits; a positive size is still bounded by supported limits.

Scene Cut Limit controls how dense the editing can be. 0 keeps all detected cuts; a positive value sets the maximum cuts per second. 2 requires at least half a second between retained boundaries, while 1 requires one second. A stricter positive limit merges some scenes without moving the retained cuts, so it also changes what each description must cover.

Disk frames, Together and scene coverage

The sampler supplies frames through Smart LM's frame_source socket. These are the selected source images loaded for the active request. Every task that supports image input can use this source, including ordinary descriptions and image-capable MiniMax H3 tasks. Connect images or frame_source, never both. The chosen model still needs vision support.

Together on: one request for the ordered selection. Use this for scene action and changes between frames. Explicit Together uses the supplied selection without the older Max frames tail limit.
Together off: one request per supplied image. This is useful for individual captions, but separate stills give the model no cross-frame motion to compare. Florence and WD14 use individual processing for multiple images.
Text tasks: keep their text instructions. A supported vision model can use a supplied image as context, but changing to disk input does not turn a text task into scene analysis. MiniMax H3 T2VA Timeline remains text-only.

Together sends the selected samples after the LM stride, rather than every decoded video frame. With stride 1, the selection includes the window's first and last samples. The dedicated Scene Motion Analysis and Scene Description (T2V) tasks remove exactly black samples; an entirely black selection keeps an explicit Black screen. entry instead of losing a scene. Other tasks keep their own image behavior.

The opening prompt has its own output. Scene Analysis to H3 returns first_part_prompt for scene 1 and next_part_prompts starting at scene 2. Looking only at the next-part preview can make the opening scene seem missing. Prompt Review shows the opening as First prompt, followed by the later scenes. The planner and saved-run part report show what will actually be rendered.

Joining three source clips does not imply three final scenes: detection can find additional cuts within each clip. Sampling windows describe portions of a scene, while render limits can split one scene into several generated parts. Use the measured frame/time ranges when comparing source clips, scenes and render parts.

Choose the description task for your purpose

Smart LM taskWhat it describesScene Analysis to H3 settingScene JSON buttonScene Motion AnalysisTransferable camera framing, pose, action, gaze and expression; the target image supplies appearance and settingExclude source appearanceNot required; the task already returns structured scene observationsScene Description (T2V)Visible subjects, appearance, clothing, setting, lighting and motionInclude source appearanceNot required; the task already returns structured scene observationsSimple / Detailed / Ultra Detailed DescriptionThe task's ordinary scene caption, at the requested level of detailInclude source appearanceOn when connected to the scene collectorMiniMax H3 Scene or an image-capable H3 Timeline taskH3-style prompting from the ordered scene samples when Together is onInclude source appearanceOn when connected to the scene collector

For lip-sync with a separate character image, use the first row as the starting point. Scene Motion Analysis requests concise performance direction and omits source appearance throughout its output. Scene Description (T2V) includes that appearance in its generation paragraph. Neither scene task should infer audio or unseen events from still frames. Both distinguish positions already present in the opening frame from movement actually observed between samples.

The appearance dropdown must match the task. It does not rewrite a full caption into a motion-only description. Switching from motion-only to full descriptions requires another analysis run to obtain the omitted appearance. Keep the task consistent throughout one analysis loop.

Ordinary Simple, Detailed and Ultra Detailed tasks cover action, visible people and their appearance, overall pose, camera angle, background and lighting. Simple is concise; Detailed develops the scene; Ultra Detailed asks for fuller connected paragraphs with distinct visible details. Both Standard and NSFW provide examples for all three levels. Ultra Detailed is a request for more supported detail, not a minimum length or a guarantee of greater accuracy. Natural poses such as “seated with legs crossed” are usually clearer than a forced inventory of every limb. Unshown or uncertain details should be omitted.

With Together on, image-capable H3 tasks use the whole selected sequence for one scene description. Intermediate samples remain available, and first/last reference roles retain their endpoints. An empty user prompt in this scene-analysis route does not request a new invented story. Normal single-reference/text prompting outside this route retains the H3 task's own behavior.

Scene JSON: ordinary text versus collector input

Scene JSON defaults to off in Smart LM's Mode bar. Keep it off when an ordinary task's text goes directly to a prompt consumer. Enable it when ordinary image descriptions or H3 output feed Scene Description Collector.

With Scene JSON on, SmartLLM builds the JSON container in code around the task's normal answer. The model keeps its usual instructions and does not need to format a new JSON schema; packaging adds no inference call. The container preserves the original answer and supplies the scene paragraph and IDs required by the collector. For H3 responses, the visual-description field becomes the scene prompt; its soundscape/music fields remain in the original answer, and audio selection stays with the workflow.

Scene Motion Analysis and Scene Description (T2V) already request structured observations. Their format stays the same with Scene JSON on or off. SmartLLM accepts supported wrappers and simple field variations, and can make one correction attempt for invalid structure. A persistent structural failure still reports its scene/window and saves the response for inspection. A format error from an ordinary task with Scene JSON off is therefore different from a truncated structured response.

Usable structured descriptions with content review warnings remain available for editing. JSON packaging keeps ordinary text from breaking the container; it cannot fix a wrong pose or invented action. Review the words before using them for generation.

Training profiles and Thinking

Standard / NSFW selects the node's system instructions independently of Training. NSFW selects the NSFW prompt file even with Training off. When Training is on, SmartLLM also supplies that profile's few-shot examples. These are examples in the request, not weight training. Native ComfyUI text encoders receive them as demonstrations in the system instructions; other supported backends use their prompt paths. A custom system prompt takes precedence over the profile's system instructions.

The NSFW description examples retain full scene coverage and use direct anatomical terminology for visible details. The same visibility rule applies: omit unsupported content and statements about unseen body parts. Selecting this profile does not change the model's capabilities or ensure an accurate caption. Customized prompt/example entries are preserved when bundled defaults update, so local wording can differ from the guide.

Thinking defaults to off. Leave it off for a straightforward lip-sync description pass. Enable it when a supported model/template offers reasoning and you want to compare its results. It appears only on supported backend paths; an uncensored model label does not establish whether thinking can be switched.

Max tokens includes reasoning and the final answer. Thinking can therefore increase runtime or leave too little space for a usable description. The token progress bar measures the output allowance, not a reliable time-to-completion estimate, and the model may stop well before 100%. Native Qwen3/Qwen3.5 Simple/Detailed/Ultra Detailed captions have a bounded thinking allowance; that limit does not apply to every task or backend. If a scene run spends too long thinking, turn Thinking off before the next queue rather than assuming a larger token limit will improve the description.

Correct an analysis result

Review before rendering: A vision model can invent movement, miss a brief gesture or include unwanted appearance. Compare questionable scenes with the source. Use Prompt Review to correct the selected first and later prompts before Continue.

Live analysis has priority while enabled. Keep it enabled when editing its text through Prompt Review. For a fully manual plan or changed cut times, copy the complete cut/first/next set into the manual fields, edit it and turn Live analysis off. The reference-video group can stay on to supply its timing and audio. Changing the plan or source material requires New run; saved-part text corrections use Part Prompt Editor on Resume.

The analysis group saves its original results under output/scene_analysis/ for inspection or later reuse. Its reports include scene/window information and available sampled-frame evidence. Prompt Review changes the downstream prompts, while the saved analysis remains available to compare with those edits.

Reference images for native H3 scenes

Load Image Timeline provides one place to load, order and crop images to the shared canvas. Choose the assignment in Video loop · saved inputs + resume → reference_mode:

text_only — generate without using the loaded images.
first_only — use the first enabled image at the start of every clip.
last_only — use the last enabled image at the end of every clip.
first_and_last — use the first and last enabled images as endpoints. With one image, only the start is anchored.
first_per_scene — image 1 starts scene 1, image 2 starts scene 2, and so on. If there are more scenes than images, reuse the last image.
adjacent_pairs — scene 1 uses images 1→2, scene 2 uses 2→3, and so on. Once no next image remains, the last image supplies a start only.
reference_images — use up to nine visual references without fixing the opening or ending frame. Select a matching Ref2VA model; prompt tags such as <Picture 1> refer to enabled image order.

For three different scene starters, load three images, select first_per_scene, and define three scenes through cuts or scene prompts. Image slots do not create scene cuts by themselves. If one scene is split because of the render limit, its extra parts retain that scene's image assignment.

About fades: Image-timeline blending is not a crossfade between the final generated clips. The H3 assembly joins the planned parts. Lip-sync warmup helps prepare the next part; it is not a visible dissolve. This workflow does not provide a final clip-crossfade control.

Audio selection and timing

The shared audio choice follows this priority:

Audio override — an explicitly selected recording, or audio supplied by an enabled music-generation group.
Reference-video audio — when no override is supplied and the selected video has usable, unmuted audio.
H3-generated audio — in Native H3 when neither source supplies a recording.

A reference video without audio can still supply the duration and scene timing. With neither audio nor a video, Native H3 uses the planner's Duration. A silent but present audio track still counts as supplied audio; mute that source if you want H3-generated sound instead.

Lip-sync needs a recording and reference image. LS vocal isolation can guide the performance with separated vocals while keeping the selected full recording as the final soundtrack.

When using reference cuts with a separate audio file, align their excerpts yourself. Cut times are relative to the selected start; they are not automatically retimed to another song or tempo.

Optional music generation

Enable either YuE2 or MiniMax Music 3 when you want to create the recording first. Enable Audio override and set the full-song Load Audio source to Auto to receive generated audio. For your own recording, use Selected file instead. Keep only your intended generator active so the audio source is unambiguous.

YuE2

Select the YuE2 checkpoint. With Auto Prompt off, write musical direction in Style and the sung words in Lyrics. With Auto Prompt on, describe the song in Smart LM; its selected text supplies the generator instead of those manual fields.

Enable ABC planning adds a musical-score planning pass before music generation. Smart LM prepares the style and lyrics; YuE2's native ABC generator creates the score. Leave both native generators in full mode for a normal song test.

Set the generation duration deliberately; 360 seconds is a full six-minute song, so use a shorter duration for testing. The group saves the full song as FLAC; the shared audio controls then select the excerpt for H3.

MiniMax Music 3

Choose its model, text encoder and audio VAE. With Auto Prompt off, use Caption for musical direction and Lyrics for sung text. With it on, edit the Smart LM request instead. A duration of 120 seconds generates a two-minute song; use a shorter duration for a quick test. This group also saves its full song as FLAC before the H3 excerpt is selected.

Time-saving tip: Reduce the music generator's duration when testing. Selecting a short H3 excerpt afterward still requires generating the full requested song first.

Optional lyrics and captions

Enable lyric extraction when you need to transcribe an existing recording. Review the transcript and correct the wording before using it as lyrics. For generated songs, the generator's selected lyrics can supply the caption text; manual full-song lyrics and correction fields are also available.

Automatic transcription and lyric alignment use the local Whisper large-v3 conversion. Download these five files from the supported Systran revision into ComfyUI/models/whisper/large-v3/:

model.bin
config.json
tokenizer.json
vocabulary.json
preprocessor_config.json

These files are checked against that revision. Qwen3 · compare only is an optional alignment comparison. It needs Eclipse's additional Qwen dependencies and model setup; select Whisper for a first test without that extra pass. Follow the lyric-caption setup guide for installation details.

Enable lyric captions, choose a font and adjust the style. Review Caption alignment report, Clip-relative SRT, Full-song timing JSON and Cleaned lyrics. If a word was not aligned, check the report before adjusting its timing manually.

The run saves the complete caption recording and the selected excerpt start. Captions use that saved pair, so changing the current source controls cannot trim a loaded run again. With H3-generated sound, the workflow reads the audio from the assembled video; transcription then waits until that sound is available. Demucs is optional.

Use Load saved run while changing caption appearance or re-exporting an already generated video. Leave captions off for a clean video. Older runs saved before caption recording/offset support use their saved soundtrack from zero; any manual timing JSON for those runs must match that recording.

Common fixes

The wrong model runs after switching mode: select the matching loader preset and the separate LoRA preset as well as the generation path.
A downloaded file still appears missing: select its real local filename. Provider names and your example subfolders can differ; a browser reload also clears pending automatic download assignments.
The first prompt seems missing: select First prompt in Prompt Review. The next-part output starts at scene 2 by design; check the planner and saved-run part ranges before assuming source frames were skipped.
The first scene keeps repeating: inspect the selected prompts in Prompt Review. Each next scene needs its own nonempty item, and Live analysis overrides the manual source while enabled.
Clothing from the reference video appears: use Scene Motion Analysis plus Exclude source appearance, then correct unwanted details in Prompt Review. Check that your intended reference-image mode is enabled.
The collector reports “Expected scene JSON”: for ordinary description/H3 tasks, enable Scene JSON and use Together on. Increasing Max tokens does not turn ordinary prose into collector input. For dedicated structured scene tasks, inspect the saved response and address truncation or the reported missing field.
Smart LM stops after thinking without an answer: turn Thinking off for the next description run, or allow more Max tokens if you deliberately want reasoning. Both share the output budget. A higher limit can also make the run longer.
Prompt Review edits are ignored: enable Pause before queueing, edit when it is waiting and press Continue. Pause off always passes the incoming text through, including when the workflow contains saved drafts.
The queue waits at Prompt Review: press Continue or Cancel run. This is the selected pause, not a stalled model load. Use Refresh status to reconnect the editor, or ComfyUI's interrupt control if it is unavailable.
The run list is empty: generate with Disk + Keep in output, then Refresh runs. Keep the complete saved folder under ComfyUI output.
Resume makes no new video: a completed run with Restart part 0 is already finished. Set the part number to replace, or select an unfinished run.
A replacement looks the same: change Render seed before queueing; changing it during execution does not change that queued job.
Prompt edits do not affect Resume: use Part Prompt Editor, click Read selected run, edit the numbered part and enable Apply edits. Restart at or before a changed completed part. The ordinary manual prompt boxes apply to New run.
Memory runs out: use Disk, reduce generation resolution, and keep analysis windows bounded. Disk storage reduces accumulated decoded-frame memory; it does not remove the model's own VRAM requirements.

Further documentation

Eclipse Nova README — installation and the full node documentation index.
Smart Model Loader Nova README — model loading, presets, LoRA Stack and downloads.
SmartLLM Nova README — vision models, backends, prompting tasks and image handling.
Eclipse saved runs and resume — storage lifetimes and recovery details.
Eclipse native H3 clips — image assignment, duration and audio behavior.
Eclipse scene analysis — detection, frame sampling and description controls.
Eclipse Prompt Review — pause, direct editing, navigation and same-run continuation.
Smart LM Loader guide — task choices, disk-frame input, Scene JSON, training profiles and Thinking.