Score to Screen — Songs, voices and scenes: a music video studio (YuE2 × MiniMax H3)
Write one prompt and pick your images. You get an original song made with YuE2 and a music video made with MiniMax H3. The singer lip-syncs, the band plays along with the music, and every scene change lands exactly on a bar line.
🆕 v9.1 — a list for the lines. Each scene on the 🎞 Timeline has a new 🗣 Lines box: one row per line, with who says it, the words and, if you like, the second it starts. H3 gets the lines as you entered them: the same words, the same speaker, the same order, each in a close-up of its own.
🔊 v9.0 — the sound of every scene. Until now the song played under the whole video. In v9.0 every scene chooses its sound: 🎵 the song, 🎬 H3's own sound (the people speak their lines, with the sound of the place under the voices), 🎛 the song with the place's ambience under it, or 🔇 silence. The song keeps running behind a talking or silent scene and comes back further on. Plus 🎚 master volumes for the song and for H3's sound. The cover video is one 6-scene run: a talk on a train, the song, a dance in the aisle, song + ambience, a silent end.
🎬 v8.2 — a major update since v7.0. 🎬 Up to 64 scenes, cut on the song's own changes and written scene by scene by the LLM. 🎬 A Scene Director that tells each scene's LLM who of your cast plays what, and when, so the camera follows the band. 🧾 The H3 prompt in MiniMax's official six-section Ref2VA shape. 🚀 Three H3 engines: Standard, ⚡ Turbo and FastH3 Omni (8 steps, about 4× faster on my PC). And a band that behaves: the right person in each shot, only the singer singing, the main character in front, the drummer swinging big with the kicks timed and solos played hit by hit.
🎞 v7.0: one screen for the whole MV. The 🎞 Timeline gets a left column with five tabs (🎬 Main, 🎵 Song, 🎚 Sound, ⚙ Settings, 🔍 Details) and can fill the whole window: 🎭 a cast of up to four people with their pictures, picked per scene, 🎵 the song in its own field, 🎚 the finished MV's sound (EQ, reverb, width, fades) with 🔁 remaking only the sound, and ⚙ models, LoRAs and VAEs in one place. Plus 🎸🎻 the hands play along: the fretting hand changes chord shape on the score's chord changes, a violinist's bow and fingers follow the note lengths, and the camera moves in to the fingers when a solo starts.
v9.1 Simple: everything in v9.0, plus 🗣 the Lines box (speaker, words and start second for every spoken line, entered as a list on the Timeline), ⚠ a warning and a one-click fix when a scene with lines is not set to 🎬 H3's sound
v9.0 Simple: everything in v8.2, plus 🔊 each scene's sound (🎵 song / 🎛 song + ambience / 🎬 H3's sound with spoken lines / 🔇 silent), 🗣 Japanese lines read in hiragana, 🚃 the place's sound raised under the voices, 🎚 master volumes for the song and H3, 🔁 remake that knows each scene's sound
v8.2 Simple: 🎬 1–64 scenes cut on the song's changes, the LLM writing every scene from 🎵 a music brief and 🎬 the Scene Director (who plays what, when), ✋ cuts set by hand, 🧾 MiniMax's official Ref2VA prompt shape, 🚀 three H3 engines (Standard / ⚡ Turbo / FastH3 Omni), 🎭 a 4-person band without mix-ups and 🌟 the main character in front, 🎤 only the singer sings (and an instrumental keeps every mouth closed), 🥁 big stick swings, timed kicks with independent feet and drum solos hit by hit; plus everything from v7.0: 🎞 one screen, 🎭 the cast, 🎚 the MV's sound, 🎸🎻 the hands, 🔒 Audio Lock, 🔁 remake only some scenes, 🔬 sync check
Older versions on this page: v7.0 Simple, v6.2 Simple, v6.1 Simple, v6.0 Simple, v5.9 Simple, v4.1 Simple (1–4 scenes, LM Studio) and v3.9 Simple (2 scenes)
Each version comes in ONE download: the workflows (English + Japanese) and the custom nodes. Only a few switches; everything else is automatic, or set to the values that worked best.
New in v9.1 (2026-11-18): spoken lines get a place of their own. In v9.0 you wrote them in quotes inside the scene's words and the workflow had to guess whose line it was, whether it was a line or what a sign reads, and in which order. Now the scene's 🗣 Lines box holds them as a list: the speaker (one of your cast, or a name you type in any language), the words, and an optional start second. Whatever the LLM does with the scene, H3 gets the list as entered, each line in a shot of its own, close on its speaker. A 🎵 scene with lines shows a ⚠ and a button that makes it a 🎬 scene. A scene with an empty list works as in v9.0.
New in v9.0 (2026-11-14), a major update since v8.2: every scene chooses its sound on the 🎞 Timeline. 🎬 H3's sound makes a scene without the song: write a line in quotes and the person says it, with the sound of the place (raised to 14 dB under the voices) around it; Japanese lines go to H3 in hiragana so they are read right. 🎛 Song + ambience keeps the song and lays H3's sound of the place under it (a second, sound-only take). 🔇 Silent is silent. The song runs on behind those scenes and returns later, untouched to the sample. 🎚 Master volumes for 🎵 the song and 🎬 H3, each 🎬 scene first brought to the song's loudness. From my real runs: said is not sung, the scene after an intro sings again, a copied "scene before" is asked again. See 🆕 WHAT'S NEW IN v9.0.
New in v8.2 (2026-11-07), a major update since v7.0: the scenes are no longer limited to four. 🎬 Up to 64 scenes (24 in v8.0), one picture each (+ on the Timeline), cut automatically on the song's changes and always on a bar line; drag a cut by hand to put a picture on the singing coming in. Every scene's LLM gets 🎵 a music brief (tempo, song parts, singing, energy, how the scene before ended) and 🎬 the Scene Director's list of who plays what and when, and writes its own shots. 🧾 prompt_format = ref2va_structured pours it all into MiniMax's official six sections. 🚀 Pick Standard (20 steps), ⚡ Turbo or FastH3 Omni (8 steps, no LoRA). From real 19-scene band runs: 🎭 actions given to the wrong member, a second drummer, "her" with four women on screen and the bassist singing are fixed before H3 sees them; 🌟 about two of every three shots frame the main character and a close-up stays on its person; 🥁 the drummer swings big, the right foot kicks on the real kick times while the left stays on the hi-hat pedal, and a drum solo is written hit by hit. See 🆕 WHAT'S NEW SINCE v7.0.
New in v7.0 (2026-10-26): one screen. 🎞 The Timeline gets a left column with five tabs: 🎬 Main (🎭 the cast), 🎵 Song (the song's description, on-screen singing, a pre-made song, song only, the YuE2 model), 🎚 Sound (the finished MV's sound), ⚙ Settings (⚡ Turbo and its steps, the Turbo LoRA, the H3 model, ➕ more LoRAs, the text encoder, the video and audio VAEs, 🎸🎻 playing & camera, 🔒) and 🔍 Details (each scene's prompt as H3 really got it, its seed, steps, sampler, CFG, model and LoRAs, and the final video's Civitai metadata, ready to copy). ⛶ It can fill the whole window (Esc to go back), opens by itself, and says in one line what the next run will do, with anything that would make the run stop shown first (the panel's ▶ Run refuses such a run). 🎭 A main character (S1) and three more (S2–S4), each with a picture, a name and a part, are given to H3 as reference pictures only in the scenes they are in, with "S2 is Ken, the person shown in <Picture 3>, who plays the drums." in the prompt, so the drummer stays the drummer. 🎚 EQ, reverb, stereo width and fades on the finished video's sound only, and 🔁 "🎚 sound only" joins a kept run again with the new sound (no H3). Also since v6.2: 🎸🎻 the hands play along (see 🎸🎻 THE HANDS), ⚡ one Turbo steps setting for every scene. See 🎞 THE NEW SCREEN IN v7.0.
🆕 WHAT'S NEW IN v9.1
🗣 The Lines box: select a scene on the 🎞 Timeline; under "This scene's sound" there is a new box, 🗣 Lines. + Add a line adds a row with the speaker, the line and the second it starts (up to 12 lines a scene). ▲ ▼ change the order, ✕ deletes
Speaker: one of your registered cast (S1–S4), or ✎ type a name in any language (
the hawk,鷹). A typed name is put into English the way that scene's cast is called, by the workflow's own LLM; 🔍 Details shows itStarts at: empty = placed by itself, in order, with the time the line takes to say. With a second, the line starts then and the others make room
As entered: if the LLM leaves a line out, rewords it, gives it to somebody else or changes the order, H3 still gets your list. A line the LLM gave no shot gets one of its own, close on its speaker
⚠ Only a 🎬 scene speaks: lines in a 🎵 / 🎛 / 🔇 scene show a warning and a Make it 🎬 H3's sound button (in v9.0 they were dropped without a word)
The old way still works: a scene with an empty list reads the quotes in its words as in v9.0. In a scene with a line in the list, put every line in the list
Checked in a real run (a hamster, a hawk, three 🎬 scenes, four lines): each line spoken by its own speaker in its own close-up. Not yet tried on H3: a set start second, and speakers picked from a registered cast
🆕 WHAT'S NEW IN v9.0
🔊 Each scene's sound: the scene box on the 🎞 Timeline has a new row, "This scene's sound": 🎵 Song (as always), 🎛 Song + ambience, 🎬 H3's sound and 🔇 Silent. The clips carry 🔊 / 🎛 / 🔇 so you see the whole plan at a glance
🎬 H3's sound: the people speak: the song stops for that scene and H3 makes the sound of the place and the voices. Write a line in quotes in the scene's words (
she says: "We'll be there soon, won't we?") and it is spoken as written, in the shot where it is said. "They talk" without a script works too🗣 Japanese lines in hiragana: H3 misread kanji in my tests (
曲が止まったねwas spoken as "ki ga tomatta ne"), so every quoted Japanese line with a kanji is rewritten in hiragana by the workflow's own LLM before H3 gets it. Lines in English are spoken in English🚃 The place under the voices: in a talking scene H3 makes the place far quieter than the voices (in my train scene: 37 dB under them, "no sound but the talking"). The quiet sound between the lines is raised to 14 dB under the voices and carried on under each line, so the train does not stop when someone speaks. No extra H3 run
🎛 Song + ambience: the song plays as it is and H3's sound of the place (an audience, wind, rail joints) lies under it, 14 dB under the song by default. It takes two H3 runs of that scene: the picture on the song (lip sync and playing stay exact), then a sound-only take, so such a scene takes about twice the time
🔇 Silent: nothing is heard in that scene (real digital silence)
🎵 The song keeps its place: behind a 🎬 / 🔇 scene the song runs on and comes back further on, so the video is as long as before and no cut moves. The 🎵 scenes' sound is the song's own, to the sample
🎚 Master volumes: 🎵 Song, 🎬 H3 and 🎛 Ambience depth on the 🎚 Sound tab. Each 🎬 scene's sound is first brought to the song's loudness (BS.1770), then your volume is added. What H3 hears never changes
💾 The finished video is encoded once (thanks to bno84): it used to be encoded at CRF 12 and again by the save node at CRF 23, with the sound at about 128 kbit/s. Now the join's file is the finished video and the save node copies it. 🎚 Sound, 💾 The finished file: 🎞 picture quality (CRF 18 by default) and 🔊 sound quality (AAC 256 kbit/s by default)
🔁 Remake knows the sound: the sound each scene was made with is kept with the run. A kept scene that becomes 🎬 is made again; changing to 🔇 or 🎵, or only moving a volume, reuses the kept scenes
✍ Scene prompts, from real runs: a spoken line may talk about singing (
"Shall we try singing that song?") without being dropped as "music"; the scene after an intro sings again; when a small LLM copies the scene before instead of your words it is asked once more (🔍 Details says so); the scene-words box is bigger; the ✋ notes are in Japanese in the Japanese workflow
🆕 WHAT'S NEW SINCE v7.0 (v7.1 → v8.2)
🎬 Up to 64 scenes (v8.0: 24, now 64): + on the 🎞 Timeline (or drop a picture on "+ Add a scene"), ◀ ▶ to move a scene, 🗑 to delete it. The cuts go on the song's changes (intro, verse, chorus ...), always on a bar line, 15 s at most per scene. One H3 set per scene is built when the run starts, so the graph stays small. A whole 3-minute song is about 12–19 scenes. 🎭 a sub member can be in any scenes (e.g. 3,5-8)
⏱ Every scene's own seconds (v8.2): drag a clip's right edge and that scene gets longer or shorter, the scenes after it move along (as in 🎞 H3 Timeline); Alt + drag moves only the cut (✋); or type the seconds; it snaps to the singing entries, the song parts, bars and beats (Shift = free); ↺ back to automatic
✂ Editing the scenes (v8.2): select a clip, then the buttons in the scene's box or a right click on the clip — add after, duplicate, split at the playhead (the second half continues with ⛓; the two halves are exactly the old scene), delete. Shift / Ctrl + click selects several; Delete takes their pictures off
💾 More scenes cost disk, not RAM (v8.2):
decode = streamon 🎬 Make every scene writes each 17-frame chunk of H3's video VAE straight to the scene's file — one chunk in RAM instead of one scene (about 0.2 GB instead of 3.2 GB for 15 s at 768×1024), the same pixels. Each scene's latent is kept in a small file next to its frames🎬 The whole video's idea is on the Timeline now. Until this update that prompt was a box on the canvas only, and the workflow shipped with a sample in it ("She picks up the Stratocaster … No singing.") that ran for every scene even when you worked from the Timeline alone. The box is shown next to each scene's words, ships empty, and a scene's own words come first where they differ. If you use a workflow saved from an earlier version, open the Timeline and clear or rewrite that box.
💃 Dancing, and dancing while singing: the cast part
danceshas a partner,dances + sings. She waits with closed lips until the vocals come in and sings in lip sync like any singer, and every dancer gets a line per shot: "dances on the beat of the song, every step and turn in time with the music" (not in close-ups). It follows the beat; it is not choreography beat by beat, and H3 decides the moves. The cast card decides who sings, and a cast in which nobody plays gets no instrument cues.👀 A look for each face (v8.2):
lookon each cast member — hair and clothes in a few English words, "long silver hair, black hoodie" — goes into H3's definition line, so the faces of a 4-person band stay apart in close-ups🎵 Music brief + 🎬 Scene Director (v8.0 / v8.1): before each scene's LLM writes its shots it is told the tempo, the song parts, where the singing is, the energy and how the scene before ended, and for every person of the cast when they really play (drums, keys, strings, the singing from the score). Each cast card has a role (auto / lead / rhythm / backing / rest)
🧾 Official prompt shape (v8.2): prompt_format = ref2va_structured writes MiniMax's six Ref2VA sections (subject_definitions / summary / retention_analysis / detailed_description / overall_soundscape / non_diegetic_music), with a <Subject n> and a retention line for every person. shot_list is still there
🚀 Three H3 engines (v7.1): Standard (H3 ref2va, 20 steps), ⚡ Turbo (4-step LoRA) and FastH3 Omni (8 steps, VSA sparse attention, no LoRA). On an RTX 5060 Ti 16 GB (OCuLink) one 13.8-second scene took 2000 s / 458 s with Standard / FastH3 Omni NVFP4
🎭 The band, from real runs (v8.2): an action the LLM gave to the wrong member ("the bassist keeps striking cymbal") becomes their own instrument, in the definitions and in the shots; a resting player is never made to play; a second drummer made up by the LLM becomes the cast's drummer; "her / she" with several people on screen is named from the instrument or the singing in that sentence
🌟 The main character in front (v8.2): the LLM is asked to frame the main character in about two of every three shots; a medium shot on the main character puts the band behind, out of focus; a close-up keeps only its person's lines and stays on them to the shot's end; later shots keep the same place and the same people
🎤 Who sings (v8.2): the score decides whether the song is sung; "Only S1 sings; the others keep their mouths closed" goes into the definitions and the retention lines; an instrumental keeps every mouth closed
🥁 The drums (v8.2): 🥁 stick_swing = big (a full swing from above the head); 🦶 feet = timed (the right foot on the real kick times, the left flat on the hi-hat pedal) or hide in wide shots; a drum solo with no steady pulse is written hit by hit with its silences; the drummer stays in the background of someone else's close-up
⚙ New defaults: adaptive drum cues, big swings, FastH3 Omni 8 steps, int8 video VAE, no Turbo LoRA, structured Ref2VA prompt
🎞 THE NEW SCREEN IN v7.0
The 🎞 Timeline is now where the whole MV is made. It is still only a second view of the workflow's own nodes, so everything can also be set on the canvas as before.
🎬 Main: the cast. 🧑 Main (S1, every scene) and 👤 Sub 1–3 (S2–S4): a picture (click or drop), a name, a part (sings, guitar, guitar + sings, bass, drums, keyboard, piano, violin, cello, dances ...) and a note for the LLM. Subs: the scenes they are in (1–64, e.g. 3,5-8) (or the chips under the selected scene's text). The clips show the faces of who is in each scene, and the scene's line shows each person's picture number ("S1・Aya・🎻 violin・🖼 2")
🎵 Song: the song's description (mood, genre, tempo, "Instrumental.", "Lyrics in English."), put in front of scene 1's text for the LLM; 🎤 does the person on screen sing; 🎵 a pre-made song and ▶ where it starts; 📜 its ABC score (the .abc with the song's name by itself, or pick / paste another; one line says which is used); 🎵 song only; 📦 the YuE2 model
🎚 Sound: the finished MV's sound, never what H3 hears: 🔊 even loudness (LUFS target), 🎛 EQ low / mid / high (±12 dB), 🏛 reverb, ↔ stereo width, 🌅 fade in / out. The loudness is matched again after the EQ and the reverb, peaks stay under -1 dBFS, nothing moves in time
⚙ Settings: ⚡ Turbo on / off and steps, ⚡ Turbo LoRA and strength, 📦 H3 model, ➕ LoRA 1–3 (every scene, Turbo on and off), 📝 text encoder, 🎞 video VAE and 🔊 audio VAE (each list shows only its own kind), 🎸🎻 playing & camera, 🥁 drum cue style, 🔒 Audio Lock
🔍 Details: for each scene of a run, the prompt exactly as H3 got it (with the cast lines, the timing cues and ⛓), its seed, steps, sampler, scheduler, denoise, CFG, model and LoRAs, and 💾 the final video's Civitai metadata (fields, prompt, negative, the whole text), each with 📋 Copy. Pick a recent run in the list; ⤢ Wide reads it over the whole panel
⛶ Full screen with its own ▶ Run; "🗗 Back down" or Esc for the bottom panel. The left column's width can be dragged, and ⏳ a progress strip shows the node working now, its steps, the time, H3's live preview and ■ Stop while a run goes
▶ Next run: every scene / one scene again / 🎚 sound only / song only, the scenes and seconds, Turbo, the cast, 🔒, 🥁. In orange first: a VAE of the wrong kind, 🔁 without a scene, 🎚 sound only without a run
🔁 Sound only: pick the run in the 🔁 strip, "⤵ Load that run's settings", "🎚 sound only", Run. Every scene is read from its file and joined again with the new sound
Every extra cast picture adds reference tokens, so H3 gets a little slower with each person.
🎸🎻 THE HANDS (v6.3)
The players' hands now get times from the score, not only words. Three settings on the 🥁🎤 node in 🔧 Details, all on by default:
🎸 fret_cues: the fretting hand changes to the next chord shape at each chord change of the score while the player strums or sings ("At 00:04.000 S1's fretting hand changes to the next chord shape"; a bassist moves to the next root note). In a solo, the hand moves along the neck where the melody jumps (guitar / bass: 5 semitones or more; violin: when a note leaves the hand's position)
🎻 bow_finger_cues (violin, viola, cello): from the length of the notes. Quick notes = "the left-hand fingers drop onto the strings rapidly, one after another"; note by note = "the bow changes direction with each note, about every 0.7 s"; a held note = "one finger stays pressed down with a slow vibrato ... the bow draws slowly in one direction". Slurs are played in one bow
🎥 hand_closeup: when a solo really starts, "the camera moves in to an extreme close-up of S1's left-hand fingers on the fingerboard, the face out of frame" for 2.5 to 4 s, then back out. The fingers are a few pixels in a medium shot; big, H3 draws them much better
All three are also in the Timeline's ⚙ Settings (🎸🎻 playing & camera). Two position shifts the same way less than 1.5 s apart are said once
⚠ on the 🎻 lane: the strings you set by hand on the 🎞 Timeline are checked against the audio. A span dragged or deleted by mistake (the violinist holds the violin down while it plays) is marked orange there and in the 🎼 report
Still hints: H3 decides the fingers, and correct fingering is beyond any video model today. What changes is a hand that moves when the music changes. In our violin tests the hand mostly stayed in first position (which looked right for those songs); some seeds may follow the position shifts. Hands look best with ⚡ Turbo off (20 steps).
🔒 AUDIO LOCK (v6.2)
Why: H3 makes video and audio together. When the song is only a reference (<Audio 1>), H3 composes its own soundtrack from it and animates to that; the finished MV then plays the real song, so every motion is off by however far H3's soundtrack drifted, differently in every take (measured drum strokes: +0.14, -0.14, -0.09 s). The old 🎚 +0.2 s only fixed the average.
What it does: the scene's piece of the song (exactly what plays under that scene in the finished video) is encoded with H3's audio VAE into the audio part of H3's latent, and that part is masked so sampling keeps it (audio mask 0, video mask 1; a ⛓ guide's video mask is kept). H3 draws the picture on the song's own clock. Three settings in 🔧 Details:
frozen (default): the song IS the audio track, kept as it is
anchor: the song as a keyframe at frame 0 (ComfyUI's Add Guide with audio; H3 still makes its own audio). Known to misbehave on some ComfyUI versions: keep frozen unless you are comparing
off: as before (v6.1 behaviour; 🎚 A/V sync 0.2 s applies again)
With frozen, 🎚 A/V sync is 0 by itself; anchor and off use the 🎚 value (anchor only guides H3's own audio, so measure first). 🔁 needs the same lock setting as the run it reuses (it says so, and ⤵ Load that run's settings sets it). 🔬 and the 🥁 estimate use the shift the video really had. Still hints for the motion itself: H3 decides how the lips and hands move, but now on the right clock.
🥁 THE DRUMS IN v6.2
Until now the drummer got one steady beat for everything, and the fills were "both sticks sweep across the drums". v6.2 adds a third drum cue style, adaptive, that uses what STRUM (an AI drum transcriber) tells apart: kick, snare, hi-hat, ride, crash and three toms.
Ride or hi-hat: when the beat is kept on the ride, the right stick taps the ride (on the drummer's right). When it moves from the hi-hat to the ride inside a shot, one line says when ("From 00:09.000 S1 moves the hi-hat hand over to the ride cymbal")
Tom fills by name: "a fill around the toms in this order: high tom, low tom, floor tom", only the toms heard, in the order heard. Never a mid tom STRUM did not hear
Crashes only where one is heard: a new section with no crash in the song gets none (simple puts one on every section start). "Together with the kick" only when the kick is heard with it. At most 2 crashes a shot
Rests: where the drums are silent for 2 s or more, "S1 rests in this shot: ... holds both sticks still just above the drums", or "stops at 00:08.000, plays again at 00:11.000". Before, the drummer kept drumming through the silence
Follows the real groove (simple and adaptive): no snare hits where STRUM hears no snare (the left stick stays just above it), a kick on every beat where the song has four on the floor
Kept from simple: the beat positions (snare on 2 and 4, kick on 1 and 3), the kit layout (also left-handed), and short shots keep the groove
The 🎼 report says what was taken from STRUM: "🥁 adaptive drum cues (STRUM): S1: ride from 00:09.000; tom fills: 00:07.500 high tom → low tom → floor tom; rests (silent): 00:12.000-00:15.000; section crashes not heard (left out): 00:04.000"
STRUM is needed (install guide in the download: STRUM_INSTALL.md). Without it (Demucs only, or drum sync off) adaptive gives exactly the simple cues, and the report says "⚠ STRUM is not working". simple stays the default
🔁 + adaptive = an easy A/B test: make the video with simple, switch to adaptive and make only the drum scene again with the same seed, then compare the two with the 🥁 estimate and 🔬
These are still hints. Whether H3 really draws a ride and a floor tom differently is what v6.2 is for finding out: please share what you see.
About this workflow
This is a personal, experimental workflow. It started from wanting to make songs with YuE2 and have MiniMax H3 move in time with them, and from wondering whether a short, simple prompt could be expanded by a local model (such as Gemma 4) into prompts tuned for H3.
For the local LLM, I recommend a model that can understand images (a vision-capable model), but which model you use is entirely up to you.
Because it grew out of experiments, some features may seem odd and some things may not behave as expected. Please treat it as something to experiment with, not a finished tool. If that sounds interesting to you, give it a try!
このワークフローについて
これは個人的な実験として作っているワークフローです。YuE2 で曲を作り、その曲に H3 の動きを合わせたい。そして短く簡潔に書いたプロンプトを、Gemma 4 などのローカルで動かすモデルで H3 向けに最適化できないか試してみたい。そんな思いから始まりました。
ローカル LLM は画像を理解できるモデルをおすすめしますが、どのモデルを使うかはご自由に選んでください。
実験の積み重ねでできているので、変わった機能があったり、思った通りに動かないところもあります。完成品ではなく、あくまで実験目的での使用をおすすめします。こういったことに興味のある方は、ぜひ試してみてください。
⚠ THE CUSTOM NODES ARE NOT IN COMFYUI MANAGER
It is normal that Manager can't find them. They are inside the download: unzip it, then copy the 4 folders from custom_nodes/ into ComfyUI/custom_nodes/ (see QUICK START).
"Image Saver Save Video" comes from ComfyUI-Video-Saver (by diodiogod), NOT from ComfyUI-Image-Saver.
In Manager use "Install via Git URL": https://github.com/diodiogod/ComfyUI-Video-Saver
Searching "Image Saver" in Manager finds the image-only package, which has no video node. If it is installed, remove it (the node names overlap).
🎬 WHAT IT DOES
One prompt, two models: a local LLM splits your prompt into YuE2 style tags and lyrics, plus an H3 video prompt for each scene
🧠 LLM inside ComfyUI (NEW in v5): put a GGUF (the same files LM Studio uses) in ComfyUI/models/LLM/ and pick it. It runs on llama.cpp's official llama-server, downloaded automatically the first time, started only for the split and stopped right after, so YuE2 and H3 keep their VRAM. The default, Qwen3-VL-4B-Instruct Q4_K_M (about 3.3 GB, sees your images), downloads itself on the first run. Hugging Face models (transformers) and LM Studio (also on another PC) still work
🎞 Timeline (NEW in v5.5): the 🎞 Timeline button at the bottom left opens a video-editor style strip. Drop an image file on a scene clip, drag a clip's right edge for its seconds, drop a song file on the audio track and drag it sideways to choose where it starts, click ◆ for the transition, ▶ Play to check the flow with the stills and the song. It edits the same node settings, so nothing else changes
🔎 Analysis on the timeline (NEW in v5.7): under the waveform, what the last run found, in video seconds: 🎬 the real cuts, camera shots and sections, 🎤 when each singer sings, 🥁 every drum hit coloured by drum (▼ = the drums come in; from STRUM, Demucs or the score), 🎻 when the strings are heard, 🎹 piano key strikes. Players the audio can't check (guitar, bass) are marked "no audio check". It reloads after every run
💾 Kept with the song (NEW in v5.9): the analysis is saved in song seconds as <song>.s2s.json next to the song (copy it along with the song and its .abc to another PC). The same song again, any start and any scene lengths: the scenes are cut from it and STRUM / Demucs are skipped. A new render of the same score (another vocals seed) keeps its own analysis and hand edits. Parts not analysed are marked. Picking a pre-made song shows its analysis on the timeline at once; "🔎 Analyse this song" analyses just the song (no LLM, no H3) in ComfyUI's queue, never at the same time as a run; hand edits are saved with the song, come back when it is picked again and come off for another song
✎ Fix it by hand (NEW in v5.8): drag the analysis on the timeline. 🎤 singing and 🎻 strings spans: drag the ends (start / stop) or the middle (move), double-click an empty spot to add one, a span to delete it. 🥁 ▼: drag for when the drums come in. From the next run, the H3 cues (sing / wait with lips closed, bow up / down, drums in) follow your marks, only where the scenes had an analysis when you drew; the rest stays as analysed. Strings can be drawn in songs with vocals too, where there is no audio check. ↺ = back to the analysis. Mouse wheel = zoom (0.1 s ruler, finer drags), Space = play / stop (v5.8.1)
🔬 Sync check (NEW in v6.1): on the 🎞 Timeline, pick a run and a video (the finished MV or a scene's "H3 only" video) and step it frame by frame (← → one frame, Shift + ← → one second; the frame comes straight from the file, "frame 68 / 288"). 📍 Mark where singing, drums, strings or piano really start or stop (or one drum hit) and for whom (S1–S4). The error against that run's music analysis is shown in seconds and frames, as H3 was told and as the finished MV plays (A/V sync taken in), and the marks sit on their own lane under the analysis (the music moment each is measured against as a pink line). ⚠ on marks inside a crossfade or dip. Marks keep the video and the frame and stay after a restart; a video made again is another video
🔁 Make only some scenes again (NEW in v6.2): every run keeps its scenes in
output/score2screen_runs/ (the newest 5 runs, about 50–200 MB per scene). On the 🎞 Timeline press 🔁 Remake, pick the run, "⤵ Load that run's settings", press 🔁 on the scene to make again, change its text, picture or seed and run. The other scenes are only read from their files (no H3, no decode, read frame by frame so the RAM doesn't grow), everything is joined again and saved as a new video; the old one stays. A scene that ⛓ continues from the one made again is made again too. A different song, start, scene length or size stops the run with the reason (scenes are never reused on a guess)🥁 Drum timing from the picture (NEW in v6.2): when the scenes are joined, every scene with a drum analysis is checked: the moment the sticks and the kick foot come down in H3's picture (optical flow, camera pans taken out) against the song's kicks, averaged over all the beats. The 📺 Join report and 🔬 say how late the strokes land, in H3's video and in the finished MV, with an 🎚 A/V sync to try and how well it matched (good / fair / weak). It is an estimate from the motion, not a stick detector: check weak ones with 🔬
🔒 Audio Lock (NEW in v6.2): the song is H3's own audio track, kept while the picture is made, so H3 draws to the real song (frozen / anchor / off, 🎚 A/V sync 0 with frozen). See 🔒 AUDIO LOCK above
🎞 Scene transitions (NEW in v5.5): cut / crossfade / dip to black / dip to white at each scene change. The video length never changes, so the song and the lip sync stay in step. From v6.0 the crossfade keeps the scene before moving while it dissolves (it uses the frames H3 made past the cut)
✍ Prompt check (NEW in v5.4): auto = the prompts go straight to H3. manual = the run pauses after the prompt split and a window opens with one text box per scene: fix anything, press ▶ Continue, and H3 runs with your text
One song, several scenes: YuE2 writes a single song. H3 renders each scene (1–64) from its own image, and the scenes are joined while the song plays straight through (no audible seam)
⛓ Scenes that continue (NEW in v6.0): for each scene from the second on, choose own picture (a separate shot) or continue: the frame right after the cut of the scene before is anchored as H3's first frame (H3's own Add Guide), so positions, poses, the place and the light go on across the cut. Handoff: 1 frame, or 5 frames to carry the direction of the motion too. A continuing scene needs no picture (picture 1 is used as the look). On the 🎞 timeline the ⛓ under each cut toggles it (blue = continues), the clip shows "⛓ continues 1", and the panel below has Start and ⛓ handoff for the selected scene
🧑 One main character (NEW in v6.0): a picture of the main character (face, hairstyle and clothes clearly visible) goes to every scene as <Picture 2>, with a line to keep the same face, hairstyle and clothes. Pick it in its own group above the scene pictures (with a preview), or with the 🧑 button on the timeline. Best together with ⛓
💾 Low memory (NEW in v6.0): each scene is written to temp/s2s_scenes/ the moment it is decoded, and the final mp4 is joined from the files one frame at a time, with 🔍 super resolution 24 frames at a time. Only one scene's frames are in RAM at any point. Disk: up to about 1 GB per scene (1280×720, 15 s), cleared when ComfyUI starts
📺 Join report (NEW in v6.0): frames per scene, how each scene started (own picture, or ⛓ with the frames asked, available and really used) and the RAM / VRAM peak of the join and of each scene's decode
⏳ Prompts first (NEW in v5.3): every scene's prompt is read before any scene is drawn, so H3's text encoder and the H3 model are swapped once per run instead of once per scene. Same pictures, less waiting with more scenes
The score drives the picture: YuE2 also writes an ABC sheet-music plan. The workflow reads its tempo, time signature and chords, then turns them into timed cues in each scene's H3 prompt
Bar-accurate cuts: every scene change snaps to the start of a bar
Band performance: write who plays what ("the white-haired woman plays the drums") and each player gets cues
🥁 Drum sync: an on-screen drummer follows the song's real drums (STRUM, which you install with the included guide; if it fails, the Demucs drum stem, then the score)
A steady beat H3 can draw: one stick taps the hi-hat on every beat, the other comes down onto the snare on beats 2 and 4, the right foot plays the kick
One stick in each hand, drums that come in later (the drummer waits through a guitar intro), kick-only intros and single crash hits
Follows the real groove (v6.2): where STRUM hears no snare, the left stick stays above the snare; a kick on every beat when the song has one
🆕 adaptive cues (v6.2): ride or hi-hat, tom fills by name, crashes only where heard, rests where the drums are silent (see 🥁 THE DRUMS IN v6.2 above)
Fill or no fill: a short fill, a big fill, or just one kick + crash (no made-up tom fill where STRUM heard none)
Kit layout: tells H3 where the snare, hi-hat, crash, toms and ride are, as seen from the drummer's seat (standard / left-handed / off)
The song is transcribed once; the later scenes reuse it, even after a ComfyUI restart
🎹 Piano sync: an on-screen pianist follows the real key strikes (STRUM + Basic Pitch, optional)
🎤 Waiting performers (NEW in v5.6): H3 tends to make anyone holding a microphone sing and anyone holding an instrument play. A performer who doesn't sing / play yet is told the posture and until when ("S1 keeps the lips closed and does not sing until 00:12.000"), and the LLM's words that make them start too early become that posture. The singing start can also be set by hand (vocal_in). The singer is found from "who sings", "vocalist", "singer", "lead vocals" and (v5.8) "S1 ... while singing" / "S1 sings ..."
🎞 One screen (NEW in v7.0): the Timeline's left column (🎬 Main / 🎵 Song / 🎚 Sound / ⚙ Settings / 🔍 Details), ⛶ full screen, ▶ next-run summary. See 🎞 THE NEW SCREEN IN v7.0 above
🎭 Cast (NEW in v7.0): a main character and three more, each with a picture, a name and a part, given to H3 only in the scenes they are in; their parts drive the band cues
🎚 MV sound (NEW in v7.0): EQ, reverb, width and fades on the finished video's sound, and 🔁 remake only the sound
➕ More LoRAs (NEW in v7.0): up to three LoRAs on H3 for every scene
🎸 Fretting hand timed (NEW in v6.3): chord-shape changes at the score's chord changes, hand-position shifts in a solo from the melody's pitch (guitar, bass, violin). See 🎸🎻 THE HANDS IN v6.3 above
🎻 Bow and fingers from the score (NEW in v6.3): a violinist's / cellist's two hands follow the note lengths: quick notes, note by note, held notes with a slow vibrato; slurs in one bow
🎥 A close-up of the fingers (NEW in v6.3): when a solo really starts, the camera moves in to the fingers (the face out of frame) for 2.5 to 4 s, then back out
🎻 Violin sync: an on-screen violinist (viola, cello) plays only while the strings are really heard, and keeps the violin down with the bow off the strings while they are silent. Automatic for instrumental songs
🔍 Super resolution (NEW in v5.2, optional): NVIDIA RTX Video Super Resolution enlarges and sharpens the finished video just before the song is put on. H3 is not run again, so movement, lip sync and timing don't change. From v6.0 it runs 24 frames at a time while the scenes are joined from their files. Off by default
🎚 Even loudness: every song is brought to -16 LUFS with a peak limiter
Singing on screen: auto / yes / no. A song can have vocals while nobody on screen lip-syncs
Instrumentals: write "Instrumental." The lyrics are removed, the melody moves to an instrument, and everyone's mouth stays closed
Skips the intro automatically: vocal onset detection (Demucs) starts the clip just before the singing
Use your own songs: load a song you made earlier (with its .abc score), or "(newest song in
output/)", and ▶ Start at any second (the chorus, for example)A/V sync: H3's motion lags the sound by about 0.2 s, so the final audio is shifted to match (adjustable)
Run log inside the workflow: 📺 Runs so far adds one line per run (video name, seeds, steps, memo), also saved as a CSV
Civitai-ready output: the MP4 is saved with metadata (all scene prompts, YuE2 style and lyrics, H3 + YuE2 model hashes); 🏷 a switch on the Timeline (🎚 Sound → 💾 The finished file) saves it with nothing inside instead
⚠ The singing, drum, piano, violin and hand cues are hints
These features only give H3 hints: when to wait, when to sing or come in, where to hit. H3 decides how the scene is finally drawn, so it may not always follow them. The player may start a little early, keep playing through a rest, hit the wrong drum, or keep the hand in one place. Results vary from seed to seed, so if a take doesn't look right, try another seed. Shorter scenes and fewer camera changes also make the cues easier to follow.
歌・ドラム・ピアノ・バイオリン・手の動きの機能は、H3 に「いつ待つか・いつ歌う/入るか・どこを叩くか・いつ手を動かすか」のヒントを出すだけです。最後にどう描くかは H3 しだいなので、いつも思った通りになるとは限りません。うまくいかない時は seed を変えて何回か試してみてください。
⚙️ HOW IT FLOWS
Prompt Splitter (LLM, inside ComfyUI): makes the YuE2 style and lyrics, plus one H3 prompt per scene
YuE2: writes the ABC score, then sings the song (skipped when you pick a pre-made song)
Loudness + vocal onset: evens out the volume, trims the song, cuts the scenes on bar lines
🎵 Music brief + 🎬 Scene Director (v8.0 / v8.1): each scene's LLM is told its part of the song and who plays what, when, then writes the scene's shots
🎭 Cast fixes (v8.2): wrong instruments, a second drummer, unnamed "her", who sings and the main character's focus are put right before H3
ABC Timing: adds tempo, chord, band, drum, piano and violin cues to each scene's H3 prompt
✍ Prompt check: straight on (auto), or a pause to fix the prompts by hand (manual)
🎭 Cast: who is in each scene goes into its prompt (definition and same-face lines) and their pictures to H3 (v7.0)
H3 ref2va: reads every scene's prompt first, then renders each scene from its image (⛓ a continuing scene waits for the scene before and starts from its frames; 🧑 every scene gets the character picture)
💾 Each scene is saved to a file as soon as it is decoded. 🎞 The scenes are joined from the files frame by frame (with an optional transition and 🔍 super resolution), the song is laid over them, and the video is saved
💾 The run keeps its scenes for 🔁, and 🥁 the drum timing is estimated from the picture when the scenes are joined
🔬 Afterwards, if you like: the Sync check on the 🎞 Timeline measures how well the finished video follows the song
🔁 Next time, make only the scenes you want again: the others come from the kept files
The internals are packed into subgraphs. From v7.0, day to day you can do everything on the 🎞 Timeline (⛶ full screen if you like); on the canvas the scene columns, the 🎭 Cast & song group, the 🎛 switches and the 🧠 LLM settings are where things are.
🎛 THE SWITCHES
From v7.0 all of these are also on the 🎞 Timeline (header and the ⚙ / 🎵 tabs).
🚀 H3 engine (v7.1): Standard (20 steps) / ⚡ Turbo (4 steps, drafts) / FastH3 Omni (8 steps, the default)
🎤 On-screen singing: auto / yes / no
🥁 Drum sync: auto / off (piano sync follows this switch too)
🎵 Song delay: seconds of silence at the start of the video
🎵 Song only: makes just the song; H3 and all video saving are skipped
🎬 Scenes: 1–64, added on the 🎞 Timeline (+); the cuts are automatic, ✋ or set by hand
✍ Prompt check: auto / manual (timeout_min: continue by itself after N minutes, 0 = wait)
🎞 Scene transition: cut / crossfade / dip to black / dip to white, seconds (0.5 s)
🔍 Super resolution: off / rtx (scale 1.5, quality ULTRA, strength 1.0)
🧑 Character picture: (none) / a picture (<Picture 2> in every scene)
⛓ Each scene's start: own picture / continue, handoff 1 frame / 5 frames (also on the 🎞 Timeline)
🔁 Scene remake: (off) = make every scene / a kept run + the scenes to make again, seed same as that run / new, keep_runs (also on the 🎞 Timeline)
🧠 LLM settings: gguf (default) / transformers / lm_studio, model, GPU layers, context size
Seeds (video, vocals, score) can be locked separately in the 🎲 Seed details group. One video seed is used for all scenes of a run, so one number brings back the whole video. Rarely used settings live in 🔧 Details:
bar snapping (keep fps at 24: H3 makes 24 fps video, and from v6.0 any other value stops before the run)
🔒 Audio Lock (frozen / anchor / off), drum cue style (simple / adaptive / detailed; adaptive needs STRUM), 🎸 fret_cues (on / off), 🎥 hand_closeup (lead / off), 🎻 bow_finger_cues (on / off), when the drums come in (auto / start / a second in the song), kit layout, 🎤 when the singing comes in (auto / a second in the song), ✎ timeline marks (the singing / strings spans drawn on the timeline; filled in by the timeline)
A/V sync
song volume: even loudness (-16 LUFS, on by default) + optional volume curve
📦 REQUIREMENTS
Custom nodes (my own, included in the download):
ComfyUI-PromptSplitter v3.20: 🗣 the lines of the Lines box reach H3 as entered (words, speaker, order, a shot for each), 🔊 each scene's sound for the LLM (🗣 spoken lines kept word for word, Japanese lines in hiragana, the sound-only take), prompt split (🎵 the song field and 🎭 who is who for the LLM; 🧾 the official Ref2VA shape, v3.4) (lyrics in the language you ask for) with the LLM inside ComfyUI (GGUF / transformers / LM Studio), vocal onset (the separated vocals kept per song), song loudness. Run
pip install demucsin ComfyUI's PythonComfyUI-ABCTiming v2.59: score → rhythm, band, drum (simple / adaptive by instrument), piano and violin cues, 🎸 fretting-hand and 🎻 bow / finger cues, waiting cues for the singer and players, the analysis for the timeline (kept with the song), and the spans you fix by hand
ComfyUI-ABCInstrumentalize v1.5: instrumental score and bar-snapped scene cuts (24 fps check)
ComfyUI-ABCScoreIO v1.86: 🗣 the Lines box on the Timeline, 🏷 Civitai metadata on / off, 💾 the finished video encoded once (picture CRF and sound kbit/s on the 🎚 Sound tab), 🔊 each scene's sound (🎵 / 🎛 / 🎬 / 🔇), the mix with 🎚 master volumes and 🚃 the place under the voices, 🎬 1–64 scenes, 🎬 the Scene Director, 🎭 the cast fixes, 👀 a look for each face, ✂ add after / duplicate / split / delete several scenes, 💾 stream decode, 🚀 the three H3 engines, 🎞 the Timeline (left column, ⛶ full screen), 🎭 the cast, 🎚 the MV's sound, ➕ more LoRAs, ⚡ Turbo steps, 🔒 Audio Lock, 🔁 scene remake with the runs kept, 🥁 drum timing estimated from the picture, switches, scores, pre-made / newest song, run log, A/V sync, scene pictures, ⛓ scene continuity, 🧑 character picture, 💾 low-memory join from files with 📺 report, 🎞 transitions and timeline (with 🔎 analysis, 🔎 analyse-only, ✎ hand fixes kept with the song), ⏳ prompts first, ✍ prompt check, 🔍 super resolution, 🔬 sync check (frames read with PyAV, which comes with ComfyUI)
Other custom nodes:
ComfyUI-Video-Saver by diodiogod ("Image Saver Save Video", final save with Civitai metadata): https://github.com/diodiogod/ComfyUI-Video-Saver
Only for 🔍 super resolution = rtx: ComfyUI Nvidia VFX Nodes (official, in ComfyUI Manager): https://github.com/Comfy-Org/ComfyUI_Nvidia_VFX_Nodes. Needs an NVIDIA RTX GPU and CUDA 13 PyTorch (cu130 or newer)
Models:
diffusion_models: minimax_h3_ref2va_pruned_int8_convrot (Standard / ⚡ Turbo) and a FastH3 Omni file for 🚀 FastH3 Omni (the default): fasth3_omniref int8 22.2 GB (any GPU) or NVFP4 QAD 13.8 GB (RTX 50 / Blackwell only)
Optional, worth a try: Minimax-h3_Singularity_ref2va_Pruned_v1.3_int8 (a fine-tune that held the faces of a 4-person band better in my test; 🚀 engine = h3)
loras: minimax_h3_ref2v_turbo_4step (⚡ Turbo)
text_encoders: qwen3vl_32b_minimax_h3_nvfp4_awq
vae: minimax_h3_video_vae_int8_convrot (Kijai, faster; the default) or minimax_h3_video_vae_fp16, minimax_h3_audio_vae_fp32
checkpoints: yue2_3b_bf16 (default) or yue2_3b_int8_convrot (lighter)
LLM: models/LLM/ (any GGUF; put its mmproj file in the same folder so it can see your images). The default Qwen3-VL-4B-Instruct Q4_K_M downloads itself
LLM: no LM Studio needed any more. Nothing to pip install or compile: llama.cpp's llama-server is downloaded from GitHub the first time (CUDA build on NVIDIA, otherwise Vulkan). A vision model (Qwen3-VL, Gemma 4 …) with its mmproj is recommended so the LLM can see your images. Big MoE models can keep their experts on the CPU (extra_args, e.g. --n-cpu-moe 9). LM Studio on this or another PC still works: set backend to lm_studio.
Disk: free space for temp/ (up to about 1 GB per scene while a run is going; cleared when ComfyUI starts). 🔁 keeps the newest 5 runs' scenes in output/score2screen_runs/ (about 50–200 MB per scene; keep_runs = 0 keeps none). The 🔬 marks are small files in output/score2screen_sync/.
STRUM (needed for the drums from v6.2): an AI drum transcriber (free, MIT) that tells the kick, snare, hi-hat, ride, crash and toms apart. The drum cues are built on it: when the drums come in, fills, crashes, the steady beat that follows the song, and the adaptive cues. It runs in its own Python next to ComfyUI, so ComfyUI's Python is not touched. A detailed step-by-step guide with a check at every step is in the download: STRUM_INSTALL.md (Japanese: STRUM_INSTALL.ja.md). Without STRUM the workflow still runs (Demucs hits without drum names, or the score), and the report shows "⚠ STRUM is not working".
🚀 QUICK START
Download and unzip. Inside:
workflows/,custom_nodes/, a README and the STRUM install guide (English + Japanese)Copy the 4 folders inside
custom_nodes/intoComfyUI/custom_nodes/(not one folder deeper). Updating? Replace the old folders, and reload the page after restarting (Ctrl+F5)Install ComfyUI-Video-Saver via Git URL, and
pip install demucsin ComfyUI's Python. Restart ComfyUIInstall STRUM by following STRUM_INSTALL.md: Git, Python 3.11 and ffmpeg, STRUM in ComfyUI/models/strum with its own venv and CUDA PyTorch, the drum weights, and a quick test that makes a notes.mid
Drag a workflow from
workflows/onto the canvas. Leave 🧠 LLM settings at the default (the first run downloads llama.cpp and the model once), or pick your own GGUF in models/LLMThe 🎞 Timeline opens at the bottom (⛶ for full screen). 🎵 Song: describe the song (mood, genre, vocals or "Instrumental.")
🎬 Main: optionally a 🧑 main character and 👤 subs with a picture, a name and a part; pick the scenes each sub is in
Press + to add scenes (up to 64) and drop a picture on each clip. In each scene's box pick 🔊 its sound (🎵 Song is the default). The cuts follow the song by themselves (drag an edge to set one by hand); a scene's own text is optional, the LLM writes every scene. Width / height are in ① Shared on the canvas
⚙ Settings: 🚀 the H3 engine (FastH3 Omni by default, Standard 20 steps for the cleanest fingers, ⚡ Turbo for drafts); models, LoRAs and VAEs if you use other files
Check ▶ Next run under the header, then ▶ Run (or ComfyUI's Run). Check the 📺 previews afterwards: the scene cut, the prompts H3 got, and Runs so far
Optional: 🔬 Sync check on the Timeline to measure how early or late H3 was
Optional: one scene not right? 🔁 Remake, pick that run, "⤵ Load that run's settings", 🔁 the scene, change it and run. Only the sound? 🎚 Sound, then 🔁 "🎚 sound only" and run
💡 TIPS
🗣 Spoken lines (v9.1): set the scene to 🎬 H3's sound and enter each line in the scene's 🗣 Lines box (speaker, words). Keep them short: H3 speaks about 3.5 English words a second, so two lines of about 10 words fit a 12-second scene. The scene's words are for what is seen and heard, the list is for what is said
🗣 Spoken lines, the v9.0 way: in a scene with an empty Lines box you can still write each line in quotes with who says it:
The silver-haired woman says: "Shall we try singing that song?". Three short lines fit a 13-second scene. Say what else is heard ("the clatter of the rails all the time, footsteps pass in the aisle")✍ One action, one sentence (v9.0): "they stand up and dance, singing" in one sentence reached H3 as "dance and sing", seated. "The three stand up from their seats. Standing in the aisle, in a wide shot that shows them whole, they dance and sing." stood up. One action per sentence, in order, and say how much of the people is seen
💬 Subtitles with Singularity v2 (v9.0): with
Minimax-h3_Singularity_ref2va_v2the quoted lines were also drawn on the picture as subtitles, and the scenes that continue from that scene's last frame went on drawing text. The same prompts with Singularity v1.3 and with FastH3 Omni had no text. For spoken lines use v1.3 or Omni🎛 Song + ambience (v9.0): write what should be heard in the scene's words ("the train speeds up, the rail joints get louder"). No spoken lines in a 🎛 scene; use 🎬 for those
🎭 Band MV: give each player a picture and a part (drums, bass ...) and pick their scenes: the part drives their cues and stays the same from scene to scene
🎚 Only the sound: always use 🔁 "🎚 sound only". With 🎲 random seeds an ordinary run draws a new picture too
Prompts can be in any language the LLM reads (English, Japanese, Russian ...). Add "Lyrics in Russian." (or any language) to choose the lyric language; without it the lyrics follow the language of your description
For a band, always write who plays what
🥁 Try adaptive drum cues: install STRUM (STRUM_INSTALL.md), set the drum cue style in 🔧 Details to adaptive and look at the 🎼 report ("🥁 adaptive drum cues (STRUM): ..."). To compare with simple, 🔁 make only the drum scene again with "same as that run" for the seed
🔁 Change only the text or the picture of a scene and keep "same as that run": the rest of the take stays the same. "new" gives another take of that scene
🔁 Keep the song, start and scene lengths of the run (⤵ Load that run's settings does it): the kept scenes only fit where they were in the song
🎸 Want two-hand tapping in a guitar solo? Write "two-hand tapping" (or ライトハンド / タッピング) in that scene's text. Without it the picking hand is kept over the guitar's body
⛓ Continue is for the same person going on (a wide shot, then a close-up of the same player). A scene with another person or instrument (guitar → drums): own picture, or the drummer may start holding the guitar. The orange ⛓⚠ on the timeline tells you
⛓ For a clean join, leave the continuing scene's picture at (none) (picture 1 is used as the look), or use a picture of the same people and place. A very different picture can pull against the frame handed over
⛓ alone can drift a little in the face over several scenes: add a 🧑 character picture with the face, hairstyle and clothes clearly visible
The 📺 join report tells you whether a 5-frame handoff really happened (it falls back to 1 frame when H3 made fewer frames past the cut)
Don't want someone (or a pet) to sing? Say so in the scene text ("the dog doesn't sing"): the LLM sometimes writes "... and singing" for everyone on screen
Guitar and bass have no audio check (the timeline says "no audio check (from the score)"): their cues come from the score. Drums, piano and violin are checked against the real song
Guitar intro before the drums? Leave "drums come in" on auto with drum sync on, or type the second in the song
🎻 Hands: render the final with ⚡ Turbo off (20 steps): fingers are fine detail. Ask for the fingers, not "the hands": "an extreme close-up of her left-hand fingers on the fingerboard, her face out of frame" ("a close-up of her hands" tends to become her face). A sharp picture with the neck and the hand clearly visible helps too
🎸🎻 Compare the hand cues: 🔁 make the scene again with the seed "same as that run" and fret_cues / bow_finger_cues off, then look at the chord changes and the held notes with 🔬
Drummer images: a kit with a cymbal low beside the snare on the drummer's left works best (it is played as the hi-hat)
Violinist: H3 tends to start playing when the camera cuts to a close-up of the player. Put the close-up where the violin comes in
✍ Prompt check (manual) is also a quick way to compare LLMs: look at the prompts, then ✖ Cancel before H3 runs
A take looks wrong? Open the 🎞 Timeline first. If the 🔎 analysis is off (the singing, the strings or the drums at the wrong time), drag it right there (✎) and run again. If the analysis is right but H3 didn't follow it, try another seed, a picture in a waiting pose, or a shorter scene
🔬 Compare takes with numbers: mark the same moments (the first sung word, the first drum hit) on two runs, and the errors tell you which seed or setting followed the song better
🔬 To judge H3 itself, use the scene's "H3 only" video (no crossfade, frames past the cut included); to judge what a viewer gets, use the finished MV. A mark inside a crossfade gets ⚠: two scenes are mixed there
🔒 Comparing with and without the lock: make the scene with frozen, then 🔁 make it again with off and "same as that run" for the seed. The 🥁 estimate and 🔬 show the difference in numbers
🎚 With 🔒 frozen, leave A/V sync alone (it is 0 by itself). If the lips still look early or late, measure with 🔬 first: a constant offset means the song's start, not the lock
🥁 The 📺 Join report already estimates how late the drummer's strokes land (e.g. "0.31 s after the kick in H3's video -> 0.14 s late in the finished video"). "close to half a beat: H3 may be playing between the beats" means a steady pattern looks the same half a beat later: check with 🔬
🔬 Every mark in the finished MV about the same amount late (say +0.1 s)? Then 🎚 A/V sync in 🔧 Details can go up by about that much. Videos made with v6.0 can be measured on their "H3 only" videos (the finished MV needs the scene borders v6.1 keeps)
Pre-made song: keep its .abc (and, from v5.9, its .s2s.json) next to it. Pick it, press "🔎 Analyse this song" once, fix the lanes by hand if needed, then run: the scenes use the saved analysis
⚠ on the 🎻 lane: the strings you set by hand are silent (orange) or play (yellow) where the audio says otherwise. Moved a span by mistake? ↺ goes back to the analysis
Hand edits belong to their song: picking another song takes them off, picking the song again brings them back. ↺ clears them
Want to try a few songs first? Turn on 🎵 Song only (with 🎵 pre-made song on (none): with a song picked there, nothing new is made), then pick "(newest song in
output/)" and make the videoSinger starts too early or too late? Set 🎤 vocal_in (the second in the song where the singing starts), or press "🎤 Singing starts at the playhead" on the Timeline
Transitions: crossfade for a soft change, dip to black for a new chapter. Keep the seed fixed and only change the transition: it finishes in moments
Turbo LoRA ON for drafts, OFF for final renders (H3 follows detailed cues better with more steps)
Scene length: up to 15 s per scene by default; up to 24 s has worked. For a long song, 4 shorter scenes usually hold up better than 2 long ones
🔍 Super resolution and the join now work from the saved scene files 24 frames at a time, so long songs no longer need the whole video in RAM. The 📺 join report shows the RAM / VRAM peak, handy if you want to measure
Resolution: multiples of 32, short side up to 768 (1024×768 recommended)
Errors? Paste the ComfyUI console log and the workflow .json into Claude or ChatGPT and ask what is missing
🗂 OLDER: v4.1 / v3.9 / v2.34
v4.1 (1–4 scenes) and v3.9 (2 scenes) use LM Studio for the prompt split; get them from the older versions on this page. The older "every option" v2.34 Full / Legacy workflow also still works with these custom nodes.
Version history
v9.1: 🗣 the Lines box on the Timeline: every spoken line entered as a row (speaker, words, optional start second) and passed to H3 as entered, each in a shot of its own; ⚠ lines in a scene that is not 🎬 are flagged with a one-click fix; the v9.0 way (quotes in the scene's words) still works in a scene with an empty list
v9.0 (major update since v8.2): 🔊 each scene's sound: 🎵 song / 🎛 song + ambience / 🎬 H3's sound / 🔇 silent; 🗣 spoken lines in quotes, Japanese lines in hiragana; 🚃 the place's sound raised under the voices; 🎚 master volumes (🎵 song, 🎬 H3, 🎛 ambience depth), H3's sound levelled to the song; the song runs on behind 🎬 / 🔇 scenes; 🔁 remake keeps each scene's sound; scene-prompt fixes from real runs (said is not sung, the scene after an intro sings, a copied "scene before" is asked again)
v8.2 (major update since v7.0): 🧾 the official Ref2VA prompt shape; ✋ cuts by hand; 🎭 the band without mix-ups (wrong instruments, a second drummer, unnamed "her", resting players), 🌟 the main character in front, 🎥 close-ups that stay on their person; 🎤 who sings from the score, only the singer sings; 🥁 big swings, 🦶 timed kicks with independent feet, drum solos hit by hit, ordered cue times; new defaults (FastH3 Omni, adaptive drums, int8 VAE, structured prompt). Includes v8.1, v8.0 and v7.1 (below), which were not released on their own
v8.1: 🎬 the Scene Director (who plays what, when, for each scene's LLM); roles per cast card (auto / lead / rhythm / backing / rest)
v8.0: 🎬 up to 24 scenes cut on the song's changes, built when the run starts; 🎵 a music brief for every scene's LLM; scene cuts that reach the song's end
v7.1: 🚀 three H3 engines (Standard / ⚡ Turbo / FastH3 Omni)
v7.0: 🎞 one screen: the Timeline's left column in five tabs (🎬 Main / 🎵 Song / 🎚 Sound / ⚙ Settings / 🔍 Details), ⛶ full screen (Esc) with ⏳ the run's progress, opens by itself, ▶ next-run summary and run guard; 🎭 a cast S1–S4 with pictures, names and parts, per scene; 🎵 the song in its own field; 🎚 the MV's sound (EQ, reverb, width, fades) and 🔁 sound only; ➕ up to three more LoRAs; VAE lists by kind. Includes v6.4 (⚡ one Turbo steps setting) and v6.3 (below), which were not released on their own
v6.3: 🎸 the fretting hand timed from the score (chord shapes on the chord changes, position shifts in a solo); 🎻 bow and fingers from the note lengths (quick notes, note by note, held notes with vibrato; slurs read); 🎥 a close-up of the fingers when a solo really starts; ⚠ hand-drawn strings checked against the audio; fixes (an instrumental's melody, a 'rest' error with cue_detail changes)
v6.2: 🔒 Audio Lock (the song is H3's own audio track: frozen / anchor / off, 🎚 A/V sync 0 with frozen); 🥁 adaptive drum cues (ride / hi-hat and when it moves, tom fills by name, crashes only where heard, rests where the drums are silent; needs STRUM, simple unchanged); 🔁 make only some scenes again (the runs keep their scenes, kept scenes read from file frame by frame, settings checked, ⛓ followers remade too); 🥁 drum timing estimated from H3's picture (📺 Join report, 🔬); 📘 STRUM needed, with a detailed install guide in the zip; 🥁 the steady beat follows STRUM (no snare where none is heard, kick on every beat); ⛓⚠ warning when a continuing scene changes instrument; 🎸 the picking hand stays on the guitar's body
v6.1: 🔬 sync check on the timeline: step the finished MV or a scene's "H3 only" video frame by frame, mark where singing or playing really starts or stops, and see the error against the song in seconds and frames (as H3 was told and in the finished MV, A/V sync taken in); marks kept per video; the join keeps each run's scene borders
v6.0: ⛓ scenes 2–4 can continue from the scene before (1 or 5 frames handed over, ⛓ under each cut on the timeline); 🧑 one main character picture for every scene (<Picture 2>); 💾 low memory: each scene saved to a file and the video joined from the files (super resolution chunk by chunk); crossfade keeps the scene before moving; 📺 join report (frames, real handoff, memory peaks); fps must be 24
v5.9: 💾 the analysis is kept with the song (<song>.s2s.json) and used again (no STRUM / Demucs for the same song); a pre-made song's analysis shows on the timeline at once; 🔎 analyse only the song (no LLM, no H3, in ComfyUI's queue); hand edits saved with the song; each recording of a score kept apart; "not analysed" / "older analysis" marked on the timeline
v5.8.1: 🎞 timeline wheel zoom (down to 0.1 s), Space = play / stop
v5.8: ✎ fix the analysis by hand on the timeline (🎤 singing and 🎻 strings spans, 🥁 drums in, only where the scenes had an analysis, in seconds of the song); the H3 cues follow it, ↺ back to the analysis; the singer is found from more wordings; the timeline panel resizes at any height
v5.7: 🔎 the last run's analysis on the timeline (singing, drum hits, strings, piano, real cuts); auto start shown where the last run started the song; only the newest song's analysis, each scene in its place, real cut lengths on the clips ("7.1s → 8.0s")
v5.6.1: fixes: the singing start set by hand works in short scenes, the LLM's other actions (waving, dancing) are kept when a singer is set to wait, "vocalist" / "singer" / "lead vocals" are recognised
v5.6: 🌐 lyrics in the language you ask for; 🎤 waiting performers are shown waiting (lips closed / hands still until their cue), the LLM's too-early words are fixed, singing start by hand (vocal_in)
v5.5: 🎞 timeline editor (drag & drop images and songs), 🎞 scene transitions, drum sync fixes (backbeat, no made-up fills, kick-only parts)
v5.4: ✍ prompt check (auto / manual): pause after the prompt split and fix each scene's prompt by hand in a window
v5.3: ⏳ every scene's prompt is read before any scene is drawn (one model swap per run); inputs laid out scene 1 → 4
v5.2: 🔍 optional RTX video super resolution on the finished video
v5.0: 🧠 the prompt LLM runs inside ComfyUI (GGUF via llama-server, transformers, or LM Studio); load / unload handled automatically
v4.1 / v3.9 (update 2026-10-14): 🎻 violinist plays only while the violin is heard; prompt split with thinking off and a 600 s timeout
v4.1: choose 1–4 scenes with all the v3.9 features
v3.9: one-zip download, even song loudness, song-only switch, newest-song pick, drums that wait / come in later, kick-only intros, fill or no fill, kit layout for H3
v4.0: choose 1–4 scenes (🎬 scene count switch)
v3.8: drum cues H3 can draw
v3.7: piano sync
v3.6: A/V sync
v3.4 / v3.5: run log in the workflow
v3.2: ▶ Start at for pre-made songs
v3.0: the new Simple line (4 switches)
v2.09: first public release
Description
Score to Screen v8.2 Simple — major update since v7.0
v8.2 brings together everything made since v7.0 (v7.1, v8.0 and v8.1 were not released on their own): 🎬 up to 64 scenes cut on the song's changes, the 🎬 Scene Director that tells each scene's LLM who plays what and when, 🧾 MiniMax's official Ref2VA prompt shape, 🚀 three H3 engines including FastH3 Omni (8 steps, about 4× faster here), and a long list of fixes from real 19-scene, 4-person band runs: the right person on screen, only the singer singing, the main character in front, and drums that follow the song hit by hit.
🎬 Up to 64 scenes (v8.0: 24, v8.2: 64)
On the 🎞 Timeline, + (or a picture dropped on "+ Add a scene") adds a scene, up to 64. ◀ ▶ move a scene, 🗑 deletes it
Automatic scene lengths: the cuts go on the song's changes (the score's sections: intro, verse, chorus ...), always on a bar line, 15 s at most per scene ("max" makes it shorter). Each clip shows the part of the song it plays
The LLM writes every scene: it gets the 🎵 song description, who of the 🎭 cast is in the scene and where the scene is in the song (e.g. "scene 5 of 12, the song goes verse → chorus"). Your own words for a scene are optional
One H3 set per scene is built when the workflow runs (ComfyUI graph expansion), so the graph stays small: no more scene columns
🎭 a sub member can be in any of scenes 1-64 (e.g.
3,5-8). 🔁 remake, 🔍 details and 🔬 sync check handle 1-64 scenesH3 runs once per scene: a whole 3-minute song is about 12-15 scenes
When the scenes can't take the whole song, the Timeline says how many more scenes would (with a + button); the cuts never go past the song's end, the last scene ends at the song's real end, and ☑ whole song required stops the run before anything is made when the end is left out
The 🧠 main text is scene 1's text when it has none of its own, and the whole MV's idea for the other scenes; scene 1's rhythm cues are made as in v7.1
✍ Prompt check (auto / manual), ⛓ handoff (1 / 5 frames) and 🔍 RTX quality / strength are on the 🎬 Make every scene node
🔁 remake also compares the scenes' real cuts with the kept run
🎵 Music brief: before each scene's LLM writes its shots, it is told the tempo, where the song part changes, where the singing (or melody) is, the energy (quiet → loud) and how the scene before ended, so the camera and movement follow the music
🎬 Scene Director (v8.1)
For every person of the 🎭 cast in a scene: when they really play, from the analysis kept with the song — 🥁 drums (where they play, fills, crashes), 🎹 piano / keyboard, 🎻 violin / cello, 🎤 the singing from the score's vocal line. Every line says where its times come from ([fixed: source] or [unknown])
Guitar and bass have no instrument-only analysis yet: their timing is passed on as unknown, never made up from the whole mix. The melody player of an instrumental comes from the cast note (lead / solo / melody), not the order
Drums: a kick-only part is given as kick times, however many (hands still); fills have a confidence (toms, a crash after, a density jump count for it; a steady repetition against it) and only sure ones are camera moments
Two players of the same instrument do not both get its one analysis: the one marked lead does, or the player is passed on as not identified. Unknown parts are told to hold the instrument, not to play. Each cast card has a role (auto / lead / rhythm / backing / rest); a denied lead note does not count; with two leads nobody is assigned. [fixed] = heard, [assigned] = the cast card picked who. rest reaches H3 (the definition says not playing, ABC Timing adds no cues) and the final prompt is checked for a resting player made to play; role_by_scene changes the role per scene; ABC Timing v2.53 never takes a resting player's instrument; the timeline's cast cards edit the roles
The brief tells the LLM to write visible states in the positive for H3 (no negative prompt) and moments as single-time cuts (At 00:04.000), per the H3 prompt guides
The moments the camera can follow: the singing coming in, the drums coming in, a fill, a crash, a melody coming in, the song part changing, a break and the music coming back
The playing times are fixed (🎼 ABC Timing still writes the exact cues); the camera, framing and faces are left to the LLM
A part not analysed yet is said so — run 🔎 Analyse this song on the Timeline first and every scene has it all
scene_brief on 🎬 Make every scene + join: director + scene before (default) / music + scene before / music / off. 🔍 Details shows each scene's brief and the director's state (full / partly / no playing times needed / could not be made)
🧾 The H3 prompt in MiniMax's official shape (v8.2, PromptSplitter v3.4)
The last field of the 🧠 prompt split node: shot_list (default, as before) or ref2va_structured — the same definitions and [Shot n] list poured into the six sections of MiniMax's official Ref2VA prompt guide: subject_definitions / summary / retention_analysis / detailed_description / overall_soundscape / non_diegetic_music
The [Shot n] lines do not change by a letter and 🎼 ABC Timing writes into them as before; the "Audio: <Audio 1> exactly as it is" line becomes "<Audio 1>: fully_copy" in retention_analysis, a soundscape, and "non_diegetic_music: N/A". The summary opens with the official task line, every person gets a <Subject n> label and a retention line of their own (face, hairstyle, outfit, the instrument held), <Picture 1> is partially_preserved so the camera may move; the final cast (roles, rest) rewrites the summary and the retention lines
Compare the two with the same song, picture and seed: the official shape is the one the model was taught, so faces, clothes and playing may hold better — not verified on a GPU yet. 🔒 Audio Lock is needed either way
The shape is also on the Timeline (settings tab, 🧾 H3 prompt shape, under 🔒 Audio Lock)
v8.1 / v8.0 / v7.1 workflows still open (prompt_format is shot_list there)
🚀 Three H3 engines (v7.1)
Standard: H3 ref2va, 20 steps, res_multistep. The cleanest hands and fingers (finals)
⚡ Turbo: H3 ref2va + the 4-step Turbo LoRA, 4 steps (drafts)
🚀 FastH3 Omni: FastH3-OmniRef (fasth3_omniref_8step_pruned_int8_convrot), 8 steps, euler, no Turbo LoRA. The same settings as ComfyUI's official template: sigma shift 12/3 and VSA sparse attention (keep 10 %)
Pick it with the three buttons at the top of the 🎞 panel's ⚙ Settings tab, or on the new 🚀 H3 engine node
🔍 Details and ▶ Next run show the engine. The 💾 Civitai metadata gets the model, steps and sampler really used
The cast, 🔒 Audio Lock, ➕ more LoRAs and 🔁 remake work with all three. The text encoder and VAEs are H3's
🔁 remake remembers each run's engine: it says when the engine differs, and ⤵ Load that run's settings sets it back
⚠ notes when FastH3 Omni is picked: settings that differ from the template, 🔒 frozen + Omni (not tested on a GPU yet), extra LoRAs made for H3
🔍 Details shows whether VSA really ran in a FastH3 Omni run (or fell back to dense attention, and why); vsa_log also prints it in the console
🧪 First-test settings for FastH3 Omni: 🔒 Audio Lock off, 1 scene of 5 s, the official settings
🔁 remake also keeps the samplers, the Omni shift / VSA settings and the extra LoRAs of each run, and ⤵ loads them back
What you need for 🚀 FastH3 Omni
A FastH3 Omni file in ComfyUI/models/diffusion_models: int8 22.2 GB (the template's, any GPU), NVFP4 QAD 13.8 GB (RTX 50 / Blackwell only), FP8 23.1 GB or BF16 44.2 GB. Only Omni files are offered; another FastH3 file stops before loading
A recent ComfyUI with ModelSamplingMiniMaxH3 and Model Sparse Attention (the versions with the FastH3 template). Standard and ⚡ Turbo still run on older versions
On a 16 GB card ComfyUI offloads part of the model, so it is slower: start with short scenes
How fast, on my PC
One 13.8-second scene, the same song, picture and seed, RTX 5060 Ti 16 GB on OCuLink (PCIe 4.0 x4, so slower model offloading than a desktop x16 slot): H3 int8 20 steps 2000 s, Dasiwa V4 Turbo 8 steps 875 s, FastH3 Omni NVFP4 8 steps 458 s
All three drew the 🥁 big stick swings; H3 at 20 steps still has the cleanest fingers
🎭 The band and the main character (v8.2, from real runs)
🎬 With 2+ people, every shot from the second on keeps the place and everyone in view (a real run's pan turned into another room and the keyboardist vanished); not when the LLM's shot goes elsewhere on purpose
🎭 A 4-person band, no mix-ups: actions the LLM gave to the wrong member ("the bassist keeps striking cymbal") become their own instrument; a shot naming nobody ("Close-up of her face") gets the right person; only the singer sings, the others keep their mouths closed
🌟 As much of the main character as possible: about two of every three shots frame the main character, in front and in focus, with the band behind - the opening shot of each scene too, and a close-up on "her hands" is the main character's (a whole scene once showed only the drummer)
🎥 A close-up stays on its person: the others' lines leave that shot, and it says the camera stays there to the shot's end (H3 kept cutting away to the bassist and the drummer)
🎭 No second drummer: a person the LLM adds who plays a cast member's instrument becomes that member; the non-singers keep their mouths closed in the retention lines too
🎬 The whole video's idea is on the Timeline now. Until this update that prompt was a box on the canvas only, and the workflow shipped with a sample in it ("She picks up the Stratocaster … No singing.") that ran for every scene even when you worked from the Timeline alone. The box is shown next to each scene's words, ships empty, and a scene's own words come first where they differ. If you use a workflow saved from an earlier version, open the Timeline and clear or rewrite that box.
💃 Dancing, and dancing while singing: the cast part
danceshas a partner,dances + sings. She waits with closed lips until the vocals come in and sings in lip sync like any singer, and every dancer gets a line per shot: "dances on the beat of the song, every step and turn in time with the music" (not in close-ups). It follows the beat; it is not choreography beat by beat, and H3 decides the moves. The cast card decides who sings, and a cast in which nobody plays gets no instrument cues.👀 A look for each face:
lookon the cast member ("long silver hair, black hoodie") goes into H3's definition line and the LLM hint, so four women stay apart in close-ups (a real run showed the bassist's face in the main character's close-ups). One unnamed woman in a shot is the main character; the drummer's kit paragraph is written only in scenes whose shots show her; the singer's mouth agrees with the shot's singing state.
✂ Editing the scenes, 💾 less RAM (v8.2, ABCScoreIO v1.69)
⏱ Every scene's own seconds, like 🎞 H3 Timeline: drag a clip's right edge and that scene gets longer or shorter while the scenes after it keep their lengths and move along. Alt + drag moves only the cut. Or type it into ⏱ this scene [ ] s. Once seconds are set by hand, add after / duplicate / delete / ◀ ▶ leave the other scenes' lengths alone as well. The song is the limit (the video never runs past its end). ↺ All scenes automatic goes back to the cuts on the song's changes. C splits at the playhead.
🆕 The shipped workflow starts empty: no picture in scene 1, nothing in the whole video's idea.
✋ The end of the video is a grip too: drag the right edge of the last scene to set its length (with one scene, the only length there is). Snaps to bars and to the singing; ↺ puts it back to automatic.
✂ Editing the scenes on the Timeline (taken over from H3 Timeline v1.3.1): add after the selected scene, duplicate, split at the playhead (the second half continues with ⛓; the two halves are exactly the old scene), Shift / Ctrl + click to select several and delete them at once, a right-click menu on the clips. Up to 64 scenes.
💾 More scenes cost disk, not RAM:
decode = streamwrites each 17-frame chunk of H3's video VAE straight to the scene's file (one chunk in RAM instead of one scene, the same pixels), and each scene's latent is kept in a small file next to its frames.These two came over from my other workflow, H3 Timeline v1.3.1. Here the scenes are one list, not nodes, so adding scenes puts nothing on the canvas.
decodeis on 🎬 Make every scene (defaultfull, as before); trystreamon one scene first.
🥁 The drums (v8.2, ABCTiming v2.55–v2.59)
🥁 stick_swing = big on the Drum Cue Style node (and the Timeline settings): the drummer plays like a rock drummer, every hit a full swing from above the head (a real run's drummer barely moved the sticks)
🥁 A drum solo, hit by hit (ABCTiming v2.59): a freshly made YuE2 drum solo with no steady pulse got "keeps a steady beat, snare on 2 and 4" and H3 drew a metronome. When the drummer is the only player and the hits do not cover the beats, the drum sync's hits are written as they are, timed, and the silences as "holds both sticks still"
🦶 The feet, timed (ABCTiming v2.59, feet on the Drum Cue Style node): each shot says when the right foot presses the bass drum pedal (the kick times) and that the left foot stays flat and still; or "hide in wide shots" lets the kit hide the feet in a wide shot (a real FastH3 run moved both feet together)
🥁 The beat's kick words point to the timed kicks (no more "on every beat" next to irregular kick times)
🥁 In a close-up on someone else the drummer plays in the background, in one line (she had ~3x the words and ~40 % of the screen)
🎤 Who sings
From a real run: in an instrumental the guitarist opened her mouth at the microphone. When the prompt says nobody (on screen) sings, a "guitar + sings" cast part is written "who plays the guitar, lips closed" for H3, and the singing words the scene LLM wrote into the shots are taken out before ✍ review; one put back in review is flagged. Songs with vocals are not touched
The closed lips go with the person who lost the singing ("S1 strums the guitar with the lips closed, S2 plays the drums"), and "S1 does not sing but S2 sings" keeps the denial and still closes S2's lips
A real 17-scene run: a pre-made song with vocals and "歌は歌わない" in the main prompt became an instrumental for every scene. Now the score's Vocal voice decides (notes = sung, all rests = instrumental); the words only say whether the person on screen sings. A disagreement is shown in 🔍 Details
The main prompt is every scene's idea: "does not sing" there keeps every mouth closed - write it in scene 1's own text on the Timeline to keep it to scene 1
🎬 Shots and cues
The scene LLM's shots are not lost: times written as "4.5s", "00:04.500" or under "time" / "start" are read, and a shot list with nothing usable is asked for once more. 🔍 Details says what happened (a 2-scene run had no LLM shot in scene 2)
The LLM's own times in the words ("..., singing begins at 13.6 s" while 🎼 ABC Timing put the singing at 00:12.380) are dropped, that clause only; 🔍 Details shows them
No repeated cue: "the music moves to verse, vocals" three times in one shot (ABCTiming v2.55) - a cue only where the song part or the singing / melody really changes
A shot's times always increase, and "the intro starts" is never said twice
✋ Cuts set by hand
On the Timeline, Alt + drag the edge between two scenes (a plain drag sets the left scene's length, see ⏱ above) to set where the next one starts - its picture on the singing coming in. It snaps to the singing entries, the song parts, bars and beats (Shift: free); the other cuts stay automatic around it. ✋ on the clip, ↺ Back to automatic in the scene box; the run uses the same cuts
Also since v7.0
🎵 Music brief per scene (tempo, song part changes, singing / melody, energy with silences and short hits, how the scene before ended)
🖼 Each scene records whether its LLM saw the picture (and why not); use_image = required stops instead of going on with the text only; "each scene needs its own picture" stops before the run
Scene cuts reach the song's end, never go past it, keep bar lines (with a warning when they can't), and a "whole song required" switch
Fixes: scene 1's check video lost with 7+ scenes, missing picture files caught before the run, resampling without torchaudio
💡 Tried: Singularity v1.3 for faces
In a 4-person band, the official H3 int8 once drew the keyboardist's face in a close-up meant for the singer. The fine-tune Minimax-h3_Singularity_ref2va_Pruned_v1.3_int8 (same size as the official pruned int8) drew the right face in the same scene and seed. To try it: 🚀 engine = h3, 📦 H3 model = that file, ⚡ Turbo off, 20 steps
A later 18-scene run with it still drew the bassist's face in five of the main character's close-ups, so the model alone does not settle it: that is what the new 👀
lookfield is for. Check the lip sync by ear (v1's audio-reference path is said to be weaker)
⚙ New defaults
The workflow's defaults are now the author's run: adaptive drum cues, big stick swings, FastH3 Omni 8 steps (NVFP4), int8 video VAE, no Turbo LoRA, structured ref2va prompt
Updating from v7.0
Replace the four custom node folders (ABCScoreIO v1.69, ABCTiming v2.59.17, PromptSplitter v3.5, ABCInstrumentalize v1.5), restart ComfyUI, reload the page (Ctrl+F5) and open the v8.2 workflow
FastH3 Omni needs a recent ComfyUI (with ModelSamplingMiniMaxH3 and Model Sparse Attention); Standard and ⚡ Turbo run on older versions too
STRUM (the drum transcriber) is still recommended for the drums: see STRUM_INSTALL.md in the zip
Not yet checked on a real GPU run: 👀
look,decode = stream, the ✂ scene editing, ⏱ the seconds set by hand, 💃 the dance parts and more than 24 scenes. They pass the tests (1,038) and the real ComfyUI screen; tell me in the comments if something is off