Finally achieved, for the first time ever, an infinite loop that did not require editing. Wasn't even my goal with this workflow, it just suddenly occurred to me that it would work. Keyframing the final stage is a little tricky with this custom node, so I will post it as a version if switching is a problem. Indices apply only to section lengths, so you need a node in the end group. I'm going to try and set up optional keyframe injection for each group. See embedded WF of the post that loops in the meantime.
Please experiment with model/LoRA/sampling combinations! The defaults here are for speed. If you are experiencing degradation with I2V, put your image into the ref2 spot along with the normal spot. It helps a LOT (ref1 prompt is for facial identity - not that great yet with I2V mode).
I'm leaning towards hybrid ref/flf models, I've tried a few and they are testing well. Default sampler is er_sde but simple euler is of course faster, might be superior too, haven't got through the whole gauntlet yet. All tests so far have been with heavy distillation, or I'd never get through all the iterations. As you know, the appropriate sampling method will vary with distilled vs full. If you want to fly through tests of all 7 stages, 960x544, 4 steps, 10 second chunks takes me only 6 minutes to run and puts out a 1 minute clip (on T-800, neural net processor, a learning computer). Good enough to give you an approximation of a full run.
Here's my take on H3 infinite extension using the awesome ComfyUI H3 Motion Context custom nodes. This particular pack is a fork of this one. I can't speak to the original or the other forks, but this one is just great. It handles exactly what it needs to in the context department, with the added bonus of automated routing via mode-dependent bypassing. This is the answer to the unfortunate frame limits of H3. This model really can do everything, with the right nodeage and promptolatizing. Much fun. Awesome work, ditto for the original author of course. Thank you to the node builders who know what they are doing. I sure don't. Use whatever H3 REF version you like. I’m using the pruned int8 convrot with the 4 step turbo. Surely you don’t need links for this. The prompting flow is a bit convoluted, I'll stick notes at the bottom of the description for any non-trivial quirks I find.
You can load an existing video to continue it, generate the first clip I2V style, or do it fully T2V, using any combination of the two image and two audio slots. Keep them bypassed when not using them. Please note that the keyframe image loading (I2V mode) is completely independent of the ref inputs. They are NOT the same. I have used more than one keyframe, but default here is one, there is a minimized note on indexing. I've capped this at six extensions, so a full run here will be seven clips combined. I'm posting an example of each mode, they are of varying quality, as they are from the WF at various stages of the build, and may have artifacts due to an incorrect LoRA (*see the note on that below).
I used the example workflow provided in the repo as my starting point. It's a huge mess of wires, but that's acually good thing in an example WF, you can see exactly how everything is laid out. I've added get/set nodes to clean up the majority of the flow, but the core architecture is untouched. As is my wont, I've focused on automating as much as possible the prompting side of things. I prefer to use help from a prompt enhancer/vision model combo running locally, but I did not include that here, it's way too convoluted and specific to my setup. If you upload the json to the LLM of your choice along with your ref, it will give you decent prompts for all seven slots. That saves a lot of time, the tedious rebuilding stuff doesn't have to be specific, just amend as needed. Or argue with it until it gets it right.
There are lots of prompt replace nodes and concatenation, if it's hard to follow, queue up a run and take a look at the Proofreading node in each prompt section and you will see how it is put together. When using reference images or audio, use the tags I created to simplify repetitive instructions The tags are listed in the title of the Global Reference Instructions group.
MODES:
Existing Video - Uses your source 1:1, @24fps. Prompts start at #1.
T2V - Prompts start at #0. Use ordinary T2V prompt to start.
I2V - Uses your source(s). Add more keyframes if you like. Prompts start at #1.
Ref images/audio can be added in every mode. Prompt for 1 is STRONG, for 2 is weaker.
Audio 1 is set up for source voice clone, use a GOOD voice sample.
Audio 2 is set up for imitating source music.
All instructions for continuity are PREPENDED, APPENDED or INSERTED with global settings.
See the appropriately titled group to change them.
I have tested all modes and they all work GREAT for me. These custom nodes really are awesome. Please not that you MUST have the comfyui v0.34.0 or later for all functions to work. If the nodes installed fine but are still throwing errors at you, you know what to do. Don't forget to back up a full copy of your constrictor folder any time you change anything. The time you save may be your own.
*The original workflow as posted uses the 8 step turbo LoRA for the FLF model. I do not know if this was intentional or not. I have changed it here to the 4 step turbo LoRA for the REF model, with appropriate changes to sampling. I think that some audio and visual artifacts that show up in my example videos may be due to using the FLF LoRA. I didn't notice it right away.
Give it a try, this setup really isn't crazy complicated at all. Most of the tangle is promptomania, the extensions are identical copies, the latents just pass through and they call the appropriate prompts. I've not added upscaling and interpolation yet - it might get a bit demanding on resources. My latest testing with I2V at 1344x768, 10 second chunks, is spitting out 46 second long finals in ~13 minutes. So less than two minutes per chunk. It should take you however long an H3 run takes you multiplied by the number of chunks, roughly.
Please do let me know if you find bugs in my swamp of selectable string stream. I'm sure there are plenty, but I'm keen to share this stuff. It's getting absurdly easy to make longer videos, yet all I ever see on this site is the same old 3-5 second slop. Please make your slop longer! You're running out of excuses.
Custom nodes in WF (other than the core listed above). If you need help substituting for alternative nodes, I’d be happy to give you options, though an LLM will give you an instant answer of course.
If you choose to bypass Global Continuation Instructions Detailed, it will prepend 'continuation instructions' at the beginning of the prompt. This is due to the behavior of the concatenation node. I left that string in the concatenation box so it would appear in a promoted widget, but since a get node is connected to it you wouldn't see it anyway. So if you want to eliminate any prepend, however small, you can delete the string in the global prepend group. Or connect an empty string node, if you want to preserve the string.
I've been experimenting with it on/off. Sometimes the extra instructions help, sometimes they confuse the model and reinforce what you are trying to neg out (pink elephant stuff). A lot depends on the mode and the refs.
I haven't set up a flow that uses an abliterated instructor on the same machine. I've been running the rewriter on a different local machine, with a comfy instance and ollama- sending references and old LTX/WAN prompts or the original image prompts to that instance for VL+heretic rewrite with H3 rules. It's still an error generator half the time. Much easier when you don't have to worry about a recalcitrant instructor. For anything tame, an online LLM will spit out good prompts for all seven groups of this WF, especially if you show it the json and your ref image(s).
Damnit how do I always end up writing a novel in here? Sorry. Most of this stuff is probably obvious to anyone who's been playing around with this model, or who is even halfway familiar with comfy. If it's not, uploading the .json to any good LLM with give an instant overview, answer any questions, and point out all the embarrassing mistakes I made.
Description
This is pretty stable. A few configurations of the custom nodes are actually ahead of the latest stable comfy, if it throws errors with multiple keyframes + ref audio, this is probably why. Probably best to stay away from betas though, just wait for release.
I might have made a mistake on the model connection in the uploaded file. The connection order should be Model>LoRA> Attention>Preview. It will still run if LoRA doesn't go through attention node, but you should check that and fix it if needed.
