This is the SVI 2.0 PRO version.
v3.5 12-section fixed - An error was discovered where the prompt that should have been entered in "Section 8" was incorrectly duplicated with the 6th prompt. This issue has been fixed, so please reinstall the workflow.
Thank you so much to @MarcanOlsson for discovering the issue!
v3.5 12-Sec - This version is based on v3.5 and enables video merging through 12 generation steps.
When actually generating the video, you will notice color shifts compared to 6 generation steps. While 6 steps generally maintain better quality in practice, the ability to specify 12 steps offers advantages depending on the application, so we decided to add this version.
The 12-Section version enlarges the workflow, so unless you need more than 7 generation steps, I recommend using the standard v3.5.
v3.5 - The subgraph specification in the model input area has been deprecated and reverted to the v2 specification. Additionally, it is now possible to generate videos for only the first section.
We received multiple reports from the community that models such as CLIP and VAE were not functioning correctly due to the subgraph, and we also received feedback that the model placement was unclear. Therefore, we decided to revert to the v2 specification.
However, the subgraph of the generation section, which includes the sampler, remains unchanged. We believe that performing the generation process within the subgraph serves to prevent a decrease in generation quality. While the model area issue is simply a layout issue, the subgraph cannot be removed because it affects the quality of the generation section. If there are issues with the subgraph itself, please avoid using this workflow.
Regarding the video generation for only the first section, given the nature of SVI, we initially omitted it, believing that a single generation was unnecessary. However, we received feedback from the community requesting that a video be generated for each section, and that videos be added gradually while reviewing the generated videos. This was a very logical approach, so we added the "first video" and modified the workflow to allow videos to be accumulated while keeping the seed value fixed.
v3.4 - Layout adjustments.
v3.3 - Changed the seed node from "CR Seed" to "Seed (rgthree)". This change was made to align with commonly used custom nodes in this workflow, following reports of implementation issues with CR Seed.
v3.2 - Modified the layout to make it easier to disable Lightx2v Lora.
v3.1 - Modified the layout to make it easier to disable the Sage Attention node.
v3.0 released.
Video length can now be changed in each of the six generation sections, providing more flexible control over video content.
The frame rate (fps) was previously fixed at 16fps, but can now be changed arbitrarily. Accordingly, the RIFE-VFI node's scaling factor can now be changed in the input area.
GGUF model loader is now included as standard.
Version 2.0 changed the number of generation sections to six.
The layout has also been updated, allowing Seed node input to be processed in one place. Furthermore, the layout has been significantly redesigned to unify the user experience with Painter I2V versions, reducing the input burden. With this change, the wildcard prompt input method has been discontinued.
Please note that the explanations in this workflow are solely my personal opinions. I do not have expertise in AI generation, so some information may be inaccurate.
The main goal of this workflow is to achieve compact operation when performing repeated generation. It minimizes screen scrolling during operations such as prompt input, input image selection, specifying time, number of steps, resolution, and, most importantly, LORA selection. To further enhance compactness, all nodes are fixed to prevent accidental operation.
Links to the Models and LORAs and nodes used in this workflow
SVI LORA :
Wan Advanced I2V (Ultimate) :
https://github.com/wallen0322/ComfyUI-Wan22FMLF
This node was updated on January 27th, but the version available for installation from ComfyUI Manager may be an older version. While the older version will still work, you won't be able to set "SVI Motion Strength," and you'll likely experience more color misalignment. Therefore, if you can Git clone, we recommend installing the latest version.
Links to the Basic models of the Wan2.2
CLIP:
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/text_encoders
VAE:
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/vae
CLIP Vision :
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/clip_vision
You can generate up to six generations, and each generation is assigned a unique seed value. Normally, click "Randomize Every Time" to display "-1". Generation will be random. In this case, the seed value for each generation will be displayed at the bottom of the screen. If you want to fix the seed value, click the seed value field or enter the seed value directly. For example, you can fix the first and second generations and regenerate the third generation and beyond randomly. However, regenerating a section before the generation section you want to fix will change the final frame, so you cannot fix subsequent sections. As a general rule, regenerate after the section you want to fix.
By combining the six generated videos, you can create six different types of movement. For example, by generating and combining six videos of different durations, you can create a long video containing six complex movements. This is one of SVI's strengths, enabling complex processing that is impossible with a single generation.
However, SVI V2.0 PRO also has its drawbacks. Because SVI uses the first image as a reference point, the AI tries to restrict movements that deviate significantly from the reference point. As a result, the movement becomes sluggish and unnatural. Furthermore, this constraint imposed by the reference point also reduces the responsiveness to prompts.
In short, the use of excellent LORA is essential in SVI. In my experience, movements without LORA are very unnatural, lack impact, and resemble something out of a horror movie. Fortunately, there are many excellent adult-oriented motion LORAs available. However, if you want to create completely original movements, expect it to be difficult with the current version of SVI.
I hope this workflow helps make video production with SVI more enjoyable.
Description
v3.4 Alt - This is an another version of v3.4 that does not use the subgraph in the model input area. We received reports from the community that the subgraph in the model area was causing problems, so we decided to release a version with the subgraph removed.
Please try using this version if you are experiencing errors with the regular v3.4. Also, please be sure to update the ComfyUI core and custom nodes to the latest versions. Numerous bugs have been reported in older versions.
FAQ
Comments (61)
Thx 4 update, but subgrafs and problems with it is still there =(
Remove ALL subgrafs in wofrkflow is required.
The subgraphs in each section affect the quality of the generated output. Since subgraphing helps to minimize quality degradation, removing them has a different meaning. This is different from the model section.
If problems arise with the subgraphs themselves, unfortunately, you should assume that this workflow is unusable.
@kenpechi Unfortunately, the problem lies in the subgraphs themselves — after each ComfyUI major update, they break and begin to confuse the directions of the values being passed (in your case, the number of steps is being added to add noise, for example), which leads to the entire workflow not working properly.
The solution in this case is to unpack all subgraphs, check the correct path of the arguments being passed, and stop using subgraphs in next workflow versions. I've already done this and everything started working fine for me.
Yes, this will make the workflow visually bulkier and less elegant, but it will prevent it from breaking with each subsequent Comfy update. =(
@PerfectLoop What could be causing the problem with subgraphs? Subgraphs are probably used in most workflows these days.
In fact, I haven't experienced any problems.
Why?
In any case, I don't intend to completely eliminate subgraphs. Like you, it's possible to solve it yourself. I'm sharing my information with evreyone, but I don't need to help everyone personally.
Hi Kenpechi
Is it possible to add , VRAM-Cleanup nodes like KJNodes or ComfyUI-Memory_Cleanup in the workflow?
I ran out of memory.
Yes, it's possible. In my workflow, I only clean up the VRAM before the final RIFE-VFI, so I think it would be best to go into the subgraph of each section and connect your preferred VRAM cleanup node after the VAE Decode node.
@kenpechi ok thank you. I will try this !
Possible silly question. The results in is page look amazing. How do you guys get the image for the i2v please?
I've posted several images in the "image" section of my profile. Metadata is also embedded, so please check it out.
I appreciate your answer, what I was meaning was, to create an image normally, do you use t2i workflow/models for that?
@roger3435331 Sorry, you can't see the workflow.
It's just a standard t2i workflow with a face detailer added. It's nothing special, nothing worth making public.
I understand that is ok. I try your workflow and its excellent, thanks a lot. When making a video, is there a way with your workflow to do video 1 only then append video 2. then append video 3 etc? As it is now I think we need to do all of them in one run, or am I missing something please?
@roger3435331 It's generally possible, but since videos 1 and 2 are designed to be created together, it's not possible to extract only video 1. If you'd like, you could copy the first video yourself and create it somewhere else. After that, you should be able to do it by adding sections while keeping the seed value of the created video fixed.
However, this would need to be done manually. I don't use such methods because I generate them while I'm sleeping.
@roger3435331 We will make the necessary changes in v3.5 to address your request. Thank you for your feedback.
@kenpechi That is great thanks so much. I have been using the workflow and its excellent quality. A quick question is this, the end video is 13 seconds and that is after 6 sections. Is there a setting to make each section longer (example 6 x 5 seconds)?
@roger3435331 If you set the values for "1st second" to "5th second" at the top of the "input" area to "5," for example, you should get a video of approximately 28 seconds, consisting of 5 seconds x 6 segments. (This subtracts the 30 overlapping frames of 5 frames x 6 segments.) I haven't tested it extensively, but it should work fine as long as each section is 5 seconds or less.
@kenpechi Thanks, yes that works perfect. I reduced the resolution to 480x832 to speed it up a bit. Will try the upscale later.
@kenpechi The upscale works. I assume we only need to enable the upscale for the last video in the selected sequence? Also, are you considering a T2V version of this great workflow?
@roger3435331 That's correct.
Unfortunately, I've never generated anything using T2V.
@kenpechi Thanks, interesting you never user T2V, it is good I think when using self-trained loras. Might it be possible to consider an option switch in the fast groups bypasser to enable T2V, just a suggestion for your excellent workflow.
@roger3435331 If we're using it with T2V, does that mean we'll use the last frame of the first generation as the anchor image?
@kenpechi From google ---> Image-to-Video (I2V) generates motion from a specific, user-provided image, ensuring high character/scene consistency, while Text-to-Video (T2V) creates video from a text prompt, offering higher creative variety. Standard T2V does not use a user-defined anchor image, but rather synthesizes the first frame from scratch.
@roger3435331 Oh, I'm sorry. You meant just T2V, not T2V using SVI?
@kenpechi Yes just T2V
@roger3435331 In that case, there's no need for my workflow. There are plenty of other excellent workflows out there.
Both of my workflows are specifically designed for I2V.
Also, if it's just about layout, there's no need to go to that extent.
@kenpechi Thanks, yes that makes sense. I forgot to ask, you said there are images on your profile page (Image section), they are all videos I think, I was hoping to see what workflow/models you used for making the asian girls static images (they look excellent). Thank you for your help!
I'd like to use 3.4 with subgraphs. I'm running current stable version of comfy and I've uploaded nodes.
When I click the subgraph for checkpoint, it takes me to the subgraph, but I only see limited nodes. There are no nodes there to actually select my checkpoints. Any idea how to correct?
Same for the "Other Models" subgraph - it takes me to the sub, but nodes are missing and I cannot load the clip, vae, etc.
Have you tried v3.4 Alt?
@kenpechi I've not, but I was hoping to get non-alt to work. You have great images with WFs embedded that would be great to drag and drop for all of the loras and weights. It would be ideal to not have to try and change all of those vs. drag and drop and good to go.
@MankyPoodle You're not the only one experiencing problems with subgraphs, but I don't really understand why. If your computer's specs are sufficient, I think it should work. (It might not work with low RAM or VRAM.)
I don't recommend it, but a clean install of ComfyUI might be an option. Some people have solved the problem that way.
@MankyPoodle We are releasing a version that eliminates the subgraphs in the model input area. As a result, I will also be generating models using this new version going forward.
@kenpechi that would be awesome!
@kenpechi I'll keep an eye out for the new samples. The ones I see now are still using the subgraph.
Thanks for your hard work on this
How can i change the video lenght? cant find the option. Only getting 5 seconds
Are you using v3?
Are there input nodes labeled "1st second" through "6th second" at the top of the "Input" area?
I activated all 6 inputs on the "Fast group bypasser" and got an 11 seconds video. Is 11 seconds the max?
@anzzio487 That's completely wrong.
You should abandon this workflow and start over with the official I2V workflow.
First, try using the official workflow multiple times.
Come back here once you understand what a workflow is.
If anyone is having slow motion issues using base wan, the two things that helped were better prompting (mild help) and using a triple ksampler workflow (almost too much now sometimes but easier to modify). I swapped out the two ksamplers for the Vraethr tripleksampler custom node and it helped a ton.
hi @kenpechi
In your workflow, it is suggested for rife4.25+,
is this the correct file? "RIFEv4.25_0919.zip" https://huggingface.co/r3gm/RIFE/tree/main
thank you!
I install a custom node called "Frame Interpolation" from the ComfyUI Manager.
That's probably easier.
First of all thank you for sharing this!
This might be a beginner question, but is there a way to generate videos only with a reference image? Anytime I try to run it it requires a video
This is an image-to-video conversion, so if you have a reference image, a video is not necessary.
What exactly is the error?
Any luck on adding more samples under v3.5 with embedded WF?
Hi
Thanks for your work
I have a question: can you confirm that changing the prompt slows down the generation speed - recreate wf?
Every time I change the prompt, the generation speed triples, but if I keep the current prompt and change the seed, the speed is fast
I'm not an expert, but I think ComfyUI speeds things up by using the cache from the previous generation, so changing LORA or the prompt would make it slower because the cache can't be used. Especially since this workflow loads data multiple times (up to 12 times), it might feel even slower.
However, I think this isn't a workflow-specific issue, but rather a ComfyUI specification.
Incidentally, a three-fold slowdown probably means something like the generated data is overflowing RAM. For me, it becomes more than twice as slow when a section exceeds 7 seconds.
I guess I misunderstand for SVI, can you please explain why the 1st two sections are together and generate one video and not generate one for the 1st and one for the second, please? Also there is a fade transition between those 1st two. Is the reason we set up say a photo of the main subject for video 1 then the desired action for that subject starts in video 2?
That issue has been resolved in v3.5, hasn't it?
Sorry , I not saying there is an issue. I asking why the 1st two sections are together and how that nice fade transition works (never seen that before) and I asking is the idea that the main character in video 1 is then in video 2 (sorry about my lack of understanding) I guess I am asking why v1 and v2 are wired as they are, thank you
@roger3435331 I'm sorry, but what do you mean by "fade transition"? I'm probably not aware of what you're referring to at all. I combined 1 and 2 simply because SVIs are best appreciated through repeated generation, so I omitted the generation result only the first time. There's no special generational meaning to it.
Great thanks for the reply. When I use a solo character for the image (and say make character smile in v1) then for v2 prompt describe some action, I always see a fade in and out between v1 and v2. Its very nice and I wondering where that comes from. So instead of the scene changing in one frame the v1 fades out and v2 fades in. Thanks
@roger3435331 Are you talking about the "Image Batch Extend With Overlap" node? The node that overlaps images by 5 frames? This is a standard node in SVI. However, I tried it in other applications, and it probably won't work.
@kenpechi I did not know about that node "Image Batch Extend With Overlap" until you just mentioned it, but I see it now. For me that node (linear_blend) works in the second section (only) every time. I see that linear_blend effect between v1 and v2 which is really good. Thanks for explaining.
Is it possible to consider creating a loop video?
In one of the wf, you could upload the resulting video and it would create a looping animation
That's not possible.
Please use a different workflow.
@kenpechi What do you mean, “it's not possible”?
The question was whether you could consider creating a new workflow.
@MugenMan It means I haven't considered it at all. There's no need to expect it; you should look for a different workflow.
Is there a slight issue please? It seems the processing takes longer for each stage, example stage 5 take a lot longer than stage 3. is this is caused by where each successive stage is passing the entire accumulated video tensor through the RIFE VFI (Video Frame Interpolation) node. As the video grows in length, RIFE is re-interpolating previously processed frames, leading to redundant computation and potential VRAM swapping.
Is it possible to refactor the workflow to use segmented interpolation. Each stage must process only its unique frame set before joining the segments for the final output. I might be wrong, if i am then sorry but wonder if the above will help. Thanks
I didn't anticipate your method of generation, so I won't be doing it.
You're free to improve it as you see fit.
Thanks. Do you mean because I have all 6 sections enabled then that was not an anticipated method of generation, please?
@roger3435331 All "videos". I only enable the "video" with the same number as the last section, so the RIFE VFI processing is done only once.
@kenpechi Thanks, Ah, I did not rtealize that is how we supposed to use it, sorry, I will give that a try