This is the SVI 2.0 PRO version.
v3.5 12-section fixed - An error was discovered where the prompt that should have been entered in "Section 8" was incorrectly duplicated with the 6th prompt. This issue has been fixed, so please reinstall the workflow.
Thank you so much to @MarcanOlsson for discovering the issue!
v3.5 12-Sec - This version is based on v3.5 and enables video merging through 12 generation steps.
When actually generating the video, you will notice color shifts compared to 6 generation steps. While 6 steps generally maintain better quality in practice, the ability to specify 12 steps offers advantages depending on the application, so we decided to add this version.
The 12-Section version enlarges the workflow, so unless you need more than 7 generation steps, I recommend using the standard v3.5.
v3.5 - The subgraph specification in the model input area has been deprecated and reverted to the v2 specification. Additionally, it is now possible to generate videos for only the first section.
We received multiple reports from the community that models such as CLIP and VAE were not functioning correctly due to the subgraph, and we also received feedback that the model placement was unclear. Therefore, we decided to revert to the v2 specification.
However, the subgraph of the generation section, which includes the sampler, remains unchanged. We believe that performing the generation process within the subgraph serves to prevent a decrease in generation quality. While the model area issue is simply a layout issue, the subgraph cannot be removed because it affects the quality of the generation section. If there are issues with the subgraph itself, please avoid using this workflow.
Regarding the video generation for only the first section, given the nature of SVI, we initially omitted it, believing that a single generation was unnecessary. However, we received feedback from the community requesting that a video be generated for each section, and that videos be added gradually while reviewing the generated videos. This was a very logical approach, so we added the "first video" and modified the workflow to allow videos to be accumulated while keeping the seed value fixed.
v3.4 - Layout adjustments.
v3.3 - Changed the seed node from "CR Seed" to "Seed (rgthree)". This change was made to align with commonly used custom nodes in this workflow, following reports of implementation issues with CR Seed.
v3.2 - Modified the layout to make it easier to disable Lightx2v Lora.
v3.1 - Modified the layout to make it easier to disable the Sage Attention node.
v3.0 released.
Video length can now be changed in each of the six generation sections, providing more flexible control over video content.
The frame rate (fps) was previously fixed at 16fps, but can now be changed arbitrarily. Accordingly, the RIFE-VFI node's scaling factor can now be changed in the input area.
GGUF model loader is now included as standard.
Version 2.0 changed the number of generation sections to six.
The layout has also been updated, allowing Seed node input to be processed in one place. Furthermore, the layout has been significantly redesigned to unify the user experience with Painter I2V versions, reducing the input burden. With this change, the wildcard prompt input method has been discontinued.
Please note that the explanations in this workflow are solely my personal opinions. I do not have expertise in AI generation, so some information may be inaccurate.
The main goal of this workflow is to achieve compact operation when performing repeated generation. It minimizes screen scrolling during operations such as prompt input, input image selection, specifying time, number of steps, resolution, and, most importantly, LORA selection. To further enhance compactness, all nodes are fixed to prevent accidental operation.
Links to the Models and LORAs and nodes used in this workflow
SVI LORA :
Wan Advanced I2V (Ultimate) :
https://github.com/wallen0322/ComfyUI-Wan22FMLF
This node was updated on January 27th, but the version available for installation from ComfyUI Manager may be an older version. While the older version will still work, you won't be able to set "SVI Motion Strength," and you'll likely experience more color misalignment. Therefore, if you can Git clone, we recommend installing the latest version.
Links to the Basic models of the Wan2.2
CLIP:
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/text_encoders
VAE:
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/vae
CLIP Vision :
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/clip_vision
You can generate up to six generations, and each generation is assigned a unique seed value. Normally, click "Randomize Every Time" to display "-1". Generation will be random. In this case, the seed value for each generation will be displayed at the bottom of the screen. If you want to fix the seed value, click the seed value field or enter the seed value directly. For example, you can fix the first and second generations and regenerate the third generation and beyond randomly. However, regenerating a section before the generation section you want to fix will change the final frame, so you cannot fix subsequent sections. As a general rule, regenerate after the section you want to fix.
By combining the six generated videos, you can create six different types of movement. For example, by generating and combining six videos of different durations, you can create a long video containing six complex movements. This is one of SVI's strengths, enabling complex processing that is impossible with a single generation.
However, SVI V2.0 PRO also has its drawbacks. Because SVI uses the first image as a reference point, the AI tries to restrict movements that deviate significantly from the reference point. As a result, the movement becomes sluggish and unnatural. Furthermore, this constraint imposed by the reference point also reduces the responsiveness to prompts.
In short, the use of excellent LORA is essential in SVI. In my experience, movements without LORA are very unnatural, lack impact, and resemble something out of a horror movie. Fortunately, there are many excellent adult-oriented motion LORAs available. However, if you want to create completely original movements, expect it to be difficult with the current version of SVI.
I hope this workflow helps make video production with SVI more enjoyable.
Description
v3.5 - The subgraph specification in the model input area has been deprecated and reverted to the v2 specification. Additionally, it is now possible to generate videos for only the first section.
We received multiple reports from the community that models such as CLIP and VAE were not functioning correctly due to the subgraph, and we also received feedback that the model placement was unclear. Therefore, we decided to revert to the v2 specification.
However, the subgraph in the generation section, including the sampler, remains unchanged. We believe that the subgraph plays a role in preventing a decrease in generation quality by executing the generation process within the subgraph. The issue in the model area is simply a layout issue, but it affects the quality of the generation section, so it cannot be removed. If you are experiencing problems with the subgraph itself, you should avoid using this workflow.
Regarding video generation for only the first section, initially, due to the nature of SVI, a single generation was not anticipated, so it was omitted. However, the community requested that videos be generated for each section, allowing for gradual addition while reviewing the generated videos. This was a very reasonable approach, so we added "1st video" and modified the workflow to allow video accumulation while keeping the seed value fixed.
v3.4 - Layout adjustments.
v3.3 - Changed the seed node from "CR Seed" to "Seed (rgthree)". This change was made to align with commonly used custom nodes in this workflow, following reports of implementation issues with CR Seed.
v3.2 - Modified the layout to make it easier to disable Lightx2v Lora.
v3.1 - Modified the layout to make it easier to disable the Sage Attention node.
v3.0 released.
Video length can now be changed in each of the six generation sections, providing more flexible control over video content.
The frame rate (fps) was previously fixed at 16fps, but can now be changed arbitrarily. Accordingly, the RIFE-VFI node's scaling factor can now be changed in the input area.
GGUF model loader is now included as standard.
FAQ
Comments (142)
omg so huge, how should I understand it :D
First, specify the model in the area at the top left. Then, specify the video settings and LORA in the input area. Finally, enter instructions in the prompt area in the middle. You don't need to worry about the right side.
This workflow prioritizes completing the video settings without scrolling too much on the screen, so it is a bit complicated, but I think it is more organized than many workflows.
Unsure what different with this and your other workflow I was using last night . But with this new one im getting a error " cuda" or something like that. I did try looking in the workflow for the sagenode to disable it but wasnt able to locate it if its in the workflow. Only reason for me saying about the sagenode as in some on my other workflows I get the cuda error and once I disable the sage node it works lol. Love your work keep it up.
Thank you.
Thank you for pointing that out. I see, I was mistaken. I may release a revised version.
You can see the contents of sage attention by clicking the square in the top right of the Diffusion models subgraph node in the upper left, so try bypassing the sage node.
By the way, this is basically the same as v2, as the only change is that the generation time can now be specified in more detail.
@kenpechi your most welcome, thank you for the fast response
I was confused why outputs was green and bad quality...
you linked the SVI repo and their lora dont work with comfyui.
Please update your SVI lora links.
https://huggingface.co/Kijai/WanVideo_comfy/blob/main/LoRAs/Stable-Video-Infinity/v2.0/SVI_v2_PRO_Wan2.2-I2V-A14B_HIGH_lora_rank_128_fp16.safetensors
CUDA error: invalid argument Search for cudaErrorInvalidValue' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information. CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA` to enable device-side assertions.
1.0没有这个问题 在处理2步的时候提示
I'm sorry. This is because we changed the way we handle sage attention. We will release a revised version today, so please wait.
I haven't tried this WF yet, but, if I do have sage attention installed, it should work fine? This error is only for people who don't have sage attention?
@artificialotaku Other people are having issues with sage attention, so I'm currently having issues, but I'm not sure if this error is caused by that. I'll try to fix that for now.
@kenpechi got it, I do remember installing sage attention on ComfyUI was a major headache, but after I installed it, everything was fine... I'll try your WF and report back here if I have problems with it, even having sage attention installed.
@artificialotaku Thank you very much. I would appreciate it.
@kenpechi 期待4.0
@lemon95212 抱歉，我已经更新到 v3.1 版本了（哈哈）。这样如何？在“快速组绕过器”节点中禁用 SageAttention。
Hello, what a shame it doesn't work... the missing nodes won't install. I've tried many times, but no, I think comfyui won't install nodes that aren't official for security reasons.
That's a common occurrence. By the way, which node is it?
@kenpechi I'm going to take a screenshot.
similar issues here, if any fixes come up then please tell,
@kenpechi I can only speak for myself, but all the nodes in safetensors_models, GGUF_Models, Other_models, Sage_attention, Patch_torch_settings, 1st&2nd_Section, 2nd_Video, 3rd_Section, 3rd_Video, 3-6th_Section, 3-6th_Video, and the large majority of the nodes in input are missing.
could you put a list of all the extensions and custom nodes used into the model gallery, or maybe just a screenshot of all the nodes if thats too much?
@Copper_MausThe CR Seed node can be different nodes.For example, you can use rgthree's "Seed(rgthree)" or easy-use's "EasySeed". But first of all...
@Copper_Maus First of all, have you tried using some of the workflows with comfyUI? Have you installed ComfyUI Manager? If you can't install GGUF Loader, does that mean you're not ready to use ComfyUI in the first place?
If that's the case, first look online for basic information about ComfyUI and how to install ComfyUI Manager. There are many kind people who explain it in an easy-to-understand way.
However, Sage Attention is a little difficult to use, so you should use it only after you've become somewhat familiar with it. Even if you delete this node, it will still be generated; it will just take a little time.
@kenpechi Hello, I changed the seed, but another problem is PathchSageAttentionKJ.
There is no module named ‘sageattention’. I am trying to disable it, but I cannot see it on your feed
@lighthorsexajz830 Is that v3.0? In v3.2 you can disable the sage attention node.
@kenpechi How do I deactivate it?
@lighthorsexajz830 From the fast bypass node, set "1.sage attention" to "no", or bypass (turn pink) directly from the node menu.
@kenpechi I see that. I have a menu with six options. Tell me which one to disable. I'll post a screenshot on this thread.
OK, I found it, I disabled it. I ran a test, let's see if it works.
@kenpechi
Im not completely all new to all this, ive done comfyui before. Im running the newer AMD variant of the official Comfyui AMD ROCM release. I have had a 9060xt(with enough vram and ram, thats not the issue) and got a bit exited to not be running wan2.2 on my cpu lol. I have manager installed btw.
I honestly dont see the GGUF as a horrible issue, more that its failing at something as simple as pulling missing nodes, ive been trying to run custom code in the terminal to pull larger node packs to see if that filles the problems, I did get rgthree to appear as missing tho, which is even weirder (it was the only one that loaded and downloaded, edit: kjnodes too)
edit: a bit later, I have gotten it to recognize all the nodes, via some black magic I dont understand lol. but comfyui has decided that it cant connect to the node registry for some nodes lol. Ill come back when I fix more
@Copper_Maus Thank you. By the way, commenting here would be a nuisance to the other person, so please post in a new comment section.
Actually, I remember having a lot of trouble getting ComfyUI to work using an AMD GPU. It involved Z-LUDA and all that... Thanks to that, I was able to get used to the command prompt quite a bit. :D
It's been about a year since then, and I don't know the current situation in detail, but the AI world is largely powered by NVIDIA GPUs (especially CUDA). In this case, Sage Attention is a good example, and only NVIDIA GPUs can benefit from it.
I've heard that AMD is currently quite good for gaming, but in the world of AI, it may be a tough road ahead.
I wish you the best of luck. Good luck!
@kenpechi this is the last thing ill say in this comment section, check this out
https://blog.comfy.org/p/official-amd-rocm-support-arrives
semi recent development that made things infinitely easier
@Copper_Maus I see. However, I'm using a CUDA environment, so I don't think you can do it the same way as me, so keep that in mind.
@kenpechi Hello, so I'm back for the rest. It does work, but the problem is that it takes a very long time, so I stopped the process after 40 minutes, whereas usually all my animations are done in less than 8 minutes. So my question is, do you have a workflow FP8 to go faster or another solution? Thank you very much.
@lighthorsexajz830 I don't know about that.
For example, in my PC environment, the video I posted here takes about 10 to 15 minutes to generate a 14-second video at 720*1072 resolution using all six sections.
To speed up the process, I only used sage attention and patch torch settings in the workflow.
In other words, there are various reasons why it takes time. I've heard of ways to make it even faster, but I don't know the details.
All I know is this workflow.
@kenpechi I will make an attempt by deactivating the next 4 prompts to keep 2 out of 6
Got more troubleshooting news, Its kinda scuffed, It says that some of the nodes in your workflow arent compatible with ROCM, and I cant find any alt nodes, thanks for the help tho.
the non compatible nodes are:
ComfyUI-Easy-Use
ComfyUI-VideoHelperSuite
, those two give install errors and the hardware related error I was talking about.
Well, if you look at the link above, Rocm officially supports ComfyUI, so there may be some workaround. VideoHelperSuite is a fairly core part, so this workflow won't work at all. Try a different workflow.
@kenpechi its discouraging, but it looks like a lot of things are being developed for amd support, we will likely have it within the year, always tons to look forward to.
I am having issues with the v3.3 workflow
CMD tells
Output will be ignored
Failed to validate prompt for output 1062:823:
* WanAdvancedI2V 1062:873:
- Required input is missing: positive
- Required input is missing: vae
- Required input is missing: negative
I selected video 1&2 and 6th and activated video switch as recoemmended.
I am sure i added all models ( normal non GGUF )
and the LoRAS have also been selected.
Once i press start
It always errors / surrounds the Power LORA Nodes and marks the left side CLIP connector in red as if such needs to manual connected.
If i wire them to GET CLIP something happens but i think this should not be needed to wire such alltime?
First of all, what does it mean to activate 1.2.6?
The way this section works is that you need to activate the sections you want to generate. If you're generating three times, activate 1-3, and if you're generating five times, activate 1-5. 1.2.6 alone won't work.
If all videos are active, videos will be combined each time a section finishes. If you're only interested in the finished version, activate only the video with the same number as the specified number of generations.
Is the above okay?
Next, after installing my workflow, even if you use the same Text Encoder or VAE as me, I recommend manually re-selecting them in the Other Models column at least once. They may not be recognized properly.
These two points concern me, so please check them.
@kenpechi Thx for your help
I set it to just output section 1+2 and i switched Video 2 on.
I re added all models incl VAE and downloaded even the clip model again and it now always displays same time i click start
--
got prompt
Failed to validate prompt for output 814:843:
* Power Lora Loader (rgthree) 1111:
- Required input is missing: clip
* Power Lora Loader (rgthree) 1098:
- Required input is missing: clip
Output will be ignored
--
It seems to issue with the Power Lora Nodes Clip as i noticed.
Beside this i checked the subgraph of each section and there the samplers are set to not output video. I hope this is correct?
Are these Power Lora Nodes already Clip connected in some background subgraph ?
Then i do wonder about this error.
I do not see them clip connected.
If so i will install a new comfyui .
I am on latest stable version there are no custom nodes missing.
@LVNDSCAPE Are the yellow CLIP wires connected between High and Low at around the 4th node of the power lora node?
It seems I might have accidentally connected a version incorrectly due to a mistake on my part. In that case, please disconnect them or redownload the current version.
Please do not reinstall ComfyUI.
I don't think it's a ComfyUI issue.
@LVNDSCAPE I'm simply copying the workflow I referenced for Video Combine; I don't know if it's necessary to do it this way. However, I've never encountered any problems with the generation process.
@kenpechi Hi thx for the further help :)
The black Power LorA Loader node has been wired between High and Low like you mentioned.
So i removed this wire.
The Clip error still appeared
I then checked the custom nodes updates in Comfyui and
the error
got prompt
Failed to validate prompt for output 814:843:
* Power Lora Loader (rgthree) 1111:
- Required input is missing: clip
* Power Lora Loader (rgthree) 1098:
- Required input is missing: clip
Output will be ignored
is caused by a not up to date version of your custom nodes > It needs v1.4.0
and also RGthree / Power Lora has to use latest version.
After i installed these the generation started without this clip error but often fails after video Sampling of "Section 2"
error
RuntimeError: CUDA error: invalid argument
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.
---
What i see in the top progress bar then is the error appears while this vram cleaning node starts in section 2.
This is identical with Torch compile and Sage attention being off or on
Video Output Subgraph :
Your 2nd video subgraph nodes in v3.3 are not correct set if i am correct.
It is visibly bypassed alltime when i acesss the subgraph settings although the switch to use Section 1+2 and Video2 output are active set.
And MP4 Output is deactivated here too in the combine so it will not output anything if you just use a 2 section generation.
This is not in the other selectors.
It anyways fails with that cuda error if i activate these nodes.
If it generates Secton1+Section 2 the section 2 video is always black in the final MP4
Only section 1 video is visible while you then see 2 more seconds of just black content.
PC RAM is not flushed it stays at 99 % after generation
VRAM is flushed
I am using a 5090 32 GB local windows 11 64 Bit
64 GB RAM.
I also tried your v2 Workflow and it has identical CUDA errors but here the
Video 2 output Subgraph is activated and not bypassed visible.
I also lowered resolutions to 640x480 and all is same behaving in its errors.
@LVNDSCAPE I just activated sections 1 and 2, activated the 2nd video, and bypassed the others during generation, and there were no particular problems.
First, you mentioned that even though you activated the Fast group bypass node, it was actually being bypassed, right? This seems like a complete bug. In other words, it's not working correctly. In my experience, subgraphs are quite prone to errors, and ComfyUI's recognition can become erratic. When this happens, you need to delete the workflow's JSON file and reinstall it.
Therefore, please try updating ComfyUI itself using the ComfyUI Manager, and then download the workflow again.
If that still doesn't work...
When I had Gemini read the error you provided, it seems to be a rather troublesome error. Unfortunately, it seems my workflow isn't compatible with yours, so please try a different workflow.
You mentioned you're using a 5090; if it works for me with a 5070Ti, there's no reason why it shouldn't work for your PC.
However, as you'll see when you use it, ComfyUI is a complex system where numerous programs and versions are intricately intertwined. I don't think it's the same as mine, so something might be different, but I'm sorry, I don't know anything more than that.
@LVNDSCAPE Oh, so you're getting errors even with V2.0? Then it's not a subgraph error. If updating comfyUI itself and all custom nodes again in ComfyUI Manager doesn't work, then you should just give up. My workflow is unique in its layout, but the content isn't particularly unusual. You can generate the same video with other workflows as well.
It is working hooray :)
I installed a entire fresh / clean ComfyUI Easy install version
https://github.com/Tavris1/ComfyUI-Easy-Install
Afterwards installed Sage and Flash Attention with the included oneclick installers.
Then dragged your 3.3 workflow in and installed the missing custom nodes in the manager.
After another restart everything looked good and it now works like a charm without black screens or other confusing errors.
It also worked with the normal ComfyUI Portable Version
https://github.com/Comfy-Org/ComfyUI?tab=readme-ov-file#installing
but the Easy Installer version includes the ComfyUI Manager like auto installed and Sage Attention and Flash Attention and more great tools which are usually a pain to be installed.
The workflow output is also very fast.
I tried the non Triton patch and Triton and suprising
the Triton patched is 20 seconds faster than the native vram process ;)
Regarding the setup process:
To just install a fresh clean ComfyUI for your workflow is the way if one has issues in a installed Comfy Version :)
Thx so much @kenpechi for creating this workflows and your amazing videos <3
@LVNDSCAPE That's great!
I didn't know there were such convenient tools available now. My ComfyUI has probably become a mess after so many updates, so maybe I should try a clean install again. Thank you for the helpful information.
Please enjoy creating videos!
This workflow looks amazing , i'm trying desperately to get it to work.. i'm hoping you can help ,me
i think i have everything in need but i'm getting a n error when i run it ?
1 ERROR
RuntimeError: Trying to create tensor with negative dimension -123: [1, 16, -123, 134, 90]
any help would be very much appreciated and thanks again for taking the time to create this, im hoping i can get it to work!
I'm an AI novice, as you might think, but I asked Gemini, and it seems the video resolution doesn't match what the AI is looking for. It seems safer to use a resolution that's a multiple of 16, or at least a multiple of 8.
By the way, what resolution did you use to generate it?
I used 480*720 for testing and 720*1072 for the final version. Because the original image is 2:3 aspect ratio.
@kenpechi i havent changes the resolution Kenpechi, im using it straight out of the zip file.. so the default is 720*1072
im missing some loras but i dont think that would matter, i wish i could just download everything needed from one place, lifes just not that simple :)
thanks for replying too!
I just cant figure out whats going wrong... do you know if there are any simpler workflows that make these animations... ?
@Wezoouk There are countless workflows available, so for now, please use the workflow supported by CivitAI.
@Wezoouk Also, please choose something simple. Even popular workflows can be quite complex. I recommend starting with an easy-to-understand workflow that's just a slight modification of an official workflow.
Hi @Wezoouk @kenpechi I was getting the same symptom after first running this incredible workflow. Some quick debugging showed that a lot of inputs were somehow missing to subgraphs in the model sections. Maybe that's related to ComfyUI back/front-end version differences?
Regardless, as a brute-force workaround, I unpacked the "0.Safetensors_Models" and "Other_Models" subgraphs. Then, I just selected appropriate entries for each resulting node and the "negative dimensions" error was eliminated.
It made me wonder how models were specified as inputs to those subgraphs in the first place! But again, maybe ComfyUI version differences are at play here. Hope this helps.
=====================
"0.Safetensors_Models" section
Load Diffusion Model nodes:
- Wan2.2_I2V_high_noise_14B_fp8_scaled.safetensors
- Wan2.2_I2V_low_noise_14B_fp8_scaled.safetensors
=====================
=====================
"Other_Models" section
SVI LORA nodes:
- SVI_v2_PRO_Wan2.2-I2V-A14B_HIGH_lora_rank_128_fp16.safetensors
- SVI_v2_PRO_Wan2.2-I2V-A14B_LOW_lora_rank_128_fp16.safetensors
CLIP:
- umt5_xxl_fp8_e4m3fn_scaled.safetensors
CLIP Vision:
- clip_vision_h.safetensors
CLIP Vision Encode:
- crop: none
VAE:
- wan_2.1.safetensors
=====================
Sorry to bother you, I encountered this error: "RuntimeError: Given groups=1, weight of size [5120, 36, 1, 2, 2], expected input[1, 64, 11, 68, 46] to have 36 channels, but got 64 channels instead." I have checked my model and the VAE seems fine, and I hope to get some help.
Unfortunately, I don't have the expertise to give a definitive answer.
The simplest suggestion is to update ComfyUI and your custom node to the latest versions. A significant number of issues have been reported with older versions.
If that still doesn't work, I don't know the solution. Some people who messaged me with a different error case solved it by performing a clean install of ComfyUI. However, I'm not sure if that's necessary for my workflow.
@kenpechi Okay, thank you
WanAdvancedI2V
zeros: Dimension size must be non-negative.
Are you talking about the error log?
If so, could it be that some value has changed unexpectedly?
@kenpechi
RuntimeError 814:850 WanAdvancedI2V
↹ Resize Feed
Feed Size...
Column count...
Clear❌
It is indeed an error log. The problem was pointed out in this section, and I did not find any negative values
@2514054148374 Have you updated ComfyUI to the latest version?
I don't know of any other solutions. Please try a different workflow.
@kenpechi CLIP Text Encode (Prompt)_Negative
RuntimeError: mat1 and mat2 shapes cannot be multiplied (385x768 and 3072x768)
After updating to the latest version, I encountered new issues
@2514054148374 The following are the results of my checks with Gemini, and I cannot guarantee their accuracy:
1. The embedding used for the negative prompt is not WAN compatible.
2. The LORA you are using is not WAN compatible.
3. The negative prompt is too long.
If none of these apply, I cannot help you further, so please give up and use a different workflow.
@2514054148374 CLIP
@kenpechi Thank you for your help. This workflow can now run normally, but I would like to ask if there are any parameters that can be adjusted to reduce the usage of running memory
@2514054148374 The two biggest factors are video size and the number of generated frames (video duration).
On my PC, the maximum size is 720*1072, and the maximum duration is 7 seconds.
Naturally, reducing the size will decrease memory usage, but the quality will also decrease. Therefore, I recommend shortening the duration. Since you only need to shorten the video's time for each individual generation, doing this six times will reduce memory usage while keeping the total duration the same. This is a technique unique to SVI.
@kenpechi Although this question is silly, I really don't know how to shorten the time of each generation of videos. Let me ask you, how can I shorten the time of each generation of videos? Or at which node should adjustments be made?Thank you
@2514054148374 The nodes "1st second" through "6th second" determine the length of the video to be generated. Reducing this time is the most effective way to decrease memory usage. For example, if you input 7 seconds into 1st second and 2nd second and generate the video twice, you will get a video of approximately 14 seconds. To create a 14-second video, you could also generate six 2.5-second videos, which would produce a video of roughly the same length. So, which method uses less memory? It's the one that generates six 2.5-second videos. In other words, the important thing is what size video to make and how long it will be in each generation.
Of course, depending on the content of the video, you may want to show a single scene for a longer time, so this method isn't always the best, but it's good to know.
@kenpechi Thank you
RuntimeError: ERROR: VAE is invalid: None If the VAE is from a checkpoint loader node your checkpoint does not contain a valid VAE.
Any known fix?
I've heard of similar errors. Below are some possible solutions, but since everyone's environment is different, they may not necessarily solve your problem.
First, while most people probably use the same VAE model, it might not read it correctly the first time you install it. Even if the VAE has the same name, manually select it again.
Next, update ComfyUI and all custom nodes. Older versions won't work.
Finally, although I don't recommend it, you could try a clean install of ComfyUI again. However, this isn't necessarily the cause of the problem, and you don't need to do this if you're not experiencing issues outside of my workflow. However, I have received reports that this has solved the problem for some.
Best workflow i found so far. Works flawlessly. Thank You
Thank you !
Have a fun !
Flew on the first try. No shamanic dancing or hassle. A real stunner!
That was great!
How do you get the generate clips to stitch together perfectly? None of my results match it.
It's likely that the "continue_frames_count" value in the Wan Advanced I2V node is set to "0". It won't merge unless it's set to "1".
Please check the subgraphs in each section. It's probably set to 0, so please correct it to "1" in all sections.
It seems that this value sometimes becomes "0" when installing the node.
It's already there. Anyway, thank you for sharing the workflow. It's not working for me, it's either the VAE issue or the fact it does not stick properly with WAN.
Hi, do you know why the video output is cropping the image?
I've set the keep_portion section to 'resize' but the image is still being cropped.
My input images are generally 2k and I even tried rescaling the image down to the default width + height and its still cropping.
I checked the output of the start_image in the 1st_section with a preview image node and there was no issues, but the VAE decode shows all the frames being cropped and idk why that is happening.
In this workflow, width+height is used in two ways: in the input image's "resize image v2" node and in the "wan advanced I2V(Ultimate)" node within each section's subgraph. One possibility is that the pipeline between "wan advanced I2V(Ultimate)" and the width+height Get Node is broken, and a unique value is being stored in "wan advanced I2V(Ultimate)".
Could you please check what the value of this "wan advanced I2V(Ultimate)" is in each section's subgraph?
The width + height values are being outputted corrected when I checked with the 'preview as text' node for each section so I think that part is fine(?)
@revolti I'm sorry. I have absolutely no idea what the cause is. I've never heard of anything similar happening before.
its okay, I'm just going to assume ComfyUI_frontend v1.41.2 / v1.41.21 is the main culprit as it broke other workflows :( Thanks for helping tho
HI @kenpechi
I have 2 questions.
1. Is there an upscale_model node for 1st video? I can't seem to find it.
2. I'm using 4x-UltraSharpV2.safetensors currently, what are your thoughts on that quality?
thanks for the workflow again.
Regarding Question 1:
This is a workflow for SVI, so it's not designed for single-run generation; therefore, there is no 1st video. The reason there are five upscale groups is to allow for easy bypassing with the fast bypass node. Therefore, you only need to use one upscale group. For example, if you're generating five videos, you can enable only the 5th video and 5th upscale, and disable the others.
Regarding Question 2:
I don't do upscaling very often, so I'm not an expert, but I've only ever used "2x ultra sharpen v2." This is simply because it shortens generation time.
However, the current comfyUI offloads data exceeding VRAM to the main RAM, so I don't think it will cause an "OOM" (Out of Memory) error very often; therefore, I don't recommend upscaling too much.
@kenpechi Is it possible to run each video individually with this workflow to preview video by video? Instead of running say 3 videos, I can run 1, 2, and then 3 to make sure I like each part to save time?
@johhnnymann1 In that case, please enable video for all sections. This will generate a video at the end of each section, allowing you to check the results as you go along.
@kenpechi Ok thank you for your help and information! I will give it a try.
This is an amazing workflow and i'm impressed with the results you have been getting! However I cannot seem to replicate them. I spent a few hours searching and downloading all of the exact same models and Lora's that are in the workflow by default (even going so far as to run a fresh install to get Sage Attention up and running) but the result is always awkward movement and pseudo body horror. Is it possible there is something that I am missing from when the workflow is first loaded? I did notice that it seemed to run without issue even though at first I did not have the required Lora's which I thought was odd as I would have expected it to fail and require them. Any assistance would be greatly appreciated!
Since you mentioned there are no errors, I think the workflow itself is fine.
I don't know how much experience you have with SVI generation, but getting good results with SVI might require some tricks.
For example, even with LORA, some LORA models work well with SVI, while others don't. Sometimes it doesn't work depending on the input image.
You've only seen my success stories, but in reality, I've failed many times more times than that.
So, please try various things. However, it's best to use a large image size (720p) and a short generation time (under 5 seconds).
I forgot to mention, LORA is not essential for generation, so you won't get an error if you don't have it.
However, if you don't use LORA properly, it will result in a horror video, so please adjust the LORA Strength accordingly.
@kenpechi Is it normal for the upper body to turn to the back while the lower body remains in front of the camera?
@ethan66666 It's possible. I don't know what action you're prompting, but it might be an action that's difficult to achieve with SVI. In SVI workflows, you should prompt actions that can be complemented by the powerful LORA. For example, an action I often generate, like rolling up a shirt, would be horrifying. I think it would be difficult to generate long videos using SVI, which is not good at such operations, with the current Wan2.2.
I got an error: WanAdvancedI2V.execute() got an unexpected keyword argument 'prev_latent'
I'm sorry, but I've never heard of such an error before, so I don't know how to fix it.
@kenpechi After updating the ComfyUI-WanVideoWrapper plugin, the Wan advanced I2v(ultimate) node no longer has the prev_latent option. I asked AI and they said the new plugin has removed prev_latent and changed it to directly transmit the image to the front end, so I don't know how to connect it.
So, you're saying I'm installing something that isn't in my environment? I don't know what that plugin is, but I can update all my custom nodes without any problems in my environment.
This workflow is based on prev_latent, so I recommend you use a different workflow. You can create similar videos with other workflows. It's just a matter of preference.
Hi.
I used your workflow and I have a trouble. I think I set all the models are same default, but I got rainbow noised video. It's full of noise, but I can see how target is moving.
I'm glad if you can help me out.
The noise is most likely due to one of the models not being positioned correctly. The most common cause is a LOW model being input to a HIGH input, or vice versa. Please check this.
@kenpechi Thank you for the reply! I already checked the models connections are wrong or not 3 times. I even used Claude to compared yours with mine but there's no difference besides LoRA so I executed without LoRA but same result. I have no idea what to do next...
@iezlx As you can see from the comments here, the error messages seem to vary greatly depending on the environment.
All I can say is, please try a different workflow. You can create similar videos with other workflows. Mine simply has a unique layout.
@kenpechi Yeah, I’m gonna try another one. Also, I might re-build another comfyui setup later, so can I have your Comfyui edition(portable, desktop, cli) and version ?
@iezlx portable, ver 0.18.0
@kenpechi Thank you!
Hello, great workflow! I am trying to use it but every video I output is named "temp_save.mkv" and i cannot open it or play it. what to do?
I'm not sure why this happened, but I think it's because the video format in the "video combine" node within the "video" subgraph of each section is set to MKV. Please change them all to "H264/MP4".
Alternatively, the save function in the "video combine" node within each section might be enabled. Please disable it.
@kenpechi Thank you for the response—where is the video combine node?
@seansiena616809 Click the square in the upper right corner of nodes such as "2nd section" or "2nd video" to see the contents of the subgraph. It's there.
I faced the same issue myself.
When upscaling is enabled and no upscaling model is found, the generated video isn't merged and an .mkv file is saved in the temp folder.
You should set upscale model, or turn off upscale.
I tried the model, I really would love if I could figure out how to get it to work! I had to swap a couple FP8s for FP16s, and used GGUF. After waiting about an hour for it to render, it was straight up body horror. I really like your generations so would love to know what I'm doing wrong.
SVI is fundamentally a horror game. To make it work correctly, you need to run the appropriate LORA with the appropriate prompts.
Also, if it's taking an hour, are you generating 30-second videos? I recommend starting with shorter videos and continuing to generate them.
@kenpechi Hi! Based on your advice, I wanted to try running one of your workflows directly, so I downloaded all the same loras and used one of the workflows embedded in one of your videos, with an image from your page as well in the same position. This time, I was actually able to get the video to generate with good details and as expected, but only certain sections worked - half of the sections were just a pink tiled screen. If I figure it out, I'll let you know. But if you have any advice, I'd be grateful!
@vocironic I've received reports of the same problem from other people, but I don't know the cause. However, their entire video was full of noise, while yours, where only a part of the video is affected, is even stranger.
Normally, this would be caused by differences in the High and Low settings of each model, but your settings are correct, right?
I'm sorry, but I can't help you at the moment.
@kenpechi I just wanted to come back and say that I've actually resolved this issue after some trial and error. I changed a couple things, but nothing actually worked until I took a wild guess that the sections set to 3 seconds were causing some sort of calculation error, whereas the working sections were set to even numbers like 2 and 4 seconds. I set the glitched sections to 2 seconds, and it all worked flawlessly. Cheers!
@vocironic That's good to hear. However, I don't have any problems even when I set all six to 3 seconds, so there must be some difference after all. Thank you for letting me know.
i press start and nothing happens :( i found where to upload image i think? the workflow is so massive with no picture tutorial on how it works or manual i have no idea what to do
Are you a beginner? This is one of the easier SVI workflows.
Please try generating more simple workflows repeatedly, experiment, and then come back again.
@kenpechi ive made my own workflows ive been doing this for a long time but i havent bothered with wireless nodes or doing layer, im literally scrolling around trying to figure out whats connected to what lol
Spectacular workflow, the default setting from the workflow is already good enough to generate great content. Though I spent some hours to play with the missing models and the settings.
I have a few questions for this workflow btw:
1. My upload image is at 480P like, but the videos you generated were high resolution. The Upscale seems not recommended from the notes. So should I use upscale for every video section.
2. I hope that you can add audio in the workflow as an add-on, or do you recommend any?
3. Could you please share complete the model and Input value with your generated video daily/regularly?
4. Do I need to enable Low output? As most of your samples mentioned High model only.
Thank you for the hard work and keep going!
Regarding Question 1:
Upscaling is possible and included, but I rarely use it. I don't think you need to use upscaling unless you're using a GPU with low VRAM.
Regarding Question 2:
I don't plan to add audio in Wan2.2. However, many people are using Kijai's "MMAudio," so you might want to refer to that. I think you can add audio later.
Regarding Question 3:
I've always embedded metadata in almost all of my videos. You can see all the data by downloading my videos and putting them into ComfyUI.
The reason I posted the workflow is because there were reports of people not being able to see the metadata due to different versions of ComfyUI, and I got tired of the back-and-forth of "I want to see the workflow" and "I've already embedded it in my videos."
Therefore, I've already shared everything about video generation with everyone.
@kenpechi Thank you for the hard work again. No many people like you do it for free or just take it for granted. Appreciate your time and the effort.
Got 1 update for question 1.
Does upscaling my image helps the entire video quality? And how to extend the video length?
I am running on a 3080Ti with 64GB RAM, normally take 20-30 mins to generate 1 video. Planning to upgrade in the future. Sorry for so many questions.
@MyLittleSecrect As you may know, the creators of SVI originally trained in 480p, so they announced that they would like you to generate the video in 480p. However, since it's generally necessary to raise the resolution to 720p or 1080p to make it look better, they included an upscaling function from the start. However, after trying various things, I don't see any particular problems with 720p.
Personally, I feel that videos generated in 720p from the beginning look better than videos upscaled from 480p to 720p. This is a matter of preference. Certainly, upscaling from 480p will shorten the generation time.
Incidentally, I generate videos while I'm sleeping or out, so I don't really worry about the generation time.
Please help. The generation works fine, but the video speed keeps increasing with each section. (not generation speed,I say about playback speed.)I want generate at the same pace as the 1st section.
The SVI only references the first image; the video generated for each section is performed independently. Therefore, achieving perfect identicality would require multiple generation attempts. However, if you match the duration of each section, use the same prompts, and the same LORA and strength, the results are likely to be identical.
I intentionally shorten the length of each section to increase the playback speed, but I suspect that if the lengths were the same, the speed would also be the same.
Thank you for your reply.
I tried setting the duration to 5 seconds for all five sections with the same prompt, but the playback speed still keeps getting faster and faster.
The same thing happens even when I set it to 3 seconds.
This is quite a challenge...
@kaiboutanpaku496 That's a problem I hadn't been aware of until now.
I wonder if this tendency is specific to SVI, or to WAN-advanced I2V nodes. I'll keep an eye on it going forward.
@kenpechi I have the same issue with speed jumping up to 2x. First and last video is normal speed. Same prompt, length or even loras on first 5. No effect and it keeps speeding up. Using 5950X with 128GB of RAM and Nvidia 3090
@Menninkainen Thank you for the information. Currently, we don't know how to address this issue.
請問要從哪裡調整每段的影片秒數，現在13秒太快了，想要慢一點
请输入第 1 秒到第 6 秒节点的秒数。
Thanks you buddy, first time i achieve a long video without quality loss or cut,
and i can use my beloved Remix_Nsfw checkpoint
Everything is setup to be easily used, geat work! ❤️
That's great!
Have fun!
Hey, i have been getting this error message, i made my width 1280 and height 720 for 16:9 frame but i changed it back and i dont think thats the issue.
error message: mat1 and mat2 shapes cannot be multiplied (462x768 and 4096x5120)
Well, I've encountered this error before in a different workflow. Perhaps the text encoder type doesn't match? In other words, there's a possibility that some model is incorrect.
@kenpechi could this be the issue?
this is an image of the node
https://drive.google.com/file/d/1DUASGUelL4_P_d-lru5-ZZ5XWd0XOSJw/view?usp=sharing
@marcelinorafik01419 That's incorrect. The top CLIP should contain "umt5_xxl_fp8_e4m3fn_scaled.safetensors" which should be placed in the "ComfyUI/models/text_encoders" directory. I'll provide a link for your convenience.
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/text_encoders
@kenpechi yeah i have them but the reason i thought these were correct is because it was already placed and i wasnt told i should change it. I'm a beginner so i dont know what should be changed and what should be kept.
But anyhow this should fix it?
@marcelinorafik01419At least that problem will be fixed.
@kenpechi okay thanks a lot and i really really reallyyy appreciate how quick you respond, you have no idea how supportive and unique that is, thank you.
Ill try it out and keep you updated.
Thank you for sharing ths amazing workflow, and for always offering metadata with your artwork. You have been very generous to the AI art community and I appreciate you.
For v3.4, I found that inputs to the model section subgraphs were not present on my system, so I unpacked the following subgraphs and manually updated the selections for the resulting nodes:
"0.Safetensors_Models"
"Other_Models"
Everything worked great from there!
I never tried SVI workflows before and the results from your workflow have been great.
Thank you for the information.
Enjoy!