SVI Extend
https://github.com/vita-epfl/Stable-Video-Infinity/tree/svi_wan22
Create videos and extend them seemlessly using SVI.
Following SVI LoRAs are mandatory:
switch between default behaviour, anchor_samples and end_frames within the same subgraphs
connect an image to a part and enable the respective toggles to use end_frames or anchor_samples
NEW! v3
Extend existing videos using https://github.com/wallen0322/ComfyUI-Wan22FMLF
enable "video extension" toggle inside the settings
uses source video resolution by default
rescale video using the megapixel slider by enabling "video rescale" toggle
use included version of the nodes from inside .zip or download the latest version straight from the git if issues arise
More info inside the workflow.
AIO i2v+t2v
All in One workflow for for basic WAN 2.2 video generation.
Following features included:
Switch seemlessly between 2 and 3 sampler solutions
Toggle between i2v or t2v
Postprod
Facedetailer
uses t2v Model + LoRA for inpainting - resources needed included in workflow
Toggle between GIMM VFI and RIFE VFI Interpolation
Upscale
Tensorrt Upscale with Model
Basic Video Upscale with Model
RTX Video Super Resolution Upscale (insanely fast for decent quality)
Frame Clipper
Seamless Loops using custom RIFE nodes https://github.com/Artificial-Sweetener/comfyui-WhiteRabbit
Upscale + Interpolate
I recommend using this workflow instead of upscaling with the generating workflows, since you never really know what kind of results you get, ending up upscaling a bad video and wasting time. I included toggles so you can't use multiple interpolation or upscale nodes at once by mistake.
This includes:
WAN Facedetailer
use any WAN 2.2 T2V low model + the following T2V LoRA:
lower resolution from 768 to 512 if you have VRAM issues
Put the following file into "ComfyUI\models\ultralytics\bbox":
WAN Refiner (massive VRAM cost)
increase denoise if you want more inpainting
Sharpen, Gamma, Brightness and Contrast control
Frame clipper (remove unwanted frames at the start and/or end)
GIMM VFI + RIFE VFI interpolation (I recommend GIMM VFI, much higher quality but also much slower)
Tensorrt Upscale + Basic Video Upscale
both use basic image upscaling models
Tensorrt (faster than Basic Video Upscale) with AnimeSharp4x is recommended for anime
RTX Video Super Resolution Upscale
insanely fast
decent quality
FlashSVR + SeedVR2
experimental
Video upscale models that are more intricate than basic image upscaling models
haven't had great results for anime yet
takes a LOT longer
Saving last frame for manual extensions
mmaudio
added Audio combine node
combine audio from an existing video with the generated audio on top
generate nsfw audio with the nsfw model and then combine that video with another generated audio track from the base model for background noises
removed interpolation for easier and faster audio generation - you have the following options:
upload raw unupscaled video to MMAudio Video node and upscaled video to Combine video node
upload upscaled video to both nodes but lower custom_width and custom_height of the MMAudio video node to about half for faster generation and to prevent VRAM issues
upload raw video to both nodes and upscale afterwards
Inspired by https://civarchive.com/models/2137833
Following resources necessary (ComfyUI\models\mmaudio):
https://huggingface.co/Kijai/MMAudio_safetensors/resolve/main/mmaudio_synchformer_fp16.safetensors
Description
latest ComfyUI versions seem to break Power LoRA Loader used as widgets for subgraphs! I recommend ComfyUI versions before 0.15 - I personally use a portable 0.12.3
Fixed resizing issues on some end_frame edge cases resulting in stretching
Fixed a few switching issues
set force_rate to 16 on the video upload node to match output framerate on interpolated videos
exposed overlap frames on each part
increase frame overlap from 5 to 9 when using motion_latent_count 2 for smoother transitions
Added more postproduction options, following are now available:
Facedetailer
GIMM VFI + RIFE VFI Interpolation
Tensorrt Upscale + RTX Video Super Resolution Upscale
Sharpen
Seamless Loop
FAQ
Comments (27)
The latest SVI v3.1 should fix all the switching issues I mistakenly introduced when rushing v3. Also added some long overdue postproduction nodes like GIMM VFI and more.
The exposed motion_latent_count and overlap frames will give more control over individual part transitions.
I generally recommend motion_latent_count at 1 and overlap frames at 5. If you want to reinforce the motion from the previous part then you can increase motion_latent_count to 2 and overlap frames to 9.
Thank you so much! I did run into an issue though — the preview doesn’t work. Is that just on my end, or is it supposed to be like that?
I just redownloaded v3.1 to see if I missed it but it's working on my end - can you try right clicking the video combine (full merge) nodes and press on show preview?
How do your mmaudio prompts look like? Been playing around with it and cant really get a good result
I try to keep it relatively simple, for example: "seductive female moaning, skin-on-skin impact sounds," or "gentle female moaning, skin-on-skin impact sounds," or "passionate, powerful female moaning, skin-on-skin impact sounds," you can also add "heavy breathing". Depending on the scene it may take up to 50 tries, the longer the video the less consistent the results are. You can get good results up to 15s and still usable up to 20s but after that it tends to fall apart. It generally also works much better on more realistic styles. Results vary massively between videos too so you may not always get good results.
On shorter vids I get good results within the first few tries but it's less consistent on longer vids.
Can you explain what is the "Frame Clipper" feature doing in the latest AIO workflow? From what I found it's something that helps the FLF2V so that you always get the last frame you wished for.
The Frame Clipper exists to cut unwanted frames to get a better transition. Sometimes your initial frames take a while to get moving so it may be better to cut the first 4-8 frames to get a better transition when doing loops, same for the end. This is especially useful when used together with the "Seamless Loop" node since those take the first and last frames and try to interpolate the best transition possible between them.
You basically need to play around with the clipped amount depending on the generated scene but I definitely recommend using both the Frame Clipper and Seamless Loop together when doing loops.
Does it work good with the padl00p lora?
@TekeshiX It's actually not needed anymore if you use the nodes mentioned above, probably even better without the padl00p but may need more tinkering. I generally had great results when looping with just clipping and the seamless loop node.
https://civitai.com/images/125316154 uses both for example (same goes for none extended videos), the loop wasn't completely perfect because of the disappearing arm on loop but that's an edge case where I didn't do the end_frame at the last transition properly.
@LastAssignment Okay, I tried with frame clipper and seamless loop both on, but at the end I get a "weird" hand movement (like disappearing and appearing immediately after), but it's still noticeable and not really game ready. I used the default settings (4s clip, frame clipper - start at frame 6; clip end frames 0 and seamless loop - seam-window-frames 6).
Although, when I used the seamless loop feature alone (without the frame clipper), that weird disappearance doesn't occur anymore, so it's game ready.
@TekeshiX Yeah, it all depends on how many frames the clip needs to transition to the last frame but it's generally harder to get good loops with normal non extended videos since you can't prompt for the loop without fucking over motion.
@LastAssignment Do you have any advices on what you mean by "tinkering" and "that's an edge case where I didn't do the end_frame at the last transition properly"? What settings should I pay attention to especially to not get the same example as that arm disappearing? Do I have to change the values from Frame Clipper or from the Seamless Loop?
@TekeshiX That arm disappearing thing was an error on my part for not prompting for the hand to go back since it was 15s extended video. I just couldn't be bothered to generate that last part again. Can't really prompt for that in a single clip or you may get reduced motion if you prompt for stuff like "her arm keeps it's position" or whatever may help in that scenario.
I mostly generate raw clips first and then look at how the motion starts and ends and I change the settings depending on how slow the start is or how long the last frames linger. Really no way around it other than adjusting it on the fly depending on the clip since it will change for every scene. Sometimes you may need to generate another clip because the motion didn't transition well enough to the last frame. You basically want to reduce the amount of still frames as much as possible.
@LastAssignment Appreciate, man! So there is no established "preset" that'll work in all the cases, you say. And I guess that's true because now I got a perfect looping video with all of these activated at once (8s video):
- Frame Clipper
- Seamless Loop
- Padl00p LoRA (1.0 strength)
Btw, found this workflow a while back, maybe you find something useful or some new stuff integrated into it (the example looping videos look good enough):
https://civitai.com/models/2465351/wan-22-reasoning-loops-or-live-wallpaper-or-nsfw-or-svi-or-vbvr?modelVersionId=2771927
Keep up the good work, man!
@TekeshiX If you want proper looping of multiple clips then I recomend trying https://civitai.com/models/2024299/wan-vace-clip-joiner-smooth-ai-video-transitions-for-wan-ltx-2-hunyuan-and-any-other-video-source?modelVersionId=2811438 but I haven't tried it yet for nsfw clips so your mileage may vary.
Hey! I’ve been using AIO v2.1 workflow and I’m really impressed with the generation quality so far. I did notice something I’m a bit confused about, though.
The post-production output seems significantly slower compared to the raw and facedetailer outputs (It's almost like in slow motion).
Also, would it be possible to get the raw and facedetailer outputs (they currently generate at 5 seconds, even though I've set it at 8) closer to ~8 seconds instead of the POSTPROD, or at the very least, for the POSTPROD to have the same speed as the other 2?
Just trying to understand if I’m missing a setting or if this is expected behavior. Thanks!
Hey, facedetailer doesn't actually affect the duration, you can theoretically input >1 Minute videos if you have enough VRAM.
The slow motion issue sounds like an error with the automatic framerate calculation.
Which ComfyUI version are you running and can you upload an example video with the workflow embedded so I can see if there's anything broke?
@LastAssignment Thanks for the quick reply! I'm running ComfyUI 0.19.0. I'm relatively new to ComfyUI and animations so it's very possible I'm missing something or misconfigured a setting somewhere. Either way, I just uploaded an example video with the workflow embedded, hopefully that helps narrow it down. (Currently being reviewed) Thanks again for the support!
@Flawsh You can circumvent the slow motion issue for now by removing the connection to the framerate of the video combine node (Postprod Output) and adjusting it manually (16 * interpolation factor). I'll take a look at the workflow in your video once it's released.
Reduce your framerate to 16. WAN 2.2 generates videos at 16 FPS, anything above that will result in skewed results and motion will look weird, you're basically speeding up the video by 50% which also results in the video being shorter.
@LastAssignment I see, thank you for the insight! I was initially creating the animations at 16 FPS but the results weren't great, I'll test further once I get more time and try to apply your advice to see if it improves things. Awesome work with the workflow though, really appreciated!
What version of ComfyUI is required for this (WAN 2.2 SVI v3.1)? I cannot add any LoRAs to the video subgraphs.
In fact, even in the video example one i can not modify the LORAs that are already loaded. I can't even see them in the graph view, I have to check the properties.
Very strange.
ComfyUI's frontend fucked subgraph widgets with version 0.15.0 and above. You can still change the LoRAs by going into the subgraphs but it's definitely suboptimal. I personally used the 0.12.3 version for this workflow but any below 0.15.0 should work for the subgraph widgets.
@LastAssignment Thanks a lot for the response! I'll give it a try!
im using the AIO v2.1 video example workflow (default) . and my device is RTX 5090 + 64gb RAM .
and still getting error "OutOfMemoryError"
what im doing wrong here ? thankyou!
after disabled few loras its running but stuck at base 58% sampler, and CMD line :
[INFO] Patching torch settings: torch.backends.cuda.matmul.allow_fp16_accumulation = True
0%| | 0/4 [00:00<?, ?it/s]
@xuro Try disabling SageAttention.
What model, resolution etc. are you running? ComfyUI may not always offload some models properly, leading to running slightly over max and you end up with hour long generations because it constantly swaps between RAM/VRAM. Try to increase or decrease the resolution so it can actually offload properly.
