This workflow uses the LTX IC-LoRA for LTX-2.3.
ControlNet (Union Control):
Load an image and an audio file (either your own or the original audio from the source video), or alternatively use LTX Audio—the audio is used for lip synchronization. Then load the target video to track and transfer its movements.
Refine and Upscale (Detailer):
You can also refine and upscale an existing video by setting ControlNet to "Off", Image Bypass to "True" and loading the IC-LoRA file for the detailer "ltx-2-19b-ic-lora-detailer.safetensors" instead of the ControlNet model "ltx-2.3-22b-ic-lora-union-control-ref0.5.safetensors".
Info:
The length of the output video is determined by the number of frames in the input video, not by the duration of the audio file.
For upscaling, I use RTX Video Super Resolution.
Tips:
If you experience issues with lip sync, try lowering the IC-LoRA Strength and IC-LoRA Guidance Strength values. A value of around 0.7 is a good starting point. If you notice issues with output quality, try lowering the IC-LoRA Strength as well.
Description
FAQ
Comments (16)
Great Workflow! Love it :D
Thank you for the workflow, I'm having trouble with the RTXVideoSuperResolution Node. How can I install it?
Did you try it with the Manager? But you can google it, too. It's on github. Do you know how to install with git clone?
Git clone https://github.com/Comfy-Org/Nvidia_RTX_Nodes_ComfyUI
@Thalion7 I have the same problem installing the node (import failed, ModuleNotFoundError). From the manager, or manually git cloning, trying fixing the node... nothing works. As I understand, this is another NumPy conflict.
@Aiariar Did you install the requirements for Nvidia_RTX_Nodes_ComfyUI using pip install -r requirements.txt?
Please make a backup of your ComfyUI folder first.
@Thalion7 yep, nothing works... I'm just bypassing the node
I don't understand anything, where do I upload the models?
This workflow has a subgraph (Main Settings). To open it, click on the upper right corner of the Main Settings node. The top will now display "LTX-2.3 Image + Audio + Video CN to Video / Main Settings". To return, click on "LTX-2.3 Image + Audio + Video CN to Video". If you don't see any nodes, click on the "eye" icon next to the percentage indicator (Fit View) in the lower right corner.
Did RTX VSR get fixed? It's been broken for everyone for a while...
In my case it works fine
@Thalion7 got it, will have to try it again, thanks.
thanks for your workflow, I wanted to find somethign to help add a detail lora to my generated LTX video.
But I'm running out of memory.
google AI suggested
1. Reduce Batch Size / Frame Count: If you are generating a video, lower the number of frames. If you are generating images, ensure your batch_size in the Empty Latent Image node is set to 1.
2. Lower Resolution: If you are trying to generate at
or higher, drop it to
or
to see if it clears the sampler stage.
3. Use Tiled VAE: If the error happens at the very end (during decoding), add a VAE Decode (Tiled) node instead of the standard VAE Decode.
Could you please provide tips on which nodes I should be adjusting?
What's your Graphic Card and how much VRAM do yo have?
The workflow is set to Tiled the first time you run it. Mabye you changed it? Yes, try a lower resolution. Maybe your input video is to long. I can run 30 sec. and 720x1080 with my 16 Gbyte VRAM rtx Card. I am sure your batch size is 1. If not reduce it. But that is a ComfyUI setting.
the video im trying is 1280x720 at 20 seconds.
4070 with 8 GB VRAM. I think it's probably too low, right?
I'll try the settings you metioned. thanks
@gackt2 well, yes 8 GByte VRAM is very low.
@Thalion7 sigh.. I envy those that can afford VRAM lol