Welcome to my 💫🎦 Friendly LTX-2 T2V+I2V+Lipsync
LTX-2.3 better in everything! Coming soon...
✨ Less mess, more magic
UniVibe - Lipsync all-in one version with HQ TTS VibeVoice model is released.
New v1.2 with simplified model loading, with quality and perfomance improvements.
LTX-2 is a new video generation model with 19b parameters under the hood. This is the first DiT-based (Diffusion Transformer) foundation model that generates synchronized audio and video simultaneously in a single pass! It supports native 4K resolution at up to 50 FPS, providing cinematic-grade fidelity suitable for professional VFX and film production and it is capable of generating clips up to 10–20 seconds with consistent style and motion.
💻 System requirements:
Minimum system requirements for 540p i2v and 720p t2v:
RTX 3000-s, 8GB+ VRAM, 45GB+ RAM, 8-core processor, SSD, latest ComfyUI
🚀 Low VRAM optional optimization:
For systems with low VRAM use --reserve-vram ComfyUI parameter in run_nvidia_gpu.bat:
--reserve-vram 4(or other number in GB).
📌 Detailed tips and links to models in the workflow
✨ Workflow features:
Extremely user-friendly interface
Maximum performance and optimization from 8GB of VRAM: GGUF or 8-step distilled model with fp4 or fp8 text encoder + MultiGPU memory optimization
All-in-one: i2v, t2v, and interpolation
Convenient one-click mode switching
Generation time setting in seconds
Lora support (up to 3)
Detailed tips and links to all necessary models
Manual random seed for complete control over generations
🤗🙏🏼 Thanks to Lightricks Team
Original repo — GitHub
Description
· Ultimate LTX-2 version: t2v, i2v, lipsync, speech generation, interpolation
Some recent fixes:
- Fixed a bug and adjusted the logic for setting the generation duration
- Fixed a vae bug that caused artifacts in the final generation
- Added new Kijai nodes for VRAM unloading for extra perfomance
- Added an image strength parameter setting, which controls the accuracy of the original image
· Choose 1 of 3 models in one click: Dev, Distilled, GGUF
· Powered by VibeVoice HQ TTS model for voice generation with up to 2 speakers
· Step-by-Step Generation Control: set audio first then go to video generation
· Modular workflow: generate lipsync with audio sample or lipsink with voice TTS model or just regular LTX-2 generations without audio samples
· Perfomance optimization
· Updated detailed tips in the workflow
FAQ
Comments (15)
Hi, does it work with 24GB VRAM and 32GB RAM? Like total 56GB combined?
@Samsura Hi! I think it should work. Despite the fact that the models are heavy and primarily load into RAM, the latest ComfyUI is able to distribute the load well between the paging file, RAM and VRAM
It is working but number of frames and length is something confusing it has given me length of less then 1 sec. Though i have asked to generate 5 sec video
@salunkedprasad228 Hi! In which mode are you encountering this issue, with lipsync mode with voice generation or others?
@RusselX with the lip sync mode
@salunkedprasad228 Ok. Lipsync mode length depends on a sample 1 crop. In your case crop maybe set for 1 sec. so you get that length. I upload new 1.0.2 version with some bug fixes including generation length bug. So try it please
now it is giving me length as per my requirement, but still video has artifacts, i think user should have freedom to use multiple combination, like gguf and non gguf clip, sorry for the non-technical language. and your workflow has potential to pull great result.
@salunkedprasad228 thanks for your feedback! What kind of artifacts do you have and with which model do they appear?
Regarding freedom of models choise, including gguf, it's difficult to implement in the current workflow realization. But I'll consider the options.
Using the Distilled FP8 version, I am now getting "Latent Switch - No latent inputs connected".
@nickgiannotti811 Did the distilled model work before? I'll be releasing an update soon that recommends combining new fp4 model + distilled lora. It is more stable and balanced
@RusselX It did - just started not working a couple of days ago. :(
@nickgiannotti811 This may be related to the Comfy update. I have Comfy 0.12.3 Stable now and have no this issue. You can try new UniVibe v1.1 with fp4 model and distilled lora
@RusselX Running 0.12.3 stable as well here. UniVibe 1.1 almost works but VibeVoice refuses to install, either via manager or via git clone. I'll keep trying the Distilled to see if I can't figure out what's wrong with it.
@RusselX I found it - it was the ComfyUI-Light-N-Color custom node. Disabling this and it works fine now!
@nickgiannotti811 Glad you found a solution. Have super generations!