Okay, This straight up is the coolest thing I've seen in a while.
Thansk to What Dreams Cost for this insane set of nodes
With this node you can create T2V, I2V, add sound, combine, set keyframes, and basically driect your own video, all within one workflow
requirements:
If you can run LTX2.3, you should be able to run this
At least 12gb VRAM
Nodes and models are listed in the workflow
Watch the video here:
Instagram: https://www.instagram.com/synth.studio.models/
Buy me a☕ https://ko-fi.com/lonecatone
This represents many hours of work. If you enjoy it, please 👍like, 💬 comment , and feel free to ⚡tip 😉
Description
FAQ
Comments (21)
Dude I love your workflows, always clean and expansive, they are usually always great... BUT I tend to always get a similar error on first run:
RuntimeError: mat1 and mat2 shapes cannot be multiplied (1024x3840 and 1920x4096)
I usually can get GPT to point me to the problem but this time I cannot figure out the issue here? It's literally bone stock, using the GUFF (ltx-2.3-22b-distilled-Q4_1) mode instead of the 48gb checkpoint lol Should I be using the taeltx2 instead of 2.3? any other suggestions? I am using this on windows 11.
That's literally always a cross between clips and the model. almost without exception.
If you load a Q4_1 GGUF model, you must have a matching GGUF Clip.
Those are the most annoying and common errors
Cant wait to try your take on this one! The director is amazing! I just need to get better at prompting it lol.
ERROR: VAE is invalid: None If the VAE is from a checkpoint loader node your checkpoint does not contain a valid VAE.
Doesnt matter what model i use. Diffuser or checkpoint.
Use the standalone VAE. More then likely you are using a diffuser model in a checkpoint without a built in VAE. That's fairly common.
If the checkpoint is on and its VAE is connected, disconnect it of it continues to give you this error
OR.. you have the video VAE in the audio VAE location, or visa versa
@lonecatone23 Thanks, i just bypassed the Vae audio and it went through but now im getting 'NoneType' object has no attribute 'tokenizer' on LTX Director node? Any ideas?
@djjayem940696 tokenizer is the clip. always. Either missing, the wrong one, or a gguf, fp8, fp16 vs. model issue.
This worked Just fine for me. If the models I had listed do not work, please lt me know. It's a lot of work putting these together, so I may have mislabeled one
@lonecatone23 I just changed the LTX Audio VAE with the standard one like you said and all runs great now. Thanks.
@djjayem940696 awesome!! Glad it works for you 👍
Why aren't there conversions between latent SDXL images and whatever some other checkpoint uses? I like using one checkpoint to start generating a sketch of my image, then hand it off to a more powerful (but with less vocabulary) renderer. Sometimes I get spectacular results but I never know what will work with what.
why would you just not use a picture? This is a video workflow.
Are you sure you are commenting in the right place?
This looks really awesome, but when I try to run this I am getting:
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I have double, and triple checked everything and still no luck. Any advice would be greatly appreciated!
did you change a setting?
That's basically a fancy way to say something is empty. Either you do not have a prompt/ image, or you have something set to zero.
Try this. Load a simple image and g-=create a prompt for like a 4 second video and run it.
@lonecatone23 Thanks for responding! I didn't realize that an image needed to be added. I haven't changed any of the workflow settings. I added a pic to the default prompt using z image turbo, but got the same error. Looks like it might have something to do with the rgthree loader. I tried to paste the complete error dump here but the site won't let me.
EDIT: Power Lora Loader (rgthree) contains the error.
EDIT-EDIT: Found out that one of the loras was a botched download of only 1kb. Pretty sure that's the issue now. Redownloading Lora to check for fix. Sorry to bug you lol!
@nunyabizness1 ah.... Was it a bad link om my part, or just a corrupted download?
@lonecatone23 Really sorry for super slow reply, but I had to do some stuff and forgot about checking messages here for a bit. I got it running, and it works great on shorter videos but I'm getting OOM's with longer (+20 sec) videos. Would normally try to increase tiling to help with VRAM but can't find the setting in your workflow. Also can't use GGUF setting without a weird error with Gemma3:
DualCLIPLoader (GGUF)
ValueError: Unexpected text model architecture type in GGUF file: 'gemma3'
Presently using gemma-3-12b-it-qat-Q4_K_M, but it throws the sane error with any gguf gemma3.
And no, your link was fine, just my trashy internet connection botching a download lol!
@nunyabizness1 I ahev nooo idea why it is faulting out on teh model.
I ll add a place to tile the decoder. It's there., I just have it collapsed
Hi, I think there may be a small wiring issue in LTX 2.3 Director Ver 1.01.
When generating a 5-second video at 24fps, the output was consistently around 6.04 seconds / 145 frames instead of the expected 5 seconds / 120 frames.
After checking the workflow, I found that the issue seems to come from node #14 LTXVCropGuides, located in the lower-left area of the Brains-Stage2 group. Its positive / negative guide inputs were still connected to the earlier guide source instead of the LTXDirectorGuide node in Stage2.
Because of this, the extra Stage2 guide/keyframe latent frames were not cropped correctly before VAE decoding, which caused the final video to become longer.
Fix:
In Brains-Stage2, find node #14 LTXVCropGuides in the lower-left area. Keep its latent input unchanged, but reconnect its positive and negative inputs to the LTXDirectorGuide node in Stage2.
After making this change, the output duration and frame count became correct.
Hmmm. Thanks. I'll look at this.
fixed. Thank you 🙏
HI TY for your work but i was not able to run it because i have RTX 3060TI card.
Can you make a new workflow file to low vram GPU users so they can test it too?
I am using: ltx-2.3-22b-distilled-1.1_transformer_only_mxfp8_block32
Can you tell me which models i can use with this or add a new workflow for low vram users?
Will buy you a coffee as well.
Thanks