DONT USE THIS
It was a fun run. I2V proper is released, download that. I'll leave this up only for a weird old initial way of janking a i2v process before it was released
Final version! (probably)
V4 introduces the Refinement speed hack (works great with a guiding video which depthflow uses)
Flux re-enabled
More electrolytes!
This I think is where I will stop. I have had a lot of frustrating fun playing with this and my other backend workflow for the speed hack, but I think this is finally at a place I am fairly okay with. I hope you enjoy it and post your results down below. If there are problems (always problems), post in the comments also. I or others will try to help out.
Alright Hunyuan. balls in your court. how about the official release to make this irrelevant. We're all doing this janky workarounds, so just pop it out already. btw, if you use this for your official workflow, cut me a check, I like eating.
btw, check out the other workflow on here, the leapfusion thing It actually works pretty well. less control over what you're going for, but closer to the original picture. both are cool to have.
Final update: (HA!)
Added Hunyuan Refiner step for awesomeness
Streamlined
Minor update:
V3.1 is more about refining.
Removed Reactor (pulled from Github
Removed Flux (broken)
Removed Florence (huge memory issue)
Denoodled
Added a few new options to depthflow.
V3: ITS THE FINAL COUNTDOWN!
Alright, this is probably enough. someone else get creative and go from here, but I think I am done messing around with this overall and am happy with it...(until I am not. Come on Hunyuan...release the actual image 2 video)
Anyhow, tweaks and thangs:Added in Florence for recommendation prompt (not attached, just giving you suggestions if you have it on for the hunyuan bit)
Added switches for turning things on and off
More logical flow (slight overhead save)
Shrink image after Depthflow for better preservation of picture elements
Made more stroking colors (Follow the black) and organization for important settings areas
Various tweaks and nudges that I didn't note.
V2:
More optimized, a few more settings added, some pointless nodes removed, and overall a better workflow. Also added in optional Flux group if you want to use that instead of XL
Added in also some help with Teacache (play around with that for speed, but don't go crazy with the thresh..small increments upwards)
Anyhow, give this a shot, its actually pretty impressive. I am not expecting much difference between this vs whenever they come out with I2V natively...(hopefully theirs will be faster though, the depthflow step is a hangup)
Thanks to the person who tipped me 1k buzz btw. I am not 100% sure what to do with it, but that was cool!
Anyhow
(NOTE: I genuinely don't know what I am doing regarding the HunyuanFast vs Regular and Lora. I wrote don't use it, and that remains true if you leave it on the fast model..but use it if using the full model. Ask for others, don't take my word as gospel. consider me GPT2.0 making stuff up. all I know is that this process works great for a hacky image2video knockoff)
XL HunYuan Janky I2V DepthFlow: A Slightly Polished Janky Workflow
This is real Image-to-Video. It’s also a bit of sorcery. It’s DepthFlow warlock rituals combined with HunYuan magic to create something that looks like real motion (well, it is real motion..sort of). Whether it’s practical or just wildly entertaining, you decide.
Key Notes Before You Start
Denoising freedom. Crank that denoising up if you want sweeping motion and dynamic changes. It won’t slow things down, but it will alter the original image significantly at higher settings (0.80+). Keep that in mind. Even with 80+, it'll still be similar to the pic though.
Resolution matters. Keep the resolution (post XL generation) to 512 or lower in the descale step before it shoots over to DepthFlow for faster processing. Bigger resolutions = slower speeds = why did you do this to yourself?
Melty faces aren’t the problem. Higher denoising changes the face and other details. If you want to keep the exact face, turn on Reactor for face-swapping. Otherwise, turn it off, save some time, and embrace the chaos.
DepthFlow is the magic wand. The more steps you give DepthFlow, the longer the video becomes. Play with it—this is the key to unlocking wild, expressive movements.
Lora setup tips.
Don’t use the FastLoRA—it wont work using the fast Hunyuan model which is on by default. Use it if you change the model though
Load any other LoRA, even if you’re not directly calling it. The models use the LoRA’s smoothness for better results.
For HunYuan, I recommend Edge_Of_Reality LoRA or similar for realism.
XL LoRAs behave normally. If you’re working in the XL phase, treat it like any other workflow. Once it moves into HunYuan, it uses the LoRA as a secondary helper. Experiment here—use realism or stylistic LoRAs depending on your vision.
WARNING: REACTOR IS TURNED OFF IN WORKFLOW!
(turn on to lose sanity or leave off and save tons of time if you're not partial to the starting face)
How It Works
Generate your starting image.
Be detailed with your prompt in the XL phase, or use an image2image process to refine an existing image.
Want Flux enhancements? Go for it, but it’s optional. The denoising from the Hunyuan bit will probably alter most of the Flux magic anyhow, so I went with XL speed over Flux's clarity, but sure, give it a shot. enable the group, alter things, and its ready to go. really just a flip of a switch.
DepthFlow creates movement.
Add exaggerated zooms, pans, and tilts in DepthFlow. This movement makes HunYuan interpret dynamic gestures, walking, and other actions.
Don’t make it too spazzy unless chaos is your goal.
HunYuan processes it.
This is where the magic happens. Noise, denoising, and movement interpretation turn DepthFlow output into a smooth, moving video.
Subtle denoising (0.50 or lower) keeps things close to the original image. Higher denoising (0.80+) creates pronounced motion but deviates more from the original.
Reactor (optional).If you care about keeping the exact original face, Reactor will swap it back in, frame by frame.If you’re okay with slight face variations, turn Reactor off and save some time.
Upscale the final result.
The final step upscales your video to 1024x1024 (or double your original resolution).
Why This Exists
Because waiting for HunYuan’s true image-to-video feature was taking too long, and I needed something to tinker with. This (less) janky process works, and it’s a blast to experiment with.
Second warning:
You're probably gonna be asked to download a bunch of nodes you don't have installed yet (DepthFlow, Reactor, and possibly some others). Just a heads up.
Final Thoughts
This workflow is far from perfect, but it gets the job done. If you have improvements, go wild—credit is appreciated but not required. I just want to inspire people to experiment with LoRAs and workflows.
And remember, this isn’t Hollywood-grade video generation. It’s creative sorcery for those of us stuck in the "almost but not quite" phase of technology. Have fun!
Description
FAQ
Comments (29)
I get this error:
Prompt outputs failed validation TeaCacheForVidGen: - Failed to convert an input value to a FLOAT value: rel_l1_thresh, None, float() argument must be a string or a real number, not 'NoneType' VAEDecode: - Required input is missing: samplesI solved that now I get:
SamplerCustomAdvanced|
shape '[1, 26, 48, 32, 16, 1, 2, 2]' is invalid for input of size 2635776
This is the Hunyuan vid section
So welcome to my journey haha. I fixed in by plugging nodes from KJ nodes in. (above the Hun Vid section)
Main line is between Get_deepImages and Set_smallPics. Main line uses the KJ resize image with divisable by 16 set.
The height and width input come frome a resized image into a KJ nodes Get Image Size that provide the scaled height and width wich plug into the resize node of the main line. That resize node in the main line makes sure the sizes are correct for Hun Vid
@Kyll Hi. yes, fun times. so I recommend leaving that squishy little area at 0.25. when choosing sizes, make sure its divisible by 32...I believe 16 works, but I stick to increments of 32, so 512 x 512, 480 x 544, etc...basically just add/subtract 32 and you should be fine. if you can divide the number by 32 and it gives a whole number answer, it works. I did add a nice little chart for help.
@saturngfx I know, but I don't want to do the math so I let the nodes do it for me.
I'm still getting weird glitchy results, so I was looking what the Hun Vid was giving me without the rest by setting denoise to 1. I think something is going wrong because I only get the colorful masaic as an output which to me looks like unresolved latent noise. I have no Lora's or anything turned on. It does change when I select a different shard from the LLama in the dual clip loader, which is weird. Might it not load the whole LLama mode properly?
I don't know what I'm doing wrong
@Kyll output at what stage? can you share your workflow? post it somewhere maybe? I'll grab it and see whats going on and see if I can spot the issue...but downloading the latest version and just running it (after changing to your models)..works, right? just something you changed did something odd?
@saturngfx I barely changed anything. And maybe I should wait a lil with commenting, because I since found out it atleast has something to do with the LLama. I had the model in 4 shards from huggingface. Per chance I stumbled on the model tab of my comfy manager and saw I could download it as a single file from there. Just tried the smaller model of the two. I now get a veeeery blurry image of the prompt, as through frosted glass. Now downloading the big model to see if that clears up the blur.
I'll let you know
@saturngfx I tried some stuff with succes. I also uploaded it to https://civitai.com/models/1170317?modelVersionId=1316774
My findings/journey:
TeaCache changed quality a lot. That gave me the frosted glass look, though more steps helped fix that but it got to the point that it led to the same render time with TeaCache as without and still had artifacts.
Then I found PerpNegGuider, once that was set up, holy shit it looks so much better (keep cfg at 1 for some reason).
It looks so much better even with no neg promt. Need to experiment more with the impact of the neg promt.
I tried plugging TeaCache back in again with the neg promt and the output with was drastically different. It seemed to make the neg promt have the opposite effect and add those to the image instead of detracting. Then I wonder if this happens without TeaCache too, just less obvious. And by upping the neg scale without, yes it does the same thing. I do not get how these latent vectors are being calculated behind the scenes and how the node calculates them. Setting the "empty conditioning" with the neg promt instead seems to add that to the image aswell. Maybe this can be abused in a good way?
Also surprised how well Flux Lora's work this well on this video model. Are the weights somehow similar? Or is it all from the clip part of the Lora? Normally, different model architectures shouldn't be compatible with each other. Glad it works though
Other experimented with is making a very crude mask to keep some elements like the background still, even with ugly seems in the input vid, at 0.5 denoise HunVid solves those.
@Kyll yeah, lots of tweaking and fiddling. I am throwing whatever at the wall and seeing what sticks. I seen your workflow. awesome. I grabbed it but gonna try it later. for now I am chasing something else. I discovered something interesting and focused on that moreso than this workflow at the moment.
how do you get the subject to move and not just the background?
ive tried a few things and cant seem to get it to move the person rather than the background
up the denoise a bit. see what .7 does. also try different seeds for the hunyuan. use terms like "a steady camera catches her walking" type stuff to let the model understand the direction, and of course use the right motion for that...all about experimentation. seed, denoise, and depthflow choices all play a role, and yeah, become a director...if you're using the circle depthflow, maybe mention the subject is bouncing up and down, if you're using the horizontal motion, then they are walking to the side. zoom would be walking towards the viewer, etc...and if you find good tips, post em. we are all beginners here.
Might delete this comment if possible lol... but I'm getting a mostly 2D-looking image that kind of pans like a fishbowl lens. Running on a 4080 32g RAM. Is there something I forgot to download? Also the TeaCache for Vid Gen & Load diffusion Model in the top right portion have red around their boxes. Is this my issue? Also there's no action movement besides the camera panning around from left to right. Sorry for the noob questions
its alight. noob questions are where we all start.
So, first, teacache. first off, make sure you're on 3.51 (the latest download). the most annoying thing about teacache seems to be a bit of a bug where the settings don't transfer over, so you probably have to set them yourself. its in 2 places, the main area and the refiner area. should be little notes on what it should be, so change it to 0.15 hunyuan_video (for the latest...ditch previous versions...although, new version..final final version will be out soon).
The first area you're seeing is just the depthflow...that is literally just the noise for the next bit. if thats all you're getting, it means its getting nerfed at the hunyuan video part...and 99% sure its because of the red nodes (click the dot next to the red circle to expand if it isn't expanded)...but once again, grab the latest version.
@saturngfx Edit: got it fixed. had to replace a few nodes and stuff. I guess I was just looking at the depthflowbase video on the older version. Question, is this supposed to look a bit.. off? The other videos look fine (still tinkering with it) but I'm curious if there's a setting I'm missing to make the video really pop from the start before refinement. Thanks man
@mmlynch23 Umm, hard to say where the issue may lie. obviously bigger sizes will make for better resolution at the cost of time. I am doing a final super final I am done version 4 and hopefully be out by the end of the week if I get time to actually work on it. But yeah, higher resolution, or up in the squishy area, I...want to say its a .25 so maybe kick that to .5 if you got the GPU to handle it...mostly its about experimentation and finding that sweet spot between speed and fidelity. just make a save of what works, then resave it as something like. V3.51 Experimental and have fun testing and tweaking. its how we all learn :)
@saturngfx I just can't get the teacache video sampler to work. I've tried everything. Installing from github, installing from manager.. nothing works. Any ideas? Thank you for your time
@mmlynch23 hmm, not sure. sounds like some other thing is clashing with it maybe.
What I would do: just for sanity testing, I would make a new install (different directory, different conda environment, etc) and install just comfyui, manager, and teacache. test it on the most basic of regular workflows. working? cool, drag/drop another node into the new environment. restart, test, etc...keep going until you find the clash and what borks it.
Comes from many years of modding skyrim and rimworld to the nines...sometimes its just the drudgery of going through big lists to filter down the problem child. I think its worth it because teacache is quite helpful for speeding things up without losing too much quality.
any tips for depthflow reuslts ?
my previews are extremely distorted but I was using 2d examples maybe thats the issue ?
also my final hunyuan videos are low quality even when going trough the upscaler
all about experimentation. up the resolution, up the denoising until you're hitting that sweet spot. just trial and error, but hard for me to suggest where to go unless you feel like posting what settings you're going with, such as size, denoising, what your refiner denoising is at, etc. really denoising on refiner should be around maybe .40 so it can just tweak whats made.
i am using the default setting with a 512x512 source image , didnt really tweak anything except for the prompt
upload your workflow somewhere and link it to me if you can be bothered. I'll see if I get the same issues and if I can track down what might be going on.
oh, and for testing, try 768x768. all about testing. trial and error. I don't proclaim to know it all...or much...anything really. just found a neat trick with depthflow and seems most are having an alright time, but yeah, milage may vary. poke around and if you break it, just redownload..its free :)
Help!!?? what is "T2V Tea Sampler"
I install all needed nodes. still a red box missing node.
teacache. you should install it, but you can bypass it if you can't be bothered. You might have to swap out the teacache sampler with the hunyuan though if thats also giving you an error. teacache helps improve speed
remove the old one and the new node and link the same again
you will find 2 nodes
1- HunYuan ( Test out Loras and denoising)
2- HunRefine (WORTH IT!!!)
@Dalleno remaking for v4, all this will be sorted nice and without teacache anymore (found a better method). I am forever refining. :)
Hmm getting this kind of error:
RuntimeError: shape '[1, 19, 20, 15, 16, 1, 2, 2]' is invalid for input of size 408576On the actual Hunyuan generation part. I had to switch out the T2V Tea Sampler for the SamplerCustomAdvanced, plus I only have a 4060, so feasibly could be too little memory.
Any ideas? Great workflow though!