Support me so I can make more models, faster: https://ko-fi.com/the_cook
This IC-Lora and accompanying workflow allows you to input a video, and get a missionary scene in return, influenced by your input video. The resulting video will keep the identity of the subject, and even certain nuances like facial expressions can be transferred to the new scene, resulting in a more believable identity.
For this to work, YOU HAVE TO USE THE WORKFLOW, which includes the BFS nodes that make the subject identity stick. That also requires an additional lora trained specifically for the BFS nodes to function: https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/blob/main/Best_FaceID_v1.0_LoRA.safetensors
LTX is notoriously bad at keeping likeness, which is why it took me so long to develop new loras for the architecture. I've trained a lot of new models for it, but the likeness was always only 80% there, and so I couldn't justify releasing the results. This method finally solves it, and I'm going to release future LTX models using this method because it works so well. Shout out to Alisson for persisting in trying to solve the problem for months.
Using Kijai's int8 version of the LTX dev model will run faster and use less VRAM: https://huggingface.co/Kijai/LTX2.3_comfy/blob/main/diffusion_models/ltx-2.3-22b-dev_transformer_only_int8_convrot.safetensors
This Distill lora is better for i2v and v2v than the normal distill: https://huggingface.co/TenStrip/LTX2.3_Distilled_Lora_1.1_Experiments/blob/main/ltx-2.3-22b-distilled-lora-1.1_fro90_ceil72_condsafe.safetensors
Using the DMD distill makes LTX better: https://huggingface.co/TenStrip/LTX2.3_DMD_Lora/blob/main/LTX2.3_DMD_reshaped_r256.safetensors
Description
FAQ
Comments (33)
Cooking again
Let him cook.
every file at https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/tree/main i cannot download. says there is nothing AccessDeniedAccess denied
This XML file does not appear to have any style information associated with it. The document tree is shown below.
<Error>
<Code>AccessDenied</Code>
<Message>Access denied</Message>
...
</Error>
https://huggingface.co/Alissonerdx/LTX-Best-Face-ID/blob/main/Best_FaceID_v1.0_LoRA.safetensors should work
Unless there are technical issues on their end
Problem on hf everywhere apparently. Uploaders get the same error other than for few kb files, even images.
I got the same trying to download from the huggingface website directly at the moment but the HF CLI worked for me.
@The_Cook k yea it worked later on. i hope this can work in wan2gp someday. will store these loras for safekeeping until then :D
Most Workflows are simply not working due to the fact that important nodes cannot be downloaded.
LTXaddVideo cannot be downloaded.
If the driving video is being used just for character consistency, couldn't a static image do the same?
My guess is that a static image still missed that 20% he was talking about due to motion and different angles. But I don't know any better. It would be really incredible to give multiple images as a batch or even transformed into a back-to-back input or something like that.
You can use a static image if you want (just repeat the image a few times and feed that in), but the benefit of using video as a driver is that the model picks up what the person’s mouth looks like both open and closed, what the person’s face shape is from different angles etc., and for a static image those details will have to be hallucinated.
@The_Cook Understood, the video is like a multiple reference source all in one, I can see the benefit over just a static image... but yeah, being able to add multiple image references would be nice, but I'm almost sure there are already wfs and loras for that. EDIT: so, in theory, if I make a loop video using different static images of the character at different angles, would that work as well?
@ArtiOtaku You just have to repeat the images (use VHS Repeat Images node) at least 8 frames or multiples of 8 per image (since LTX counts 8 frames as one latent). Then batch all images (using kjnodes' Image Batch Multi) and input that into the Resize Image v2 node in the wf.
The total input frames don't necessarily have to match the target frames. you can feed it less that what you generate, and it'll still work (same goes for video, you can cap the input frames and have a higher target frame count)
@The_Cook Thanks for sharing that. It was what I was trying to say but I'm also not knowledgeable at all in this haha
Amazing work! Keep it up!
even better with eros 1.4 I assume?
also so DMD > condsafe distiller?
thank you for this lora :p.
Based on the name i think there will be more sex position and sex act later.
I really hope you will make the wan insertion lora and BJ multisene for LTX, they are amazing :)
is there reason for this not to just be standard lora for I2v? Is there some advantage to making it V2V ic-lora?
You can use a static image if you want (just repeat the image a few times and feed that in), but the benefit of using video as a driver is that the model picks up what the person’s micro-expressions look like, what the person’s mouth looks like both open and closed, what the person’s face shape is from different angles etc., and for a static image those details will have to be hallucinated.
This is outstanding! First time that LTX actually worked for me, thanks!
Comfy not downloading dependencies!
How to solve, guys?
Solved by Vpn
Wow! Even works for random scene change. It's like making every txt2vid into a img2vid without having to make the whole starting image/background.
holy shit this is amazing! i was fully ready tos truggle for like hours trying to get this to work but no, just right off the bat.
What's the correct ComfyUI-BFSNodes version? The latest one (1.10.8) doesn't have fault option on ref_resize_mode on LTX Identity Tranfer node.
Choosing either setting there should give the same result. WF was made with version 1.10.6. I have updated the workflow to comply with the changes in the updated node
Wow, this works great! Really impressed with the output. Thank you!
Pretty rad.
The facial consistency is amazing, my prompt gets ignored somewhat, for example if I wanted to remove tanlines or change breast size the output is always the same no matter what. Any fix for that?
Amazing work, please do more of these!
After a small bit of trial and error, it works! It's the first LTX workflow I've gotten to work, so that's awesome. One thing I've noticed is I'm not seeing the facial consistency that everyone else is talking about. It actually makes my character appear a bit older with different facial structure. The video I'm using has a very clear opening frame of the character's face but the final video doesn't really match the initial video's face. Not sure what I can do to maybe improve that on my end. I'm using the workflow provided and everything required in the post here.
Isn't there a projector lora we have to download for the identity transfer node? I don't see the link to that here.
There is a projector model on the same github as the ID lora, but in this workflow it's not needed. The ID lora alone is enough in most cases