This is something I have been chasing for a while. Multiple characters from separate LoRAs interacting with each other. There are many regional conditioning solutions available, but as soon as you try to add LoRA data to the conditioning channels, the LoRA data seems to overrun the whole generation.
A8R8 just released a clean regional prompting solution - https://github.com/ramyma/A8R8_ComfyUI_nodes
If you blend that with ADE_RegisterLoraHook and ADE_ConditioningSetMaskAndCombine nodes, it is possible to have the LoRA constrained to its section of the generation.
Description
Initial Version
FAQ
Comments (13)
won't work if you have a 12GB gpu 😢
I imagine it would work with 1.5 models though.
Sorry to hear that. If you get it working with 1.5 models, I will add it as a new version on this page.
I'll try it soon on my 12gb 4070 super. If this doesn't work because of VRAM constraints, it'll probably be a good starting place to make a custom workflow that does something like generating a low step low poly base image to start with and then does each character one at a time on top of that as a way to ductape it together.
@gwshdude It will work on your 4070 because I can run it on my 3060, but the process takes nearly 17GB of memory when using SDXL models. Offloads to RAM and brings the speed to a crawl, but the wait is worth it. I wish there could be a way to unload something during the process and make it more efficient.
@gwshdude 4070 super with only 12gb.. Damn.. Nvidia be fuckin their customers in the ass Hard.. I got a 4060ti with 16gb(not that it's ever truly enough when it comes to AI..), and it don't look like they boostin Vram for anyone but the top spenders with the 50 series either.. No wonder that stupid corporation has enough money to buy a moderate sized country..
nice work! is it work for sdxl and high res fix? thanks a lot.
How much loras can interact at the same time? thanks.
Currently it is only set up for 2 loras and SDXL. I think I might need to make it a little more flexible. Lets say this is version 1 with more solutions to come.
@joeandolina987 Thanks for your reply, quite expecting the further updates.
thanks! also seems to work with rgthree power prompt to load the main prompt with style loras and the like
Can you show me what you have done? I would love to make the workflow bettter
https://iili.io/dKoUuWb.png here. i plugged in the sdxl power prompt by rgthree. can use the normal power prompt seen on left as well. that allows to load how ever many loras into the base prompt as you want.
@schmongo Nice move! I'm going to start using the power prompt in my workflows.