Just Released ControlNet Union for SDXL
1 Model Does it All (Pose, Canny, Scribble... and so on).
No more need to hold onto so many different Controlnet Models.
Choose witch preprocessor you would like to use from the drop down menu.
Image chooser also included so you don't waist time completing unwanted images.
This is just a basic workflow may need tweaking for certain pre processors.
Link for the new Controlenet Union Model:
Description
Added:
1: Mask layer for Depth preprocessor
2: Face or Body Detailer
3: Eye Restoration
4: Upscale with 2nd K sampler
5: Lora for Face / Body / Eyes
6: Save at multiple Stages as Needed
7:Image Comparers so you can see your changes at each stage
This is a bit of a resource hog but can still run on 8 gigs of v-ram albeit Slower.
Mask layer is for Depth and Zoe-depth ONLY.
Suggest leaving Clip Skip switch Off, using clip skip can give interesting images but tends to ignore your prompt. (fun to play with)
FAQ
Comments (9)
Everything seems to work so far! Any tips for a complet noob at this?
I'm new to using ComfyUI, but I've been figuring out errors along the way. I got the checkpoint, upscale model, controlnet model, etc.. I get to the Preview Chooser 4 sample images and they don't look like the lora I used or the original person. I even tried an original image and lora of the same person and it doesn't look like them. Does this not take loras into account? Or do you think some other setting is causing this? Same with the Image Comparers section at the end.. they don't look like the original person or the lora, even when they're the same people.. the result is a different looking person.
using preprocessor "None" instead of "depth" or others seems to have fixed the face to look similar to what it should.. at least up to the Preview Chooser step. I still need to mess with the enhancement options beyond that.
using preprocessor "None" is not a fix for the loras seemingly not factoring into the results, that's just combining with the original image.. so that's the only reason the face starts looking similar... using the others, like the default 'depth' option, is much more flexible, but the results don't look like the subject lora at all. Please add lora support in the next update, unless I'm missing it somehow already being there, but it doesn't seem to work when using them in the prompt
This is not meant to be used for copying faces it is for poses. what you need is Masked Conditioning with an ipadapter and that will maintain the faces. Check this out, https://civitai.com/models/536108/masked-conditioning-with-2-photo-ipadapter-sd15-and-sdxl
@john_wil111 Thanks, I'll check that
When using open poses, the extracted poses are clearly still present in the generated results.
Do you know the cause?
Just realized the resolution for the preprocessor needs to be lowered from 1024 to 512 and that will fix the problem with pose.
It seems that a resolution of 512 and a strength of 0.90 on the apply controlnet works best with Open Pose.