Hand Segmentation for ADetailer
Credits to mnemic for this article and Anzhc for this ADetailer model (see for more information)
Installation:
Download the zip archive
Extract the model into your ADetailer model folder
For the WebUIs like Auto1111, Forge and SD.next it should be in models/adetailer
For ComfyUI it should be in models/ultralytics/segm
Start it up (full restart if already had it open) and select the model from the ADetailer tab/node (tip for webui: you can go to Settings→Adetailer and change the number of models that can be used up to 10, very useful if you want to use more than 2 ADetailer models)
Make sure to have correct Detection thresholds/inpaint strength and so on for your images
Generate them hands!
It detects hands good enough for what SD hands can be. I might work more on this in the future. Please tell me what you think. Thanks! 😊
Description
Increased Training
Might detect hands it didn't before.
Might fail to detect more messed up hands.
Try and see if this works better or worse for you.
FAQ
Comments (22)
Is this model for hands or eyes? Because the description is talking about eyes.
@Meatstorm87 This is a model for Hands. Thank you for noticing! I might've pressed that Template button and not realizing when creating this.
able to use in sdxl pony?
@elternal123456942 Yes. ADetailer models can be used for all Stable Diffusion versions (and/or anything that supports Ultralytics). Just not in the onsite generator yet. 🔜™
Hi, I get an error RuntimeError: The size of tensor a (13) must match the size of tensor b (12) at non-singleton dimension 3. Do you know how to fix it?
@low_channel_1503 Hello. I don't think this error has anything to do with the ADetailer. This could be anything from ControlNet compatibility, a LORA compatibility with your Checkpoint, if you are doing an XYZ plot there might be an issue where it's not detecting stuff properly, without much context I cannot help. Make sure to always use models that are compatible. If for example you use an SDXL checkpoint, make sure the lora/controlnet/etc is made for that model. Try disabling stuff and figure out where the problem happens, I may be able to help you if you can provide more info on what exactly you are trying to do.
What is the difference between versions?
Also, updates would be greatly appreciated ^_^
Thanks.
From the version 1b changelog:
Increased Training
Might detect hands it didn't before.
Might fail to detect more messed up hands.
Try and see if this works better or worse for you.
Weird, I didn't receive a notif for this but got one from @Jaidchen. @User37 please look at the about this version on the side or look at Jaidchens comment here. Updates are planned. ❤️
This might be heavenly virtue because I was waiting for the other creator that made the eyes one to make the hands part too but never really updated their model... (It's still good tho)
I'll use and test this to see if it can fix my crippling hand problem, if so... I will literally cry from happiness, thank you so much for this!
Edit: Sadly, I forgot to edit this to say this didn't work for me sadly, it was just a bit too bad at anime styles and not as decent as the eyes adetailer for the eyes from another person, might work for anyone using this tho (and for some reason, no updates for almost a year)
I gotta say, this adetailer model does a great job of making hands look amazing, BUT in many cases, for weird reasons, the detected mask is fine, but the inpainting does a BIG MESS of the hand and transforms it into something else entirely?! This happened to me with other adetailer models, but much less often.
@princess_daphie That can happen, yes. I've seen it in pretty much any adetailer, and in normal inpaint. This is not a problem with adetailer or the adetailer models. This happens because the SD model doesn't have enough context on the surrounding parts of the image, and it generates whatever it thinks should be where the mask is, depending on the prompt and other settings. To fix this, for example, for hands you must add a prompt in your adetailer model settings for hands so the inpaint pass uses that instead of your full prompt. Additionally, you can play with the denoise strength and other settings to achieve the desired result. You can also increase the padding pixels, so the SD model can see a wider view of the mask and surrounding parts, so it gets less confused. I recommend not increasing it too much as it can make the inpaint less detailed in some cases (?). Any other trick you use for standard inpaint will work for adetailer more or less. You can read this article for some more details from FallenIncursio about Regional Prompter and Adetailer. If you need more help, you can ask me.
Hi. Multiple model scanners reports this model as unsafe.
Unsafe operator found:
- Severity: CRITICAL
- Description: Use of unsafe operator 'getattr' from module '__builtin__'
- Source: PitHandDetailer-v1b-seg.pt:best\data.pkl
@G1NG3RB3ARD Yes that is the default behavior of pt files. Unfortunately there is no way to change that as of now (ultralytics doesn't have a way to do it). As long as you download from here on Civitai, the file is safe.
@Pitpe11 This is partially true. While loading a .pt is inherently risky, your model uses "getattr" which can exacerbate risks if used unsafely during deserialization. If you look some of the YOLO model, they avoid using "getattr". Quite a few models from civitai dont use "getattr" either.
If you take a deeper look. just about all BBOX models avoid using it, However SEGM models seem to almost exclusively use it.
Loaders like "ComfyUI-Impact-Subpack" will load bounding box models safely. But will avoid any using "getattr"
A easy method would be to release bbox versions without the use of "getattr". I don't know enough about SEGM to know if that's possible.
@taithrah If you can tell me how to remove "getattr" from the trained file, sure ill do it, but this is whatever you get from the ultralytics training packages. From how I understand it, the model file is not just a bunch of numbers that can be put in a safetensors format, it has to have whatever script ultralytics uses to work properly, BBOX or SEGM.
This is a really great model for Inpainting hands! Much better than the base model hand_yolov8n. If possible, please make same model for legs/toes.
do you have any for the feet?
@hanada_civitai https://civitai.com/models/257904/adetailer-footyolov8xpt there is this model
I don't have one, might make one along with a bunch of others maybe soon™️
This is based off of hand_yolov8n.pt right? If you could make a version based off of hand_yolov9c.pt that would be awesome. I find that 9c works a lot better for anime images over the 8n version as it is better at detecting and almost never gets a wrong detection at 0.55. A segmentation version of that would be perfect.
@HaloSkull I actually had trained a v9c seg last year, but it was just a test. I don't remember how well it performs compared to the v8. Either way, I might release it and maybe get feedback and finetune if needed. v8 is the preferred version since v9 needs allot more compute for the same result, but in theory it should perform better. v10 v11 and v12 should be even better and require way less compute, but I'm pretty sure those are not supported in adetailer.
@Pitpe11 If you wouldn't mind sharing that version that would be awesome.