Pototype ControlNet models based on MediaPipe for pose and hands estimation. Higger "e" numer is better estimation but also greater impact on image. You will need external preprocessor available here!
Usage guideline:
1. Download preprocessor and txt file from gumroad
2. Install requirements - run pip install -r requirements.txt command (in folder where you have downloaded file)
3. Prepare folder with images that you want to preprocess
4. Run command python preprocess.py -mh -mp -s C:\path\to\your\folder - mh is for hands detection, mp for pose (you can try with just the pose which works great!)
5. Inside selected folder will appear detection folder (C:\path\to\your\folder\detection)
6. Copy downloaded models (.ckpt files) to ...\stable-diffusion-webui\extensions\sd-webui-controlnet\models
7. Inside Automatic1111 GUI select ControlNet enabled, preprocessor to None, one of the downloaded models and put image from earlier detection
8. Generate!
Description
FAQ
Comments (26)
Great but I'll wait for easier installation
A bit clunky to use (currently) - but it works as intended! The clearer the input image is, the better. Especially poses from MagicPoser (or something similar) with a grid on the ground produce immaculate results. The ability to just detect the hands is incredibly helpful for my workflow. Thx for this outlook of what's to come =)
preprocessor works good,
but final image is not,
fingers are still ....messed
is this work good on feet toes?
Based on the fighter image feels like it needs more steps as you can see even in the sample it didn’t quite follow fingers right and corrupted a bit
Sir, I'm thank you but I can't download e11 to original size (8.0gb).
Reuploaded, should works now.
@natakaro It's works sir! Really thank you!!
The download size 8GB but I only get a file that is a few bytes.
Reuploaded, should works now
@natakaro Thank you! I will check
You need to click eight giga times.....
Please fix the e11 version. It says 8.01 GB, but only downloads a small file. (Which contains this XML: <?xml version="1.0" encoding="UTF-8"?><Error><Code>ExpiredRequest</Code><Message>Request has expired</Message></Error>)
Reuploaded, should works now
hello there. I wanted to ask how you trained this and also say if you could train a model that is just for the position of the hands and the fingers without the rest of the body that would break the internet
You can try this model - just use preprocessor only with -mh parameter, recommended to use it for inpainting because it affects image.
@natakaro -mh parameter... maybe I'm just a noob but I don't know how to set parameters
@kleind When you write command use python preprocess.py -mh -s C:\path\to\your\folder without -mp
Here's a patch that will make it work on linux/WSL2 and fix some of the other issues I had with it.
--- ./preprocess.orig.py 2023-04-20 19:28:33.562081300 -0400
+++ preprocess.py 2023-04-20 19:29:19.172127900 -0400
@@ -77,13 +77,15 @@
parser.add_argument("-mp", "--pose", action="store_true", help="Declare pose detection")
args = parser.parse_args()
-path = args.src + '\\'
-detectDir = path + 'detection\\'
+path = args.src
+detectDir = os.path.join(args.src, 'detection')
files = os.listdir(path)
fullFiles = []
+
for f in files:
- if f.endswith("png") or f.endswith("jpg"):
- fullFiles.append(path + f)
+ if f.endswith("png") or f.endswith("PNG") or f.endswith("jpg") or f.endswith("JPG"):
+ fullFiles.append(os.path.abspath(os.path.join(path, f)))
+
IMAGE_FILES = fullFiles
with mp_pose.Pose(
@@ -129,8 +131,9 @@
if not os.path.exists(detectDir):
os.mkdir(detectDir)
+
if draw:
- fname = os.path.basename(file)
- cv2.imwrite(detectDir + os.path.splitext(fname)[0] + '.png', annotated_image)
+ fname = os.path.join(detectDir, os.path.basename(file) + '.png')
+ cv2.imwrite(fname, annotated_image)
I have a modified version of this that runs on Linux/WSL2, but don't know how to get it to you.
What about publishing SafeTensor version? Would you consider it?
Is there a way to use this with comfyUI? That would be awesome!