Eyeful | Robust eye detection for Adetailer and ComfyUI for any digital images.
Version 2️⃣ Update:
The most thorough and powerful eye detection is back and better than ever with a massive v2 update. By prior request and to fulfill multiple needs better, the model has been retrained twice, in both paired eyes and single eyes for any use cases, with a much bigger data set.
Dataset: ~7600 custom annotated images.
Training: ~440 Epochs per model, with 60 total runs to tune hyperparameters.
Sleep lost: A bunch.
🖌️ The rules for detecting eyes are carefully honed to select for things people would want to in-paint or highlight, to attempt to avoid false positives:
Match:
☑️ Pairs or individual eyes on any face like structure
☑️ Beady black eyes like animal eyes
☑️ No eyes, but bright highlights in a space where eyes would be e.g. under a hood
☑️ Side-facing eyes if sclera is visible
☑️ (For paired) Single eyes if only one is visible
Ignore:
❌ Goggles, sunglasses, or any types of visor
❌ Skull-like eye sockets or holes, absence of eyes
❌ Flat black eyes with no highlights
About: Eyeful is a custom trained YOLOv8 model meant to be used with ADetailer or any other bbox/detection compatible plugin for Automatic1111 or ComfyUI. This network can also be run completely standalone with Ultralytics.
Detection: This model has a robust custom training dataset, and a custom training pipeline that means it can detect eyes that are in various styles such as painting, artistic, 3d, realistic, anime, manga, even some pixel art.
Settings
Detection Confidence Threshold: >0.40
Model Location (A1111): Extract zip into models/adetailer folder
Model Location (Comfy): Extract zip into models/ultralytics/bbox folderFailure Cases: There are reasonable limits where an eye may be too stylized or abstract to be highlighted without introducing too many false positives in the training, so this model tries to straddle the line of what you likely would consider an eye worth highlighting, human-like animal eyes: yes, sunglasses: no, etc.
Extreme closeup images will trigger multi-detections if the whole frame is eyeball, but at that point you shouldn't be using this model anyway.
Description
FAQ
Comments (11)
I haven't been able to duplicate the issues of the other comment posters so let me know if the model's working for you by leaving a like or a review, thanks and enjoy :)
If you notice an image style that is very difficult to get detection working but should actually work like oil paintings or something, let me know by leaving a comment and it could updated in future versions to be better.
Works ok with the following settings:
Adetailer Model 2nd: (Use 1st for face)
If you want real changes, above 0.7, Mask Merge: Merge, masked 72
CFG 7, 1.0 denoise
CFG 9, 0.8 denoise Sampler DDIM 30 steps
Mask merge mode set to MERGE is key (this makes both eyes painted in reference to each other, if you don't do that it'll frankenstein eyes due to making two separate, non referenced passes and look awful (or, great if you're into that thing, no judge you fucking freaks <3)
1st photo is vanilla, PNG info will show you the rest in A1111
To add this works only well if there is only 1 person in the frame, 2 might work still but anything above 2 people will also have frankenstein eyes.
@Dutchgrasss Yeah absolutely this is for 1girl, solo kinda stuff. At this point I run 0.6 denoise, and just the regular sampler, regular config, Instructions above are outdated but they still work. for more than one person you can't try to use it with a much, much larger mask but probably not. The only way for this to work is to detect sets of eyes, not single eyes
Here everything works normally. Now i just learning how to operate. Path I installed: .../stable-diffusion-webui/models/adetailer/Eyeful_v1
yes
Sorry, I installed it on the same path but it doesn't work properly.
@cbulls D:\pinokio\api\comfy.git\app\models\ultralytics\bbox
comfy.git\app\models\ultralytics\bbox
Hi, great work. may I ask how to train such eye model? I want to train my eye work. THANKS!
So under the hood, AfterDetailer and the bbox finding nodes in ComfyUI use a python library called Ultralytics. If you train a "YOLOv8" network on a dataset that just has one class, then save the model file, you'll have a compatible model that works with adetailer, plug and play.
Ultralytics training: https://docs.ultralytics.com/modes/train/
Software for labeling: Any image annotation tool that uses bounding boxes like labelimg: https://github.com/HumanSignal/labelImg
The rest is creating a data set of images (or finding an existing one) annotate them, then using the ultralytics trainer to load a preexisting model like 'face_yolov8n.pt' and retrain it on your own data, either on your local computer or on Colab (https://www.youtube.com/watch?v=LNwODJXcvt4).