"Wait, doesn't Klein already know how to use depth maps?"

Sort of. Not really. It's inconsistent at best and there's no control over the strength that you get with a real controlnet. This LoKR aims to address that.

Trained with an experimental "double sided depth" technique using Perceptual LoRA Toolkit, which conditions the training on both a depth reference and a custom perceptual depth loss.

This is preliminary version and is trained on only 550 images. It should be able to handle a good range of scenarios requiring only short prompt guidance for things like specific clothing, setting, etc. I made it mostly to address Flux's anatomy and pose problems, and it is trained entirely on human subjects. It should also be able to handle non-human subjects to a point.

You will probably get watermarks in some images. I am trying to put together a cleaner dataset, but depth loss punishes you if you have watermarks. Having even a few will get baked into the model.

Use at strength 0.3 - 1.3 depending on how closely you want to match the reference. I recommend starting at 1.

You may also add other reference images, though this may give unexpected results. Try using other references at 0.1 megapixel to start and work up as larger references can overpower the depth reference.

The images in the gallery have Comfy workflows embedded with nodes that will create depth maps from images you provide.

You may optionally add a background removal node if you have images with backgrounds that enter the depth map.