Depth LoRA
Convert depth maps into high-quality images
Introduction
This is a depth LoRA that converts depth maps into high-quality images. It was trained in multiple stages on a variety of datasets to balance overall structural accuracy with fine-grained detail reconstruction.
HF download:
https://huggingface.co/lrzjason/Krea2-Lora-Collection/tree/main/loras
EditUtils
https://github.com/lrzjason/ComfyUI-EditUtils/tree/main
Training Details
The training followed a two-stage, progressive-resolution strategy:
Stage 1 — 512 Resolution Pretraining
Trained on a 2,000-pair subset sampled from a public depth dataset, with all captions regenerated by a VLM (VLM re-caption) to improve text-image alignment. This stage used a constant learning rate of 1e-4 to establish a solid depth-structure prior.
Stage 2 — 1536 High-Resolution Refinement
Continued training on a 1536-resolution depth dataset, focusing on detail reconstruction (edges, thin structures, and fine geometry). This stage switched to a cosine learning rate schedule for more stable convergence.
Stage 1 512 res, 2,000 pairs (public depth dataset + VLM re-caption), constant LR 1e-4Stage 2 1536 res depth dataset, cosine LR scheduleTotal Epochs50+
Usage
⚠️ This LoRA must be used together with the EditUtils plugin:
👉 https://github.com/lrzjason/ComfyUI-EditUtils
Install the ComfyUI-EditUtils plugin.
Load this LoRA in your workflow.
Feed in the source image to obtain the corresponding depth map.
Notes
Works better on 10240- 2048 with long captions; with higher-resolution inputs yielding richer depth details.
Will not work as intended without the EditUtils plugin.
RH Workflow
Experience link: https://www.runninghub.ai/post/2082077636234313729/?inviteCode=rh-v1279
📬 Contact
𝕏 Twitter: @Lrzjason
📧 Email: [email protected]
💬 QQ Group: 866612947
💬 WeChat ID: fkdeai
🎨 CivitAI: xiaozhijason
Description
FAQ
Comments (21)
It doesn't work from the word at all, just a depth map at the output, using MoGe, maybe because of the MoGe node?
it must use with editutils node
@xiaozhijason I used your workflow, and the editutils nodes. maybe it doesn't work with MoGe or the krea2_turbo_int8_convrot model?
@soyv4 I was used with krea2_turbo_int8_convrot model for testing. Could you post image and caption you used? It perfer long caption which describe the wanted target.
same problem, with your workflow
@LbhMan what model, image, prompt do you used? Could you provide some information for me to check? I tested in local and runninghub both work fine.
The workflow fails because the EditUtils: Qwen Config Preparer Irzjason node requires a configs input, but no compatible node in the workflow provides it. As a result, the workflow reports missing required inputs and cannot run. Is the workflow missing something, or is this a bug in the node/workflow? Do I need to adjust something?
configs input is optional. It is a self chained node. Qwen Config Preparer Irzjason node->Qwen Config Preparer Irzjason node for multiple image inputs. It is not the reason "the workflow fails". What is your configs?
@xiaozhijason He's right though, I'm getting the same error using your workflow out of the box. Here's a copy-paste:
Validation failed
EditUtils: Qwen Config Preparer lrzjason couldn't validate configs: tuple index out of range
Configs is set as optional in nodes.py but ComfyUI still interprets it as required. It is the reason the workflow fails.
Correction. The error message points to the Configs but deleting the preview image node between the depth node and the config preparer node makes it work for me.
@Optimum I confirm that also worked on my machine™️! And the LoRa works great! Thanks everyone!
Hi! Looks AMAZING! But... the "configs" for the EditUtils are not optional unfortunately. What can I input as configs? Also, I cannot find the RT_Captioner node anywhere, can you please help? Thank you SO MUCH! GREAT work!!
Never mind, when I detach the configs on the right side it works. But! It doesn't work AT ALL. The depthanything image doesn't seem to get used at all. Can you maybe create a workflow that functions normally with ComfyUI Manager? E.g. node packs that can be installed automatically, all nodes available et cetera. Maybe then it could work. And otherwise, oh well, I'll wait for the Krea2 creators to build the definitive ControlNet.
@Chuck648 It is avalible on comfyui manager. If configs is not optional, it means you used old editutils version, you need to update it via git or manager. It also contains krea2 edit workflow inside the repo. https://github.com/lrzjason/ComfyUI-EditUtils
@xiaozhijason I cloned the repository yesterday using git clone, so it should already be the latest version. I haven't used an old copy of the repo.
@bamib1971533 it should contains Simple Krea2 Depth.json under workflows subdir. You might try that
I got it to work! This is ***THE BEST DEPTH CONTROLNET*** I've ever used! Better than SDXL ControlNet, better than the Flux Dev Controlnet. Thank you SO MUCH!!
Hello,
im getting this error, im using your workflow "deep control workflow" i think it might be somthing with last comfy update ? it show that config node must be connected with somthing..
Validation failed
EditUtils: Qwen Config Preparer lrzjason couldn't validate configs: tuple index out of range