Update the plugin to latest version to avoid incorrect placement during large buffer and close to edge mask
Comfyui-In-Context-Lora-Utils (20241214)
v3.4 update:
Use kontext to improve workflow
v3.3g update:
a simple workflow for dev ic lora using new nodes.
v3.3f update:
Fix missing connection on crop mask to original result
Usage:
Input two images
mask the target area from image 1 as reference
mask the target area from image 2 as editing area
Parameter:
https://civarchive.com/articles/9292
中文视频讲解：
https://www.bilibili.com/video/BV19MiZYJE82/?vd_source=1f3b87e4654ce717c5f87bb51a78ffc0
Sponsors me for more open source project:
Buy me a coffee:
v3.3 update:
Add auto mask before final output.
Move parameter above load image section for more space.
Adjust fit image logic, update logic to avoid cropping image.
Adjust plugin calculation to avoid incorrect replacement.
v3.2 update:
add prompt and redux switch.
add one step refine.
Update the latest plugin, 2024-12-05, for v3.2: Comfyui-In-Context-Lora-Utils
v3.1 update:
Add AutoPatch Node to avoid patch mode and patch type selection.
Fix incorrect width height in edge case which lead to incorrect resize and incorrect replace in final result
v3.0 update: To adjust lossy detail issue, I develop new nodes for this context window editing workflow.
Download the latest plugin for v3.0: Comfyui-In-Context-Lora-Utils
v3.0 workflow enable any context window editing with provided mask and image.
It could create very small context window from source image and replace it back to the original image.
v2.4 update: Based on another person created workflow, improve image process logic with my Comfyui-In-Context-Lora-Utils
PS: demo image is created by 书墨旅人 from a wechat group.
v2.3 update: Two pass sampler to increase the consistency
v2.2 update: Fix missing flux guidance issue
v2.1 update: Remove Lora from the workflow
v2.0 update: Add Target Image And Accept Mask Input
PS: Need the latest Comfyui and newly release inpainting flux model
PS2: You might need to adjust seed a few time to get the best result
Download Inpainting Flux Model (Fill) Here:
https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev/tree/main
V2.0 extention:
https://github.com/lrzjason/Comfyui-In-Context-Lora-Utils
## How to install
Download the zip file.
Extract to ..\ComfyUI\custom_nodes.
Restart ComfyUIFor try on lora, you could download from below:
https://civarchive.com/models/950111/flux-simple-try-on-in-context-lora
v1.3 update: added new plugin for simplify in context lora workflow
v1.2 update add more description and tips
add Ratio Based on 1024 for high resolution generation
The remove background node need the ckpt from below url
https://github.com/chflame163/ComfyUI_LayerStyle?tab=readme-ov-file#TransparentBackgroundUltra
As newly discoved in context lora usage of Flux.
https://ali-vilab.github.io/In-Context-LoRA-Page/
We could use image as condition to generate similar output.
I introduce this workflow to simply remove the ic-lighting and inpainting models for product background editing.
Please use with this lora:
https://civarchive.com/models/933026/flux-product-design-in-context-lora
Modify the Main Description to descript the input image.
Modify the Extra Description to descript the wanted image. You could refer [IMAGE1] in Extra Description.
For example:
Main Description: This is a picture of a gold pendant. The design of the pendant is very unique, shaped like a pair of spread wings with a heart-shaped hollow in the middle. The whole pendant presents a delicate and elegant feeling, suitable for use as a necklace or bracelet ornament.
Extra Description: A beatiful woman wearing [IMAGE1] walking on street
Feel free to dm me if you have any query.
Contact: [email protected]
QQ Group: 866612947
Description
FAQ
Comments (29)
Hi, in your 3.2 workflow, i've changed the basic redux style apply by the "reduxAdvanced" that allows you to target specific masked part of the input image, and i get way better results. More consistent with complexe patterns, and the best part to me is that you can really infuse the style of the input image with the downsampling factor and the weight of the redux.
So that if i input a jacket with complexe parttern, i specify in the prompt that i want a dress instead, i set the downsampling to 2 or 3 and the weight around 0.8-1 and i can get in the target image a dress infused with the style and pattern of the input jacket, which is not possible with the current apply redux style in your workflow.
But this last update is very good thank you
Feels free to modify the workflow for your purpose. I know the reduxAdvanced node but I think it is more simple for prompt or redux only. Using prompt could easily modify the content on the direct guidance. Using redux could be skip prompt input and let image do all the job. Might be the mask with reduxAdvanced could help more on complicated reference image which has many disturbance. I would considerate to add that in future version. I haven't read the code of reduxAdvanced and I don't really know how would it handle masked image. But if it is just zero out other area, my workflow implementation also could set mask on reference image and zero out other pixels outside the mask.
@xiaozhijason Yeah it's just that from my tests it's more versatile and easy to use with the reduxadvanced since you can combine prompt + redux so that if i want to open a jacket in target image i can, which is not possible or less easy with the basic redux node
@Elthibertt Yeah, it is not that easy for redux to do something like make the dress with xxx material but it could be done by using prompt only. https://civitai.com/images/43846990 This is using reduxAdvanced node which is very easy to use and https://civitai.com/images/43846989 This is using prompt only with "Using the material to replace the dress surface"
@xiaozhijason also about the buffer parameter,from what i understand, the more the inpainted target image zone is large, the more the buffer should be near 0 so that we get much more details ? and the smaller the inpainted part is, the larger the buffer is so it zooms in ? Or am i wrong ? Like when to change the buffer size and how much please. Also maybe i'm wrong but the smaller the buffer size, the more the detail but the less context it gets about the input image ? So the best is to find the right balance between details and context ? Sorry for all those questions lol
@Elthibertt You are right. Buffer is a main parameter of context size. When you need more context reference from edit image, you need more buffer. When you doesn't need that more you could set it lower. Usually, I would use small buffer with prompt guidance, for example, fixing text or label. But it is not a general anwser. Larger buffer usually have a better fused result.
@xiaozhijason ok it's clear now thanks a lot !
I don' know if it's possible, maybe a too complexe workflow, but it would be perfect if the target output could be upscaled so that the first target output (the one that is zoomed in the only masked area if the buffer is low, so the full resolution) could added to the final target image at full resolution. Idk if i'm clear.
Because for specific garnments, you want a lot of specific details, but when the first high resolution target image is then merged back into the final target image, it loses details since it's zoomed out.
Here is not a really good example since the pattern is simple but you can see that the low buffer zoomed in output is highest resolution but then it's merged in the final image and we lose the details :https://civitai.com/posts/9984693
So if we first upscale the final target image size, then merge back the first target inpaint outpout to it, we get the full resolution with no details lost. Cause for specific patterns if we keep things the way it is, sure we can upscale it, but we will lose coherency in patterns.
just an idea, i don't even know if it's possible to do
@Elthibertt You could just input an upscaled target image to achieve this easily
Good day, can you tell me what the problem might be? All three of my CreateContextWindow (IMAGE PROCESSING) windows are highlighted in red.
You need to install the plugin.
@xiaozhijason I have already installed it, it's just not completely red, just a thin frame around these windows. It's a pity that there is no way to attach a screenshot. But I hope you understand what I mean.
In fact, as I understand it, a mask is not created, although I create it in ComfyUI itself.
@Jio_R You need a mask of target image to start the process. You might reference this article to get some hints. https://civitai.com/articles/9320/fix-hand-using-context-window-editing-workflow-fluxredux
@xiaozhijason Thank you, I'll try to figure it out!
I get this error when generating:
CLIPVisionLoader
Error(s) in loading state_dict for CLIPVisionModelProjection: size mismatch for vision_model.embeddings.patch_embedding.weight: copying a param with shape torch.Size([1152, 3, 14, 14]) from checkpoint, the shape in current model is torch.Size([1024, 3, 14, 14]). size mismatch for vision_model.embeddings.position_embedding.weight: copying a param with shape torch.Size([729, 1152]) from checkpoint, the shape in current model is torch.Size([257, 1024]). size mismatch for vision_model.encoder.layers.0.layer_norm1.weight: copying a param with shape torch.Size([1152]) from checkpoint, the shape in current model is torch.Size([1024]). size mismatch for vision_model.encoder.layers.0.layer_norm1.bias: copying a param with shape torch.Size([1152]) from checkpoint, the shape in current model is torch.Size([1024]). size mismatch for vision_model.encoder.layers.0.self_attn.q_proj.weight: copying a param with shape torch.Size([1152, 1152]) from checkpoint, the shape in current model is torch.Size([1024, 1024]).
update comfyui
Hi! Thank you very much, this solved the issue.
I have the workflow running, but I'm having an issue with the output. I'm sure it some likely some setting but I cannot figure it out. For REFER_IMAGE I am using a picture of a T-Shirt. For Target_Image I am using a picture of a young lady wearing a different shirt.
I want the output to be the lady in TARGET_IMAGE wearing the T-Shirt in REFER_IMAGE. However, what I get as output is bottom 1/2 the t-shirt from REFER_IMAGE that appears to have been modified slightly then an empty space and then the bottom 1/2 of the young lady.
What do I need to do to get the the person wearing the T-shirt correctly?
Could you post your reference image, target image and bad generated result in the gallery for me to check?
@xiaozhijason Ok - posted and invited you as a collaborator. Image should be here https://civitai.com/posts/10274648
@xiaozhijason Ok - it is posted and I invited you as collaborator. Image should be here -> https://civitai.com/posts/10274648
@piehound0101723 Did you use Flux Fill? What is you masked area? Please post the reference image and target image separately for me to test.
is it possible with your workflow to create an image from a prompt without a mask on another image and incorporate an element from an image? the same system but without a mask.
How do I do this?
Thank you!
You could modify the workflow to work with that. It might better to use dev rather than fill.
I don't have time to create the workflow for generation recently.
@xiaozhijason I think this could allow designers to test more possibilities, put products in different environments, etc. I've tried a few things but it's not working... I can't wait to see a workflow like this come along!
@Hippo2024 Could you post your product and target enviroment for me to test? It should be work at current workflow or at least very minimum modification.
@xiaozhijason the idea is to use any type of product ( or 3D rendering ) for example : a coffee machine on a white studio background
and mask this product then generate an image from a prompt that incorporates this identical product
without masking a second image
( this could be in a typical kitchen for the coffee machine )
@Hippo2024 I know someone created a workflow for this. input product image and background image and generate a product with that background using out painting of the product. You could try to modify the workflow on this direction.