The new SDXL instruct Pix2pix model has arrived
Description
Better Clip with DPO
FAQ
Comments (10)
I cant seem to figure out how to use this, I just keep getting errors. is this tied in with automatic1111 only? im using easy diffusion
on automatic1111 it doesn't work. you need to have support for CosXL and you need to have an ip2p workflow, I use comfyui. you can use my workflow by copying one of the png images that contains the workflow itself
@Ciro_Negrogni @Ciro_Negrogni have you tried merging with hyper to make it 1 step?
@MrNoir not yet, but I'll try
In A1111, this model causes problems, At least on Windows with NVIDIA. I spent several hours working with PerplexityAI to troubleshoot the following error:
```
RuntimeError: Error(s) in loading state_dict for DiffusionEngine: size mismatch for model.diffusion_model.input_blocks.0.0.weight: copying a param with shape torch.Size([320, 8, 3, 3]) from checkpoint, the shape in current model is torch.Size([320, 4, 3, 3]).
```
In the end I had to stash my changes and revert. Clever bit of coding that AI came up with, I was convinced it was going to work, but changing the system architecture to [320,8,3,3] caused memory issues.
This is a pix2pix model. It is not designed for normal image generation. It changes the image.
yeah, how are you guys running this? will it work with 16gb nvidia on stable diffusion? I load it as the main model at the top and then instruct it in the positive box, right? i get errors. it won't work. :(
it's a pix2pix model. on automatic1111 it should work using this extension https://github.com/Klace/stable-diffusion-webui-instruct-pix2pix . i suggest you to try ComfyUI too
@Ciro_Negrogni i installed that extension, but i can't get it to even change the sky cloudy. you use img to img, but i don't know what all the settings should be to get it to work correctly. with the old sd instruct, you could just say "turn the sky cloudy" and it would do it. now that photoshop has stolen the tech, it's impossible for me to figure out how to get it to work..
@iamarandomstudent839 I'm very sorry to hear this. It's actually very fast compared to today's edit models. Don't you have a way to use comfyui?
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.