Come check out my new Discord server and help me grow! https://discord.gg/qtMCySnn See more content not shared on CivitAI. Feel free to share your own images as well.
Snoodler's SD 1.5 to SDXL 1.0 LoRa Workflow
Update 02/14/2024: I have release V2 of this workflow. It now comes with less spaghetti. Added a few custom nodes to help ease of use and keep things tidy. Added in FaceDetailer and UltimateSDUpscale. Also added in nodes for ControlNet implementation for trying to re-create older images.
About
This is a simple workflow that helps you use SD 1.5 LoRa's with the new SDXL 1.0 models. Render your old SD LoRa's in 1024x1024 the easy way!
Are you frustrated that you can't use your old SD 1.5 LoRa with the new SDXL 1.0 models? Look no further! This workflow helps you render an image using the new SDXL 1.0 models in a 1024x1024 size or larger, then use that rendered image as a baseline for your SD 1.5 models and LoRa to change. This not only enhances your pictures by using the new SDXL 1.0 models for better detail and generation, but now your old LoRa will be rendered higher than the old 512x512 size.
Now you can use all those old futanari LoRa's you have collecting dust with SDXL 1024x1024 renders! 🤣
Usage Tips
This workflow uses Efficiency Nodes and some Impact Pack nodes. I have included note nodes in each group within the workflow to help. I want to warn you that if you get strange images or they do not seem like they are rendering completely, it is most likely due to a VAE problem. I explain this inside the note nodes and it can be easily fixed by simply reloading the workflow.
Depending on the LoRa you are trying to use, you will have to think about how your base image should be generated and prompted. Most style LoRa do all the work by themselves. If you have a very niche LoRa for example a coffee latte style LoRa, it might take some creative prompting to get the base image that will let your SD 1.5 models and LoRa complete the correct work.
Thank You!
Thanks for checking out my workflow! Please leave a like and rating if you try it and like what you see. I'm fairly new to ComfyUI and this whole AI generation, but if you have any feedback or questions feel free to post them and I'll help in any way I can.
Description
Updated nodes with some new custom node packs, removed spaghetti, added FaceDetailer, and UltimateUpscale. Tweaked the layout so all output images are aligned for easier viewing and comparing.
FAQ
Comments (8)
Can you please share an image of your workflow in Comfy? sometimes not all nodes import with the links intact, so it's good to have a reference image! Thanks for this workflow!
Sure thing! Thank you!
This would work better with SD1.5 first, then resampled through SDXL. At least in my use case.
SDXL tends to produce better hands, feet, text, etc. but it either follows my prompts extremely well or not at all. Poses/acts/scenes from SD1.5 loras could really help, if they were rendered before SDXL.
The end result wouldn't be following SD 1.5 LoRA and models very well if you rendered over the results with SDXL models. I agree the detail and face, hands, textures ect. are better in SDXL, but the main purpose of this workflow is the end result of using SD 1.5 models guided by SDXL generation and resolution.
It's giving me a huge error when I add an SDXL refiner. I have to choose None, but the results are not astounding as expected.
Q1: are there ways to speedup SDXL or Pony with Turbo or Lightning? What about 1.5 with LCM? Would there be specific tradeoffs in model stacking like this? https://civitai.com/models/196630/comfyui-any-model-and-any-lora-4-step-highspeed-image-generation-based-on-lcm-sd-and-sdxl-compatible
Q2: why the face detailer is added there and not say other body parts or clothing? Are there something special about stability of face generation?
Q3: would upscaler be considered a form of acceleration? See https://civitai.com/models/159280/sd-and-sdxl-lowvram-medvram-comfyui-workflow-with-lora-upscale-and-second-pass
1. I honestly have no idea what LCM is. I would have to do some research to give you an in-depth response. I haven't tried using SDXL Turbo at all, but from what I hear it is faster.
2. I added facedetailer in because SDXL has always struggled with faces. You can add in other things like handfixers if you desire, but I have never had them reliably work unless the hands are completely isolated and in a basic position. I would just add them in right after the facedetailer node.
3. Upscaling is a post-generation tool. It will ALWAYS add more steps and thus extending the generation time. You can bypass the ultimate sd upscaler in the workflow to save some time if you desire, but the end result will be just a 1024x1024 image.
@Snoodler The question regarding upscaling is about whether to make a smaller image than upscale, or a larger image directly. Both increase computation needed, but is more a question of which is faster, and which is better.