UPDATE:
1-added LoRA of SD XL DPO, NOT TURBO(usual settings for SD XL with this LoRA)
2-added LoRA of SD 1.5 DPO(usual settings for SD 1.5 with this LoRA, Clip Skip 1 for Real, Clip Skip 2 for Anime)
Turbo version can be used ONLY with NOT-turbo checkpoints, while the not-turbo version can be used with both turbo and not-turbo checkpoints.
LoRA based on sdxl turbo DPO, you can use the TURBO DPO with any stable diffusion xl checkpoint, few seconds = 1 image(3 seconds with a nvidia rtx 3060 12 GB with 1024x768 resolution)
SDXL DPO Turbo Checkpoint: https://huggingface.co/thibaud/sdxl_dpo_turbo/tree/main
Turbo LoRA: https://civarchive.com/models/215485?modelVersionId=242807
Turbo Checkpoint: https://huggingface.co/stabilityai/sdxl-turbo
Question: What is DPO SDXL?
Answer: DPO SDXL from hugginface, https://huggingface.co/mhdang/dpo-sdxl-text2image-v1/tree/main
DPO SD 1.5 from hugginface, https://huggingface.co/mhdang/dpo-sd1.5-text2image-v1/tree/main
For ComfyUI: use this LoRA with my workflow for sd xl turbo(change the lora_name): workflow --download here--
For webui 1111: write in the prompt: <lora:sd_xl_dpo_turbo_lora_v1-128dim:1>
1-Sampling method on webui 1111: LCM(install animatediff extension if you don't see it in sampling list)
2-CFG Scale: from 1 to 2,5
3-Sampling steps: from 4 to 8
---
If you want to support me, check out my Patreon: https://www.patreon.com/Shiroppo
Follow me on civitai(!!!): https://civarchive.com/user/shiroppo
Follow Me on Twitter/X: https://x.com/ShiroppoTwit
Deviantart: https://www.deviantart.com/shiroppoart
Pixiv: https://www.pixiv.net/users/96490888
---
Description
v1.0 DPO for SD 1.5
FAQ
Comments (10)
This is a truly great Lora, makes prompts super responsive! Improves hands / poses / proportions / background details as well. Huge fan of this.
I'm confused. Do I have to use the Turbo specific lora with turbo checkpoints? And do I HAVE to use the Non-turbo specific lora with non-turbo checkpoints? I thought you wrote that the Turbo lora could be used with ANY XL checkpoint? And if that's the case, why do we need the non-turbo lora?
1-Non-turbo has better image quality, turbo version has lower quality but as the name says the images are created in few seconds.
2-Turbo version can be used ONLY with NON-turbo checkpoints, while the non-turbo version can be used with both turbo and non-turbo checkpoints.
3-from https://huggingface.co/mhdang/dpo-sdxl-text2image-v1: Direct Preference Optimization (DPO) for text-to-image diffusion models is a method to align diffusion models to text human preferences by directly optimizing on human comparison data.
So the non-turbo LoRA improves the quality of each checkpoint, or at any rate can be a good experiment to create better images.
@shiroppo Thank you and Merry Christmas!
Helps with prompt/image consistency for sure! Thanks
what does DPO actually do? Is that makes model to understand prompts better?
Exactly, for more information: https://huggingface.co/mhdang/dpo-sdxl-text2image-v1
silly question, but if the model is not DPO as for example animagine it works, it will improve the hands and will be more faithful to the indications?
Yes.
From: https://arxiv.org/pdf/2311.12908
"Among other advantages, DPO-SDXL shows superior generation of anatomical features such as teeth, hands, and eyes"
Hello, is the production team currently interested in creating a dpo_turbo LoRA for Qwen Image? It works very effectively under SDXL