Hyper-SD is one of the new State-of-the-Art diffusion model acceleration techniques. In this repository, we release the models distilled from SDXL Base 1.0 and Stable-Diffusion v1-5
Project Page: https://hyper-sd.github.io/
Original Repo: https://huggingface.co/ByteDance/Hyper-SD
Original Paper: https://arxiv.org/abs/2404.13686
Description
FAQ
Comments (16)
What is the differrence between 8 step and 8 step cfg?
In CFG-version you will be able to use CFG values 5-8 when generating (the authors recommend a value of 8), also this version gives the possibility to work more correctly with negative prompts.
@tigerart does this work with lcm sampler (with or without lora)? has anyone tried this with animatediff yet?
@Vivipeg I'm going to play around with combining these different hyper loras with my LCM checkpoints. I'll share my findings when I have something meaningful to report!
@lucidzachary473 thanks, my initial findings show that the 8-step CFG lora doesn't work well with animatediff v3 at 8 steps dpmpp 2m (maybe I'm doing something wrong) . Maybe you'll have better luck!
@Vivipeg I could not get Hyper lora and LCM checkpoints / LCM lora to play well together. However, I decided to do extensive XYZ plot testing for LCM and hyper lora for comparison. I tested different steps (8steps and 15steps) and every single sampler I have (39?). I did this for Photon-LCM checkpoint and the regular Photon checkpoint w/ Hyper 8step lora. I can upload the 4 XYZ plot images somewhere if you want to see what samplers work well. My findings: both hyper and LCM are fast, and comparable in speed. Honestly one didn't stand out from the other in speed. Quality, however, I will give to hyper. Hyper was consistently better imo. LCM was rather stripped of detail/image complexity in contrast to 8step hyper lora. Oddly enough, the LCM sampler performed rather poorly with Photon-LCM but was one of the better samplers when used with hyper lora on non-LCM! At least, for stylized/toon styles. I was not testing with photoreal.
@Vivipeg Not sure if any of that is pertinent to your use case lol I've not used SD for anything but image generation so far.
@Vivipeg could you please list top 3 checkpoints for that please? for the animatediff v3 at 8 steps dpmpp 2m
@tigerart Where do they say that they recommend 8?
Great work ! thanks for this !
Edit: 12 step CFG version
There is also a 12 step CFG version available. Would be great if that gets added here as well.
When using the Draw Things app on iPad (haven't yet tried it with Automatic1111, etc.), the regular version of the 8-step LoRA seems to do a (slightly) better job with faces than the CFG version—at least when used with some models, such as AI Beast Mix and Uber Realistic Porn Merge. Is this just a peculiarity of the DT app or those models, or is this generally true?
I downloaded both versions of the LoRAs through Safari. The Hyper and Lightning LoRAs, when downloaded via the Draw Things app itself, seem to be locked to a certain CFG level; i.e., changing the setting of the CFG slider has no effect on the output.
What is the scheduler and sampler?
I found the DPM 2M++ Turbo (from ReForge ) the best, but Eurler A , DPM 2sa and DDIM ,DDPM work too, Schedulers, mostly Simple,Normal,SGMUniform,Align your steps,Beta,DDIM,Cosine,
Works great, for the non-CFG version 8-step reduce model strength to 0.55 for most models. Raise the strength to .60 .65 etc. as needed for your model. Some require a bit higher. Try Euler simple, the usual stuff doesn't work and strength 1.0 is too high for all.
thank you so much , i was trying hard to find a non burned sampler-scheduler combo without good results. Now i could make it work 0.55-0.9 is the range (depending on steps). I now added lcm lora (it fixes some errors in the image and also makes better contrast) the perfect kombo ist HyperLora 0.65 + LCM Lora 0.55 look in the 1step gallery for my best image so far. BTW the reason i experiment is the goal is to run Stable diffusion on CPU (i use Re-Forge for that installed with 1 Click Tool "Stability Matrix") so far i got improved time coparted to LCM only. for 512x768 i could achied 21-24sec, for highresfix 688x1032 its 79sec (but great quality) for CPU thats a gamechanger. If i lower the Res to 614x920 i can achieve 60sec for hi-res Image.