If this model seems like a clone of picXreal: blame it on convergent evolution rather than plagiarism -- one can only continuously say "no, THIS is the pinnacle of 1.5" so many times before the prophecy slowly becomes a reality. Indeed, the major difference between picXreal and SenBlend 1.9 is that the former manages to capture much deeper, richer shadows than the latter. And with that marks the final iteration of SenBlend: 1.10, because "how much better do you expect a 1.5 model to get". This adds all the new weights from picXreal on top of SenBlend with the intention of widening the variety and correcting any style problems that could be present on picXreal, a merge that is already incredibly hard to top.
With that said, I cannot prove that this model is any better, or any worse: it is an alternative in which, realistically, improvements present are so minute that they might as well be a matter of opinion. Through arduous testing, this model performed slightly better in some diffusions while picXreal performed better in others. It is for these reasons that I will no longer update this model: anything that does manage to objectively improve this model or picXreal will be so farther ahead that attempting to merge would no longer be a strategy. We have gotten that far with 1.5.
It also seems my initial prediction regarding SDXL is on the right track: we are roughly 6 months into SDXL, and we now have very promising leaps with Cascade and SD3. Another 6 months seems reasonable for a monster model that can make classic 1.5 diffusers turn their heads. We will wait and see... but for now, enjoy the latest and greatest 1.5 model.
Description
With SDXL right around the corner, it seems that the era of 1.5 models is about to come to an end. That doesn't mean that 1.5 models will instantly be left in the dust, in fact, it will probably take a considerable amount of time before we get the same level of customization and finetuning for XL as we currently have with 1.5 models, so 1.5 models will continue to dominate until further work is done to XL, and SenBlend 1.9 represents the pinnacle of the old era of models. This is hopefully the last and best 1.5 model for realism.
SB1.9 is made up of a carefully curated mix of ZovyaV2, Reliberate, Photon, and of course, SenBlend 1.8.
Tips for Dreamboothers:
This model works bonkers for Dreambooth, much more so than previous models. I am going to share my magic formula for creating perfect dreambooths every single time. (Keep in mind that current dreambooth versions are usually broken or complete crap. You have to use an older version!):
-A1111 (not Kohya!). Get yourself an older version. Specifically, I use this commit: ea9bd9fc7409109adcd61b897abc2c8881161256
-Use this dreambooth revision: 43ae9d55531004f1dedaea7ac2443e9b16739913
-Software versions (you can see this list when you boot up A1111):
[+] bitsandbytes version 0.35.0 installed.
[+] diffusers version 0.10.2 installed.
[+] transformers version 4.25.1 installed.
[+] xformers version 0.0.16rc425 installed.
[+] torch version 1.13.1+cu117 installed. This is the most important component, do not upgrade it!
[+] torchvision version 0.14.1+cu117 installed.
Dreambooth Training Parameters (leave anything not listed at default):
-Extract EMA Weights.
-Training Steps Per Image (Epochs) = 7,360/X where X == Total Number of Subject Images. If you can spare the training time, multiply this by 3 for extra leeway.
-Batch Size: 1-8 depending on your VRAM, it is suspected (though not confirmed) that lower batch size = better quality results. Experiment and report back with your findings.
-Gradient: 1.
-Set Gradients to None: Check.
-No Gradient Checkpointing.
-Gradient Acc Steps: 1
-Class Batch Size: 1
-Learning Rate Scheduler: constant
Learning Rate = 1.5e-06 to 1.65e-06 works best! Experiment around this range and try not to deviate too much from it.
Constant/Linear Starting Factor = 0.5
Scale Position = 0.25
-Use EMA: Yes
-8bit Adam
-Mixed Precision: bf16 (if you have puny VRAM, go for fp16)
-Memory Attention: flash_attention (using xformers WILL break your model, don't say I didn't warn you!)
Happy Dreamboothing and let me know in the comments if you have questions!
FAQ
Comments (6)
You suggest: "With SDXL right around the corner, it seems that the era of 1.5 models is about to come to an end."
Then follow up with: " it will probably take a considerable amount of time before we get the same level of customization and finetuning for XL as we currently have with 1.5 models.
So, which is it?
Also, SDXL is a move towards a sanitized anti CIS Pron, and a move towards pro-yiff or sexless styles.
SDXL is the new SD2.0.
If you take a moment to reread my post, you will get the answer to that question. It will also answer your concern regarding the contents of the base model: although a censored model is a big hinderance for artists, it isn't the only factor to consider regarding a model's viability. For instance, 1.5 upon release sucked for anatomy. It was because of arduous finetuning from the community that we ended up with the beautiful models we have now.
From the previews I've seen on Discord, XL is moving towards a MidJourney style, which isn't necessarily bad. The real concern is how well this model responds to training and prompting, among other things. Nobody is expecting the base model to be great right off the bat. Whether XL becomes the new de-facto model or just another useless 2.1 is just a matter of time. Either way, we have plenty of 1.5 goodness to last us a while.
Hey man -- thanks for that little dreambooth tutorial. I have been trying to track down the last stable version that correctly extracted EMA weights.
To make the repo you suggested function, I had to add "torchmetrics==0.11.4" to requirements.text in the main auto1111 folder
Update: it loaded, but when I download your model and check "extract EMA weights" it still does not exact EMA, and loads with "EMA false"
You're welcome, however, things have changed, and now it seems Kohya is the way to go for dreambooth models. Extracting EMA weights has never had that much of a significant impact anyways, so you aren't missing out on much. I don't know if this website allows dms, but if you want to, I can send you the .json file that I use for Kohya dreambooth trainings and a quick tut if you'd like. I am never going back to A1111 training myself.
Hey I now see that the full model with EMA weights is only available on the oldest version here. Two questions: How did you actually produce such a model after merging, I haven't been able to find a single other model in which the EMA weights can be extracted. And would you be willing to upload the full model for the 1.9 version?
The 1.9 model is the full model, it's not pruned or anything. Now that you mention it, all dreambooth iterations have always erroneously shown "EMA false" even when extracting EMA. However, as I mentioned in the other comment, extracting EMA is kind of outdated. The new kid on the block is Dadaptive training with Lion optimizer. The results that I have gotten from this training are beyond anything else I've trained, and it eliminates the need to set a learning rate for anything. DM me for more info.