Note: 2/27/2023 - I have removed the ckpt version of this upload out of an abundance of caution after Bitdefender software reported a Trojan. Spybot Search and Destroy, Malwarebytes, Avast (lol), and Windows Defender (lolol) didn't report any threats. The safetensor version is threat-free in all programs.
This model is specially fine-tuned to create good shadowy, dark, scenes. It's created by Nicholas Guttenberg at Crosslabs and the concept is explained here.
Nicholas explains: "not only does SD not have the ability to produce overly dark or light images out of the box, but it also can’t even learn to do it - not without changing one thing about it"
The article goes on to explain how to finetune your own models with their "darkness" technique, or you can merge the attached file into your own models with a SD 1.5 diff.
The attached images showcase the difference between their Noise Offset trained 1.5 model and the original 1.5 model, and the results are striking - true darkness and good contrast is possible with this model/method. I've also included examples of the model merged into my Churned Mix - producing true black/dark images.
This amazing LoRA has implemented the technique described in Crosslab's paper.
Please consider joining my Patreon! Advanced SD tutorials, settings explanations, adult-art, from a female content creator (me!) patreon.com/theally
Description
FAQ
Comments (64)
OMG!! This model looks awesome! I can't wait to make dark and light things with it! :D hehe. I love that you've posted this. It's wonderful for model mixes that want great contrast. I've used it with a couple of my mixes and found that the contrast change after multiple merges was not diminished. That's amazing for merges. Thanks again for sharing this!
The results really do look interesting, but would a Lora for this be possible in the future to enable such a thing across any model without the slow and messy process of manual merging?
That's the next step for sure. I Just wanted to release the model created by Crosslabs to get it out there into the hands of creators and get some visibility on this amazing tool!
Also, this fantastic LoRA has implemented the technique described in the paper.
One quick add difference to any of your favorite models, only takes a couple minutes!
PS, SD, can do dark images, and train on them. But it really does come down to prompt engineering. You can't just be like "dark image": https://prnt.sc/JbXuwh_Z28O6 (Image done on custom mix of protogen/1.5/paintinggenerator)
((close-up portrait shot)) of a (((sad solemn man))) holding roses, (((alone in the darkness, against (pitch black) background, unlit, hard shadows, deep contrast))), modelshoot style, (extremely detailed CG unity 8k wallpaper), most beautiful artwork in the world, trending on ArtStation, trending on CGSociety, Intricate, High Detail, Sharp focus, dramatic, photorealistic painting art by midjourney and greg rutkowski
You can skip using a model, and gen in the img to img tab starting from an (abstract) img that functions as a LUT. No need for a 3.5gb model. If you have photoshop you can make all sorts of background images to start from (neutral or not) and ad various visual effects to them.
Think you're missing the point. Did you read the paper by Crosslabs? Here's a LoRA generated using the technique mentioned in the paper, and demoed in the model I uploaded - https://civitai.com/models/8765/theovercomer8s-contrast-fix-sd15sd21-768
"If you have photoshop you can make all sorts of background images to start from (neutral or not) and ad various visual effects to them."
@cp27 - but isn't the point to not have to use photoshop? we want the images to be better right of the bat.
If you generate 1000 images over night, do you have time to correct all of them in photoshop?
You can use Photoshop of course, but there is the point of auto that while you're generating the image instead of looking for lens flares, stocks, etc. and put into photosop, resize in other modes. One is 50 times faster, other is fast, simple as that.
@theally was reffering strictly to img gen. Don't take it as a critique.
I feel like there are too many things (Lora, styles, embeddings) that need to be loaded for a quality image. Which in turn occupy more and more disk space and vram. Maybe a check in the ui to load such images as a filter would be more appropiate.
@danGorst Don't misunderstand me, right of the bat is what I want too.
@davizca I was thinking of a webui option to load a LUT filter image (that can be easily personalised). Not at post-processing.
@cp27 That could also be great a option too.
@cp27 have you tried ControlNet? You can do such amazing similar things with that! Very cool stuff.
@theally yes, thanks for your pruned models.
This really isn't a thing though to begin with. You just need to know how to prompt. I can get mainly black images out of 1.5 based models. It really does reiteration though, and weighting. The paper itself doesn't actually state the reasons why on a code level, and seems more like a uneducated guess of why, which appears falsifiable from my tests.
Also, if you have to use a limited model, to then get "dark images" it's of no real help, it needs to be something inject-able into any model.
@WAS I was able to get better contrast using this model than with base 1.5 and I've been prompting since the beginning of time (like 5 months ago.. wasn't that the beginning of time?).. :D I think the idea here is that you don't have to reprocess or do anything crazy to get what you need. This model can produce good contrast out of the box. For those that are model mixers, this model can be mixed and you DO get the high contrast effect out of it. It is a nice tool for the ol' toolbox.
@elldreth No, you're right, you're just limited to a models style, which may not be what you want (isn't for me, can't do all the styles I use saved). This needs to be a fix to the architecture to use with any model, or just prompt good.
And 5 months isn't too long. I've been doing diffusion for about 2 years, with GANs and Guided Diffusion before SD was even a thing.
look at my main comment, I shared a nearly all black image right from Protogen, and prompt used (though not negatives but that was mainly just about not being anime/cartoon etc).
A good way to find out how to caption dark, is look at the training data and nearly all black or dark contrast images and their captions. The issue with SD is it's dataset is aesthetics and high score, which tends to avoid dark images, as they will be interpreted with a low score, but many do exist such as portraits of girls and stuff.
For example: https://imagecache.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/ea5faa04-9cc0-4a70-fea3-6222de58bc00/width=2560
is nearly all dark, and in a sepia tone, and would likely get a low score as it's generally not what is appealing in art.
@WAS But I dont know what you're trying to say.. that we can get darker images or almost black images without init images in any custom model or just SD 1.5? Of course we can but you need to play with weights, prompts and negatives for minutes... So this is way more a correct approach.
@davizca It's inherently not a correct approach because it is a fine-tune skewed into a certain theme/direction/style. That's the issue. It's not a general model. Also these noise offsets introduce half images, people barely peeking off the side of the image, and weird random nonsense having nothing to do with your prompt. That's just way too much error over just learning how to prompt... Even with @theally latest release (TheAlly's Mix III: Revolutions), which I like, it has a ton of images cut off either vertically, or horizontally, and randomly does abstract distance shots with swirly magical stuff (that has absolutely nothing to do with the prompt at all, just random nonsense) that you have to negative "tendrils, energy" and some times other stuff, so you're still needing to do some prompt engineering anyway.
@WAS noise offset is implemented in Kohya now anyway, so I'm sure we'll see many more LoRA and fine-tunes with the better dynamic range trained in properly, it's just a setting you throw a 0.1 in, and you get the effect of this model added into your training.
@theally what about 0.0X or 0.1X or 0.25? Will affect "better"?
@theally It is just a noise offset isn't it? Couldn't this just be handled on WebUi/Diffusers/whatever side, and be implemented for whatever model loaded?
I spent a full day trying to prompt "an airport terminal at night" only to finally conclude that SD can't create proper night images. No combination of night, midnight, blackness, darkness, ink, pitch, pitch black, moonlight, etc. ever created anything that looked like the inside of an airport terminal at night. I really appreciate you making this model and LoRA available so we can finally get good high contrast images!
Upon downloading I received a warning from Bitdefender Total Security that the file attempted to install Trojan.Ciusky.CryptedAit.1 . I then scanned the file and again received a warning that it contained Trojan.Ciusky.CryptedAit.1. I do not tend to get false positives from BDTS, so I am sharing the info.
FYI: the more common name for this trojan is Win32/Nanocore.BA!MTB
Hi! I missed this comment, my apologies. I've scanned the file with a couple of anti virus/malware tools and am not seeing the same. Very odd! I have uploaded a safetensor file which you might be more comfortable with?
I'll reach out to Dr. Guttenberg (the source of the model) at Crosslabs and see what he thinks!
@theally Hi, I answered in the other thread.
For visibility - I have removed the ckpt version of this model out of an abundance of caution. The only app which picked up this threat was Bitdefender (Spyboy, Malwarebytes, Avast, and Windows Defender all gave it a pass), but it's better to be safe than pickled. I've scanned the safetensor version and it's clean.
Can you please provide a safetensor?
Done! I wasn't sure if a Safetensor would produce the same results, so I had to do a little testing before uploading, hence the delay. The good news; it works! Thanks.
@theally cool! Thank you! Do you have any recommendations on merging with models? You stated to add it with model A: model to add it to, Model B: this one, Model C: unpruned SD 1.5 with add difference. But what about the weight? Does more just mean darker/more contrast?
@unstablestDiffuser I'd need to test some more, but from the merges I made, the merge ratio definitely had an effect on how dark things got.
@theally I did just try with realisticVision, add difference at 0.5 and now at 0.25, the images certainly are darker and more contrasty, but it completely fucked up the looks of the model. People look really weird and ugly now.
@unstablestDiffuser Yeah, there's a balance. I used a reverse smoothstep mix, which I tweaked a little, to get the right effect in my latest model.
@theally I just noticed I turned on upscast cross attention layer to float32 for a different model yesterday and that fucked with it, now it's fine. The images are still the same, just darker/more contrast at 0.5 so it seems to work.
Are we not allowed to merge this with our own "merged model"? I believe you have settings set so we are not allowed to do so?
what do you suggest the percentage should be when we add difference with this model ? does it even matter what percentage you put with add difference like in weighted sum ?
Yah I added this to mine and it's not giving great results. maybe it's the way it's made? but it's not giving me an overall positive effect generally. If you have a suggestion on how to add this or what to do to merge this properly I would be grateful <3
@DarkAgent a couple of people posted LORAs with the same training method, i downloaded them but didn't try them yet, might be the way to go, also i extracted a LORA from this but again i didn't experiment with it.
@DarkAgent ok so i did some basic experimentation of all 3 LORAs, the best results are probably from the extraction of this model (320 dim), epi_noiseoffset is not bad either, Theovercomer8's Contrast Fix is decent but waay too strong, obviously it can be adjusted with weighting, so can come in handy if you want very dark images.
merge the attached file into your own models with a SD 1.5 diff? Do I use sd1.5 prund-EMA or non EMA model?
EMA-only
Can you add, if possible, a smaller 2gb pruned fp16 no-vae safetensor version ? Even better, a smaller LORA.
So how do I merge this with another model? Is it analogous to creating an Inpainting model?
I think you need to explains more about how to merge this model into my models
please could you explain how to merge ? (beginner pov)
I have a really in-depth merging tutorial on my Patreon - but I'm sure you can find this info on Google, search for Stable Diffusion Merge Block Weighted.
@hansolocambo You spent some time writing a hate comment when you could just post a link to Google. If it takes 5sec, what's a big deal? =))
@Kinky_Trixter the checkpoint merger tab in A111 works. The SuperMerger extension is a more advanced version, which can merge a bunch of checkpoints into one, bake in VAEs and Loras, and make intermediate merges (which don't get saved) for XYZ plots
Could you or someone convert this into a LORA, so that we can use it with any models and control its weight? Thank you so much!
Good news! https://civitai.com/models/13941/epinoiseoffset :)
I'd love to know how much to noise offset ? im using kohya colab and there is an offset slot and im not sure what number i need to set the offset.
thanks !
Digital nor analog photos are "true" darkness. but photographers do their shots somehow. Magic?
Learn how to use AP, AL - and you will be able to make "true darkness" on ANY image.
AP, AL ? would you explain please, I found no meaningful results searching for these abbreviations
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.