Hey everyone,
A while back, I posted about Chroma, my work-in-progress, open-source foundational model. I got a ton of great feedback, and I'm excited to announce that the base model training is finally complete, and the whole family of models is now ready for you to use!
A quick refresher on the promise here: these are true base models.
I haven't done any aesthetic tuning or used post-training stuff like DPO. They are raw, powerful, and designed to be the perfect, neutral starting point for you to fine-tune. We did the heavy lifting so you don't have to.
And by heavy lifting, I mean about 105,000 H100 hours of compute. All that GPU time went into packing these models with a massive data distribution, which should make fine-tuning on top of them a breeze.
As promised, everything is fully Apache 2.0 licensed—no gatekeeping.
TL;DR:
Release branch:
Chroma1-Base: This is the core 512x512 model. It's a solid, all-around foundation for pretty much any creative project. You might want to use this one if you’re planning to fine-tune it for longer and then only train high res at the end of the epochs to make it converge faster.
Chroma1-HD: This is the high-res fine-tune of the Chroma1-Base at a 1024x1024 resolution. If you're looking to do a quick fine-tune or LoRA for high-res, this is your starting point.
Research Branch:
Chroma1-Flash: A fine-tuned version of the Chroma1-Base I made to find the best way to make these flow matching models faster. This is technically an experimental result to figure out how to train a fast model without utilizing any GAN-based training. The delta weights can be applied to any Chroma version to make it faster (just make sure to adjust the strength).
Chroma1-Radiance [WIP]: A radical tuned version of the Chroma1-Base where the model is now a pixel space model which technically should not suffer from the VAE compression artifacts.
Quantization options
Alternative option: FP8 Scaled Quant (Format used by ComfyUI with possible inference speed increase)
Alternative option: GGUF Quantized (You will need to install ComfyUI-GGUF custom node)
Special Thanks
A massive thank you to the supporters who make this project possible.
Anonymous donor whose incredible generosity funded the pretraining run and data collections. Your support has been transformative for open-source AI.
Fictional.ai for their fantastic support and for helping push the boundaries of open-source AI.
Support this project!
https://ko-fi.com/lodestonerock/
BTC address: bc1qahn97gm03csxeqs7f4avdwecahdj4mcp9dytnj
ETH address: 0x679C0C419E949d8f3515a255cE675A1c4D92A3d7
my discord: discord.gg/SQVcWVbqKx
Description
FAQ
Comments (48)
How much of a improvement is there going from v11 to v20?
Still testing. v20 only checkpointed earlier today.
So far I'm noticing penises are somewhat better, though still have quite a ways to go.
Keep up the good work.
Cross my fingers this model will be a truly versatile one that can also do cinematic storytelling concept work (including some spicy stuff when called for)
The most lackness of FLUX ckpts are the generation of flexible animation, the real animation things.
Hope someone can solve this, to enhence the ability of anime generation.
And for real or photorealistic contents, we would like to know more details about the out-standing of this one.
Just tried v20, definitely more prompt following than base flux dev which is impressive. That said, it's obvious it still needs more training because it can be amazing with some stuff and then very weak with other stuff. Thanks for working on this!
I cant get your workflow to load in the newest version of comfy ui. Nothing loads at all.
what about forge, is that working with Forge atm?
I've tried, but no
https://github.com/croquelois/forgeChroma - I've tested and it works.
Does this need a special sampler of custom node? I just get black and white noise pixels when trying to generate with it???
Oh I found the workflow on the github that seems to work most of the time.
Can I train my own loras on this model? what trainers you already tried?
ai-toolkit will have support soon (there are a couple of bugs currently that should be fixed within the next week):
https://github.com/ostris/ai-toolkit
https://github.com/ostris/ai-toolkit/commit/7c21eac1b32e15d9883663824131fa0d9940be7d
diffusion-pipe has support:
does it works with booru style tags or only with descriptive prompts?
It works with booru style tags too - see my picture here: https://civitai.com/posts/15423995
For those who has problems with me missing nodes in ComfyUI, in the repo are the instructions for download them:
https://huggingface.co/lodestones/Chroma#manual-installation-chroma
Did all of that. Still doesn't work. Same missing nodes.
I told this in a review, but I think I have to tell this in a comment too.
This model has a crass mistake in its creation: When this model is uploaded to an online image generation service, is accepted but only black rectangles/squares is obtained from.
Pony Diffusion V6 XL wouldn't be half the success of SDXL-based models if Pony had been uploaded here with such a failure.
See you later in, in the best of cases, few months later when that detail is fixed.
It's an experimental model with architectural changes so would require the online image generation service to make changes to their inference which they're not going to do for one model that doesn't have much popularity yet.
The fact that a model of this quality is completely open source with training code and weights is a miracle, please don't put people down when they're sharing their hard earned work for free.
This is a great model, I love it - the prompt adherence is great and seems to work with descriptive and booru style prompts - nice job!
Great model in my limited test. Probably what pony could've been if it was based on Flux. Sadly I'm one of those 8GB user. Offloading to RAM helps but 30GB/31GB is way too borderline. Any plan to upload a lower quant version? I would like to experiment with it for some bias.
This is cool. The prompt adherence is really good! Looking forward to it.
Can you run Nunchaku Flux Dev/Schnell using the comfyui node on your GPU? https://github.com/mit-han-lab/ComfyUI-nunchaku Make sure you try the latest version in case you tried an earlier one, since there were some recent updates on hardware compat: https://github.com/mit-han-lab/nunchaku/releases/tag/v0.2.0
I think we may get Nunchaku 4-bit quants within the next few weeks. Nunchaku quant may have CFG baked in to get another 2x speed boost at the cost of CFG control. So it'd be about 6x faster (3x from Nunchaku, and 2x from CFG bake). Depending on your GPU you may only get 2x from Nunchaku, so worst case 4x faster in total.
Then hopefully another 4x to 8x speedup (assuming you're currently using ~45 steps) on top of that from perflow/rectification that you can see is currently being tested in the wandb training logs.
So Chroma should be at least 16x faster within a month or two I think.
i've uploaded some, check my profile or download from silveroxides huggingface.co/silveroxides/Chroma-GGUF
@levzzz gguf is slow. SVDQuant is the wat to go. Minimal quality loss and the speed is comparable if not better than SDXL. On 8gb GPUs.
@somedoby yes, i know, i've been trying to get deepcompressor (the svdquant library) to work so i could quantize but no luck so far on windows
@levzzz if I remember correctly it is resource heavy too, like 5 h100 hours for quantization. Not sure about the numbers. If so it may be inot feasible for every beta release.
@somedoby welp, sad, won't be able to get it running then
Yeah, even for people having 16Gb this checkpoint won't fit (16.58Gb! ). We all will need at least 4090 lol! X))
@Casey_JonesX http://huggingface.co/silveroxides/Chroma-GGUF is maintaining one for every release. That's what I'm using. It's kinda fun to see how better it gets for every release.
I'll be using this model for a bit. The results are exactly the vibe I'm looking for, naughty but fancy! It works so well, and it's not even finished yet! Excellent!
Flux based but with booru style tag....Sure it has potential!
man.. no more booru tags.. under natural language prompting all such models output mess. makes all of the illustrious clones completely unusable for me.
Dare I say this is the best model we have to work with that doesn't require complicated workflows, upscaling, messy prompts, or overbearing censorship!
Multiple updated versions available on his huggingface repo, I highly suggest checking them out. My only issue is figuring out how to use/train Loras with this model, but that's the case with most models right now anyways when it comes to the 50 series sadly.. Regardless, big kudos to this guy! AMAZING WORK!
Try using aesthetic 11. I think other aesthetic # tags exist, but I don't know what they do. I think 11 is meant to be have a pleasant, artsy style.
Workflow is missing nodes that don't exist.
ChromaDiffusionLoader and ChromaPaddingRemoval
@2P2 Thanks, this installs the ChromaDiffusionLoader but not the ChromaPaddingRemoval, I would really appreciate if you know how to install those. I couldn't find the process online.
@2P2 I have. I still get the same error.
Wonderful model with very strong potential. Very excited to see what it'll offer once fully trained. Thank you for making this.
Is it a de-destilled model?
This might be a dumb question, but do flux loras work with Chroma?
Never tried, might try next week, I will post updates if you are still interested. Maybe not, it doesn't feels mixable. Maybe
It works. Not perfectly, but it does work nice.
I use this https://civitai.com/models/686704?modelVersionId=768547 at very low strength to generate in 35 steps instead of 45