This model is an Early Beta version of the HelloWorld SC. Based on the training scripts provided by Stability AI official, I fine-tuned the Stable Cascade stage_c_lite model (1B version).
Please note that the current v0.1 version is the earliest test model. The main purpose is to familiarize myself with the new training process. After image generation tests, the overall performance of this model is not stable, with better effects in close-up scenes, but noticeable degradation in image quality in full-scene compositions, such as full-body shots of characters.
Here are some core data from my training that may be helpful for other model authors interested in Stable Cascade:
The v0.1 version used a total of 740 realistic training images, covering themes like portraits, science fiction, and pallas's cat. All images were tagged using our open-source GPT4V tagger. The graphics card used was one 48G VRAM RTX6000ada. The total training time was 3.5 hours.
The config file parameters are as follows:
lr: 1.5e-6
batch_size: 6
image_size: 1024
multi_aspect_ratio: [1/1, 1/2, 1/3, 2/3, 3/4, 1/5, 2/5, 3/5, 4/5, 1/6, 5/6, 9/16, 9/21]
grad_accum_steps: 1
updates: 12500
backup_every: 2500
save_every: 500
warmup_updates: 1
use_fsdp: false
adaptive_loss_weight: TrueThe above parameters would occupy about 45G of VRAM during training. The official training scripts seem to be aimed at high-VRAM cards like the A100, without much VRAM optimization, so I suggest that model authors with 24G or less VRAM wait for kohya-ss's updates. Currently, even if the batch size is set to 1, fine-tuning on stage_c_lite.safetensors still requires 30G of VRAM using the official training scripts.
（Update: A netizen has optimized the official script for GPU memory usage. At present, for the stage C 1B model, fine-tuning requires only 10GB of GPU memory, which is sufficient.）
Future plans:
Once kohya-ss is updated, I plan to use the complete HelloWorld 6.0 training set to fine-tune the SDXL, Stable Cascade 3.6B, and 1B versions respectively.
I have high hopes for Stable Cascade and hope that this version can fix some of the shortcomings of SDXL in widespread use, attracting more players from SD1.5 into the new generation of SD model ecosystems.
Special thanks：
I would like to express my gratitude to Fok, the creator of the Ronghua model, for his immense help during the execution of the SC model training script. He is also optimizing and testing his own SC model; I'm looking forward to hearing good news.
本模型为HelloWorld Stable Cascade版本的早期测试模型。基于stability AI官方所提供训练脚本，我在stage_c_lite.safetensors模型（1b版本）基础上进行了微调训练。
请注意目前的v0.1版为最早期测试模型，主要目的是熟悉新版本整个训练过程。经过图像生成测试，本模型的整体发挥并不稳定，近景效果较好，全景构图如人物全身照之类，则会出现明显的画质劣化现象。
以下是我在本次训练中的一些核心数据，希望能对其他对 Stable Cascade感兴趣的模型作者有所参考：
v0.1版本整个训练共使用了740张写实训练图，涵盖人像、科幻、兔狲等主题。所有图片均基于我们的开源GPT4V tagger进行了打标，所使用显卡为单张显存48G的RTX6000ada。整个训练时间为3.5小时。
config文件中参数如下
lr: 1.5e-6
batch_size: 6
image_size: 1024
multi_aspect_ratio: [1/1, 1/2, 1/3, 2/3, 3/4, 1/5, 2/5, 3/5, 4/5, 1/6, 5/6, 9/16, 9/21]
grad_accum_steps: 1
updates: 12500
backup_every: 2500
save_every: 500
warmup_updates: 1
use_fsdp: false
adaptive_loss_weight: True以上参数训练时会占用约45g显存。官方训练脚本应是面向A100等大显存显卡，未做过多显存优化，建议24G及以下显存的模型作者，可以等待后续kohya训练器的支持性更新。目前官方训练脚本，即使batch size设为1，对stage_c_lite.safetensors进行微调也仍需要30G显存。
后续计划：
待kohya训练器更新后，我将使用HelloWorld 6.0的完整训练集，分别对SDXL、Stable Cascad 3.6B及1B版本进行微调训练。对于Stable Cascad我满怀期待，希望这个版本可以修复SDXL在普及使用过程中的一些不足，将更多玩家从SD1.5中吸引到新世代SD模型生态中来。
特别致谢：
感谢容华模型的作者Fok在SC模型训练脚本运行时对我的莫大帮助。他也正在优化测试自己的sc模型，期待有好消息
Description
FAQ
Comments (38)
I'll leave it here: I tested Stable Cascade only on Huggingface spaces, but it seem evident that there is a cheat for improving the outputs. At the end of any prompt if you write 'made by Dall-E 3' looks like it boosts quite a lot the quality (for your info works even on nudity)
Wow, fast! Appreciate the commentary also. Wanted to mention that "One Trainer" should have support for SC soon as well (re: mention to Kohya). Noticed a dev branch for it already on their repo and developer has worked w/ wurstchen and their devs before on training.....
Aw man, I hope with OneTrainer we can get training on lowend for the 1B because I really want to try that
I love OneTrainer so much. Plus those guys are down-to-earth and super helpful.
I'm also eager to try OneTrainer. This trainer has some unique features, and I'm excited to see what new functionalities will be added.
Wait a minute, I just realized... these sample images are made with the 1B model... they look awesome, SAI said that they worked so little with the 1B model... Oh My God
While The stability of the 1B model does seem to have some issues. It performs exceptionally well under some prompts but shows obvious artifacts with others. I've tried it out, and this problem exists with both this model and the official 1B model. I'm not sure if it's an issue with the image generation parameters in the notebook. I look forward to doing more comparative tests with the same seed after the webui natively supports it.
Hey, thanks a lot for the insight into the training data.
Thank you for your encouragement！
How can we use this model in automatic1111 with the Stable Cascade extension?
@MysticDaedra Can't get the model to be loaded, errors out :/
The current version of the webui plugin probably doesn't support custom model switching yet, and I'm currently generating images using the official notebook provided. However, I noticed that in the model's output section, a friend shared a few images that included prompt word parameters, and I'm very curious about what tools they used. https://civitai.com/images/6643717
@MysticDaedra Thanks, I'll check it out.
@LEOSAM Ooh interesting. I'm curious as well. Guess we'll wait for a response from him.
给猫狲大佬磕头
不敢不敢，给你回磕一个😂
附议
Nice work! This is very interesting
Does anynone know if this can run on Forge?
Would like to see A/B with the Cascade base model
Supported resolutions?
Great
那么快就有人微调了
i appreciate your work, can you tell me how we can ready a dataset for this training?
I share your interest and hope you are right, if this model is easier to train it will be a great step forward. Good luck to you
Oh i didn't knew it was this easy, Download Stage A and B and Clip from Stable Cascade. and Stage C is the model in this page, correct ?
If it's correct i'm happy to report that it generates as fast as SD1.5 for me, of course with Stage B lite, have to try better Stage B models to see how it performs.
20 steps stage C + 10 steps stage B at 1024x1024 only took 4second combined and 6sec overall to finish. with a 3060ti 8GB and only used almost 2GB vram and 12GB system memory.
SDXL turbo takes that much with +5gb vram usage with lower quality results.
I must be doing something wrong. So, my normal updated workflow that uses the checkpoint nodes in Cascade works. However, I get an error when placing this model as for the stage C. Or perhaps this is based on the original Cascade configuration utilizing the Unet nodes?
@MysticMindAi I am not expert on this but i put this model into unet folder and in stable cascade folder next to the official B stage, something like this example:
J:\ComfyUI\ComfyUI_windows_portable\ComfyUI\models\unet\Stable-Cascade
@infernahermit846 Ahh, so the original workflow configuration. Interesting. I'll try that. Thanks!
@infernahermit846 That worked!!! :D
@MysticMindAi Nice, but it's not good for people's faces if you useing lite version of the B stage.
@infernahermit846 That's definitely something I noticed in my testing. Other similar models didn't perform very well either. IT's to be expected being it's still fairly new along with figuring out optimal steps, etc
@MysticMindAi Yeah but i guess nothing will come out of Cascade, lets wait and see what will SD3 brings
Could be good to share samples with prompts to showcase your model, examples of how to prompt for it.
Getting the following error:
Error occurred when executing unCLIPCheckpointLoader: 'model.diffusion_model.input_blocks.0.0.weight' File "E:\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute output_data, output_ui = get_output_data(obj, input_data_all) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list results.append(getattr(obj, func)(**slice_dict(input_data_all, i))) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI\ComfyUI\nodes.py", line 582, in load_checkpoint out = comfy.sd.load_checkpoint_guess_config(ckpt_path, output_vae=True, output_clip=True, output_clipvision=True, embedding_directory=folder_paths.get_folder_paths("embeddings")) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI\ComfyUI\comfy\sd.py", line 507, in load_checkpoint_guess_config model_config = model_detection.model_config_from_unet(sd, "model.diffusion_model.") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI\ComfyUI\comfy\model_detection.py", line 194, in model_config_from_unet unet_config = detect_unet_config(state_dict, unet_key_prefix) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "E:\ComfyUI\ComfyUI\comfy\model_detection.py", line 78, in detect_unet_config model_channels = state_dict['{}input_blocks.0.0.weight'.format(key_prefix)].shape[0] ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.