New version is out: https://civarchive.com/models/628865/sotediffusion-v2
Anime finetune of Würstchen V3.
This release is sponsored by fal.ai/grants
Trained on 6M images for 3 epochs using 8x A100 80G GPUs.
This model can be used via API with Fal.AI
For more details: https://fal.ai/models/fal-ai/stable-cascade/sote-diffusion
Please refer to Huggingface for SD.Next UI, Diffusers or UNet models:
https://huggingface.co/Disty0/sotediffusion-wuerstchen3
CivitAI page has only the ComfyUI checkpoint models.
Inference Parameters:
Download the Main model (8.14 GB file):
https://civarchive.com/api/download/models/563950?type=Model&format=SafeTensor&size=pruned&fp=fp16
Download the Decoder model (4.24 GB file):
https://civarchive.com/api/download/models/563892?type=Model&format=SafeTensor&size=pruned&fp=fp16
Positives:
newest, extremely aesthetic, best quality,Negatives:
very displeasing, worst quality, monochrome, realistic, oldest, loli,Main:
Sampler: DDPM or DPMPP 2M with SGM Uniform
CFG: 7
Steps: 30 or 40
Decoder:
Sampler: Euler a Karras
CFG: 1 or 1.2
Steps: 10
Compression: 42 (or 32 to 64)
Resolution: 1024x1536, 2048x1152.
Anything works as long as it's a multiply of 128.
Training:
Software used: Kohya SD-Scripts with Stable Cascade branch.
https://github.com/kohya-ss/sd-scripts/tree/stable-cascade
GPU used: 8x Nvidia A100 80GB
GPU hours: 220
Base
parameters | value
amp | bf16
weights | fp32
save weights | fp16
resolution | 1024x1024
effective batch size | 128
unet learning rate | 1e-5
te learning rate | 4e-6
optimizer | Adafactor
images | 6M
epochs | 3
Final
parameters | value
amp | bf16
weights | fp32
save weights | fp16
resolution | 1024x1024
effective batch size | 128
unet learning rate | 4e-6
te learning rate | none
optimizer | Adafactor
images | 120K
epochs | 16
Dataset:
GPU used for captioning: 1x Intel ARC A770 16GB
GPU hours: 350
Model used for captioning: SmilingWolf/wd-swinv2-tagger-v3
Model used for text: llava-hf/llava-1.5-7b-hf
Command:
python /mnt/DataSSD/AI/Apps/kohya_ss/sd-scripts/finetune/tag_images_by_wd14_tagger.py --model_dir "/mnt/DataSSD/AI/models/wd14_tagger_model" --repo_id "SmilingWolf/wd-swinv2-tagger-v3" --recursive --remove_underscore --use_rating_tags --character_tags_first --character_tag_expand --append_tags --onnx --caption_separator ", " --general_threshold 0.35 --character_threshold 0.50 --batch_size 4 --caption_extension ".txt" ./
dataset name | total images
newest : 1.85M
recent : 1.38M
mid : 993K
early : 566K
oldest : 160K
pixiv : 344K
visual novel cg : 231K
anime wallpaper : 105K
Total: 5.628.499 images
Note:
Smallest size is 1280x600 / 768.000 pixels
Deduped based on image similarity using czkawka-cli
Around 120K very high quality images got intentionally duplicated 5 times, making the total image count 6.2M
Tags:
Tag Format:
Model is trained with random tag order but this is the order in the dataset if you are interested:
aesthetic tags, quality tags, date tags, custom tags, rating tags, character, series, rest of the tagsDate:
newest : 2022 to 2024
recent : 2019 to 2021
mid : 2015 to 2018
early : 2011 to 2014
oldest : 2005 to 2010
Aesthetic Tags:
Model used: shadowlilac/aesthetic-shadow-2
score > 0.90 : extremely aesthetic
score > 0.80 : very aesthetic
score > 0.70 : aesthetic
score > 0.50 : slightly aesthetic
score > 0.40 : not displeasing
score > 0.30 : not aesthetic
score > 0.25 : slightly displeasing
score > 0.10 : displeasing
rest of them : very displeasing
Quality Tags:
Model used: https://huggingface.co/hakurei/waifu-diffusion-v1-4/blob/main/models/aes-B32-v0.pth
score > 0.980 : best quality
score > 0.900 : high quality
score > 0.750 : great quality
score > 0.500 : medium quality
score > 0.250 : normal quality
score > 0.125 : bad quality
score > 0.025 : low quality
rest of them : worst quality
Rating Tags:
general
sensitive
nsfw
explicit nsfw
Custom Tags:
image boards: date,
text: The text says "text",
characters: character, series
pixiv: art by Display_Name,
visual novel cg: Full_VN_Name (short_3_letter_name), visual novel cg,
anime wallpaper: date, anime wallpaper,
License
SoteDiffusion models falls under Fair AI Public License 1.0-SD license, which is compatible with Stable Diffusion models’ license. Key points:
1. Modification Sharing: If you modify SoteDiffusion models, you must share both your changes and the original license.
2. Source Code Accessibility: If your modified version is network-accessible, provide a way (like a download link) for others to get the source code. This applies to derived models too.
3. Distribution Terms: Any distribution must be under this license or another with similar rules.
4. Compliance: Non-compliance must be fixed within 30 days to avoid license termination, emphasizing transparency and adherence to open-source values.
Notes: Anything not covered by Fair AI license is inherited from Stability AI Non-Commercial license.
Description
Initial pre-alpha release
Can fall back to realistic.
Use "anime illustration" tag to point it into the right direction.
Far shot eyes are bad thanks to the heavy latent compression.
Nudity is there but not trained enough yet.