<h1 id="kirazuri-(ernie)">Kirazuri (Ernie)</h1>Kirazuri (Ernie) is a full fine-tune of the ERNIE-Image model. <a target="_blank" rel="ugc" href="https://huggingface.co/baidu/ERNIE-Image">ERNIE-Image</a> is an open text-to-image generation model developed by the ERNIE-Image team at Baidu.Version 0.1 (Latest) is trained on ~50,000 images at multiple resolutions in three total stages up to a maximum resolution of 1024^2 pixels.This fine-tune focuses on several goals:- Learn new concepts/styles/characters associated with tag-based prompting.- Enhance the model aesthetic guided by manually applied quality, aesthetic, and style tagging- Preserve the text-rendering capabilities in multiple languages (English, Chinese, Japanese)For more details, see the <a target="_blank" rel="ugc" href="https://github.com/motimalu/diffusion-training-configs/blob/main/diffusion-pipe/ernie/notes/ernie-kirazuri.md">Kirazuri (Ernie) Training Diary</a><h2 id="training-details-summary">Training Details Summary</h2>Trainer: <a target="_blank" rel="ugc" href="https://github.com/tdrussell/diffusion-pipe">diffusion-pipe</a>Training device: NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation EditionTotal training time: ~500 hrs (~20 days)Total samples seen (un-batched steps): ~1,328,000Training resolutions: 512^2, 768^2, 1024^2<h3 id="additional-features">Additional Features</h3>- Tag Dropout: 10% with protected first 8 tags- Tag Shuffle: Applied to last unprotected tags- Natural Language: 6 total Short and Long Caption variants<h3 id="quantizations">Quantizations</h3>Int8 convrot quants of the model and text encoder are available and strongly recommended for use in ComfyUI.Generation speed *(on NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition):Before:26gb vram, 1.35it/sAfter: 18gb vram, 1.89it/s<h2 id="installing-and-running">Installing and running</h2>Workflow:<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/4a2f1f3f-a72c-4532-84f8-61345b0eeee9/original=true/4a2f1f3f-a72c-4532-84f8-61345b0eeee9.jpeg" /><span style="background-color:rgb(26, 27, 30);color:rgb(193, 194, 197);font-family:-apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Helvetica, Arial, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;;font-size:16px">The model is natively supported in ComfyUI. The above image contains a workflow; you can open it in ComfyUI or drag-and-drop to get the workflow.<h3 id="generation-settings">Generation Settings</h3>Recommendations:<ul><li>1024^2 Resolutions</li><li>50 Steps</li><li>CFG 4</li><li>er_sde or euler Sampler</li></ul>*Some previews are generated at 1280^2 resolutions - despite no training yet at this resolution, the model can still perform well for small details here.*The model can converge in as few as 20 steps, but anatomy and prompt adherence will suffer.<h3 id="prompting">Prompting</h3>This model is trained on combinations of booru-style tags and natural language captions.It does not perform well yet with tag-only prompts, and they are best used in combination with long natural language descriptions.This follows from the base models natural language only training, and intended use with a prompt enhancer that expands descriptions to long-format text.Tag orderTags combined with natural language can be placed at the start or end of the prompt.<code>[quality/meta/safety tags] [character] [series] [artist] [1girl/1boy/1other etc] [general tags]</code><code>[quality/meta/safety tags] [character] [series] [artist]</code> tag groups are also not shuffled, so their order may have some influence on generations.Quality and Aesthetic tagsHuman score based: <code>masterpiece, best quality, very aesthetic, aesthetic</code>Meta tagsabsurdres, official art, etcStylespainterly, chiaroscuro, ligne claire, flat color, no lineart, blending, etctraditional media, oil painting \(medium\), watercolor \(medium\), etc<h2 id="recognitions">Recognitions</h2><ul><li>Thanks to Baidu labs for the open research and release of the ERNIE-Image model.</li><li>Thanks to tdrussell of CircleStone Labs for the <a target="_blank" rel="ugc" href="https://github.com/tdrussell/diffusion-pipe">diffusion-pipe</a> trainer.</li><li>Thanks to narugo1992 and the deepghs team for open-sourcing various training sets, image processing tools, and models.</li><li>Thanks to silveroxides for the <a target="_blank" rel="ugc" href="https://github.com/silveroxides/convert_to_quant">convert_to_quant</a> quantization tool.</li></ul><h2 id="license">License</h2>Apache license 2.0Note that ERNIE-Image is governed by the same license; this fine-tune does not grant any rights or impose any restrictions to the base weights.

Kirazuri (Ernie)

kirazuriErnie_v01_int8.safetensors

Files with SHA256: c39d77344f22f2c89b6659951e6a3ee0c78dfe2f27786f6f63c74cc29468cd51