*Important, please read this before using the model because this is very experimental, most of the time the result are mediocreMiso Diffusion M (Alpha) is an attempt to fine tune stable diffusion 3.5 medium on anime dataset. In comfy ui it uses as little as 2.4 gb vram without the t5 text encoder. This version is fine tuned with 250k image with just 1epoch. The model with larger dataset is still in preparation but faces some challenge. During my initial attempt to fine tune with 700k image, the model would collapse after 2000 steps, it is unclear why as I find the first training run some what promising. Other issue includes the nature of SD3.5 which aim to provide diverse output meaning even if you use the same seed it will have very different outcome, so it remains unclear if it can accurately generate the characters.Recommanded setting, euler, cfg:4 , 24-28 steps, though dpm ++ 2m also works but haven't done much testing, prompt: danbooru style tagging. The model is also very picky with prompt, I recommand simply generating with a batchsize of 4 and pick the best one.You can download the text encoder or model at other stages here, -000001 represents the model with only 1 epoch. <a target="_blank" rel="ugc" href="https://huggingface.co/suzushi/miso-diffusion-m-test">https://huggingface.co/suzushi/miso-diffusion-m-test</a>Quality tag<pre><code>Masterpiece, Perfect Quality, High quality, Normal Quality, Low quality</code></pre>Aesthetic Tag<pre><code>Very Aesthetic, aesthetic</code></pre>Pleasent<pre><code>Very pleasent, pleasent, unpleasent</code></pre>Additional tag: high resolution, elegantTraining setting: Adafactor with a batchsize of 40, lr_scheduler: constant with warm upSD3.5 Specific setting:enable_scaled_pos_embed = truepos_emb_random_crop_rate = 0.2weighting_scheme = "flow"Most SD3.5 M training so far freeze all text encoder, I tried to train clip with t5 cached, but its remains unclear if this is the reason why the subsequent training failed?

Files with SHA256: 02bacb2a855f8213570a841981b3101679b5e3fb952133df4efc4e3a8f256ca2