<h3 id="final-2-(update-for-the-original-story-see-below)">Final-2 (update, for the original story, see below)</h3>You know it is never the last version when you write "final" in its name :DI re-did a training and an alternate merge thanks to a great remark by @Neural_LensFirst, i lowered the LR and did more Epoch, switched back to the classical cosine and Prodigy as optimizer. I also reduced the dimension (16) but choose a lower alpha (4) to get more impact on value change.<pre><code>{
 "engine": "kohya",
 "unetLR": 0.0001,
 "clipSkip": 2,
 "loraType": "lora",
 "keepTokens": 0,
 "networkDim": 16,
 "numRepeats": 2,
 "resolution": 1024,
 "lrScheduler": "cosine",
 "minSnrGamma": 0,
 "noiseOffset": 0.03,
 "targetSteps": 2400,
 "enableBucket": true,
 "networkAlpha": 4,
 "optimizerType": "Prodigy",
 "textEncoderLR": 0,
 "maxTrainEpochs": 20,
 "shuffleCaption": true,
 "trainBatchSize": 4,
 "flipAugmentation": false,
 "lrSchedulerNumCycles": 1
}</code></pre>On top of that, i leveraged for training <a target="_blank" rel="ugc" href="https://civarchive.com/models/2108945/neural-lens-core">Neural Lens Core</a> which feel like the perfect Illustrious model for training, much better that the base Illustrious V0.1 or a random WAI iteration.After some test, i selected several Epoch:<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/2b976dac-6ed5-4eeb-ba70-43b820948d0a/width=525/2b976dac-6ed5-4eeb-ba70-43b820948d0a.jpeg" />And did a new merge to see if TIES and DARE could work:<pre><code>from safetensors.torch import save_file
import sd_mecha as sdm

sdm.set_log_level()

base = sdm.model("test3-000010.safetensors")

models = [
 sdm.model("test3-000012.safetensors"),
 sdm.model("test3-000018.safetensors"),
 sdm.model("test3-000019.safetensors"),
]

recipe_dare = sdm.ties_with_dare(base,*models,probability=0.6,seed=42,alpha=0.5)

test = sdm.merge(recipe_dare,output_device="cpu")

for k in [ k for k in test.keys() if "lora_te" in k ]:
 del(test[k])

save_file(test,"test3-dare.safetensors")</code></pre>At this point, when i did the merge, i saw that using prodigy did not honour my TE LR of 0, that's why i am removing the TE layers.In any case, it works differently, is also nice and if you hesitate between both, you can just slap the two together :D<h3 id="first-and-last">First and Last</h3>So, a quick test. Goal was to replicate a specific style, especially for eyes and lips. After finding a combinaison of LoRA, i generated a bunch of random 1girl (60 pictures), did a pass of autotag and started a quick training to spend some blue buzz.The training parameters:<pre><code>{
 "engine": "kohya",
 "unetLR": 0.0005,
 "clipSkip": 2,
 "loraType": "lora",
 "keepTokens": 0,
 "networkDim": 32,
 "numRepeats": 2,
 "resolution": 1024,
 "lrScheduler": "cosine_with_restarts",
 "minSnrGamma": 0,
 "noiseOffset": 0.03,
 "targetSteps": 1440,
 "enableBucket": true,
 "networkAlpha": 32,
 "optimizerType": "Adafactor",
 "textEncoderLR": 0,
 "maxTrainEpochs": 12,
 "shuffleCaption": false,
 "trainBatchSize": 1,
 "flipAugmentation": false,
 "lrSchedulerNumCycles": 3
}</code></pre>Nothing fancy (PS: i know those aren't the best parameters, i was also doing some testing :D)I selected both epoch 7 and 9 that looked nice, and why i am posting this is because, i often merge LoRA training epochs.I know that's not something that is often done, i usually do it manually, for once, i tested a technique that can be reproduced:<pre><code>&gt;&gt;&gt; import sd_mecha as sdm
&gt;&gt;&gt; sdm.set_log_level()
&gt;&gt;&gt; a = sdm.model("test-000007.safetensors")
&gt;&gt;&gt; b = sdm.model("test-000009.safetensors")
&gt;&gt;&gt; c = sdm.slerp(a,b)
&gt;&gt;&gt; sdm.merge(c,output="test-slerp.safetensors")</code></pre>Anyway, that's all folks. :DBase idea is here (for the source LoRA): <a target="_blank" rel="ugc" href="https://civarchive.com/images/109655271">https://civarchive.com/images/109655271</a>It was trained on <a target="_blank" rel="ugc" href="https://civarchive.com/models/1224788?modelVersionId=2081643">PrefectIllustrious V3</a>.

Style-96

Posted on Civitai, upload found elsewhere are done without the author permission.

Files with SHA256: 1e062f3bf0e5ce0e7c42f6616cc05727443695be4dc6e5b5063a9871d0f6144e