简单高效的进行模型混合。这是我自己研究出来的模型混合方式，总之先分享下我的发现，我也好奇你们混合模型的方法。Simple and efficient model blendingThis is a model blending method I developed myself. In short, I just wanted to share my findings, and I'm also curious about your methods for blending models混合原理利用了模型混合的过拟合现象，并在此基础上进行了深入研究。因此这个混合方式很可能是通用于所有AI模型。希望如此，但显然要验证这些有点困难。The mixing principle makes use of the over-fitting phenomenon of model mixing, and on this basis, it is deeply studied.So this hybrid approach is likely to be common to all AI models. I hope so, but it's obviously a little hard to verify.在我的设想中，这个混合方式可以解决困扰模型混合最大的问题，素材的缺失。In my vision, this hybrid approach can address the biggest issue troubling model mixing, which is the lack of materials研究的阻力好多啊，版本落后、冷门领域、没有参考、素材有限、没有支持、甚至有一些发现和猜想还没有人提出。目前最大的问题反而不是时间、和个人能力。总之你们有兴趣尝试一下吗，探索未知可是很有趣的哦。There are many obstacles in research, such as outdated versions, niche fields, lack of references, limited materials, no support, and even some discoveries and hypotheses that no one has proposed yet. The biggest issues right now are not time or personal ability. In any case, are you interested in giving it a try? Exploring the unknown can be quite fun现在补充一些相关的研究信息I will now provide some additional research informationLORA的混合权重绝大部分LORA的最大权重在50~100这个阶段，超过这个阶段会出现数值溢出，因此需要开启数值溢出检测（nan-check）Lora's mixed weightsThe maximum weight of most Lora is in the range of 50 to 100. Exceeding this range may lead to numerical overflow, so it is necessary to enable numerical overflow detection (nan-check)LORA的权重阶段随着LORA权重数值的提高，LORA会出现以下几个阶段的变化。1.模型信息主导。2.LORA信息主导，模型信息补充。3.LORA信息主导。4噪点5数值溢出The weight stage of LoraAs the Lora weight values increase, Lora will undergo the following stages of change1. Model information is dominant2. Lora information is dominant, while model information is supplementary3. Lora is information-driven4 noise points5 numerical overflow每一个LORA都有不同的最大权重，我将这个最大权重称作学习深度。Each Lora has a different maximum weight, which I refer to as learning depth我现在解释混合细节I will now explain the mixed details1.LORA权重为什么要设置这么高LORA的权重代表在混合时的模型占比，调高是为了避免多余的模型影响1.Why do Lora weights need to be set so highThe weight of Lora represents the proportion of the model during mixing, and increasing it is to avoid the influence of unnecessary models2.模型混合的α数值α=-1 这是用来获取LORA信息。α=-2 则是反转了获得的LORA信息。2.The α value for model mixingα = -1 is used to obtain lora information. α = -2 reverses the obtained lora information3.如何在原有基础上再次进行混合将获得的模型进行活化处理，此过程需要13次混合。之后便可以添加下一个LORAActivation the obtained model, which requires 13 mixing. After that, you can add the next LoRA.4.混合的缺陷由于这种混合方式是获取了LORA的全部信息，所以混合的LORA越多，效果越不稳定。目前我没有找到解决方法。4. Mixed defectsSince this mixing method captures all the information from Lora, the more mixed Lora there is, the less stable the effect becomes. I haven't found a solution for this yet5. 是否还有其他混合方式可以做到这种效果其他的混合方式也可以做到这种效果，但只有这个方法的下限是最高的5. Are there any other mixing methods that can achieve this effectOther mixing methods can achieve this effect as well, but only this method has the highest lower limit6.这个混合方式的原理是什么我通过大量过拟合混合找到了一些规律，但原理仍然未知，甚至难以解释。简单来讲，便是通过一些特殊的模型来提取其他模型的信息，在对提取到的模型进行处理。所以为什么会出现这些现象呢。我只能回答，我不知道。6.What is the principle behind this mixing methodI have found some patterns through extensive overfitting mixing, but the principles remain unknown and are even difficult to explain. In simple terms, it involves using certain special models to extract information from other models and then processing the extracted models.So why do these phenomena occur? I can only answer, I don't know之后你们会获得很多对LORA的新认知。总之我的研究已经进入了停滞，并且很难会有突破了。You will gain a lot of new insights about Lora. In short, my research has reached a standstill, and it's unlikely to have any breakthroughs.以下是其余研究的关联页面。因为页面篇幅有限，其余信息将在别的页面发布。目前还在编辑中Here are the related pages for the remaining research. Due to space limitations, the rest of the information will be published on other pages. It is still being edited模型拆解研究页面 Model Deconstruction Research Page<a target="_blank" rel="ugc" href="https://civarchive.com/models/579793">https://civarchive.com/models/579793</a>使用此方法进行模型优化的演示页面Demonstration page for model optimization using this method<a target="_blank" rel="ugc" href="https://civarchive.com/models/822917/km-mix-unexpected-product">https://civarchive.com/models/822917</a>

simple and efficient model mixing method

<a target="_blank" rel="ugc" href="https://civitai.com/models/12427/old-school-shoujo">https://civitai.com/models/12427/old-school-shoujo</a>权重150 Weight 150<a target="_blank" rel="ugc" href="https://civitai.com/models/6304?modelVersionId=7387">https://civitai.com/models/6304?modelVersionId=7387</a>权重650 Weight 650原计划上传一个训练错误的LORA混合样本，不过这个组合也很不错。Noob-leyon LORA训练时出现了错误，我会上传一个图片进行演示。这个现象其实很常见，大部分的LORA都不满足混合的最低标准，据我观察合格率大概有百分之一。以下是混合中LORA的常见表现LORA未完整学习，LORA学习深度不足，LORA错误学习。还有一部分LORA训练标注有问题，总之，早期LORA存在很多很多的不足，我希望现在的LORA已经有所改善。但是在那之前我需要想办法解决提示词标注同类化造成的混合稀释，嗯看起来完全不可能做到。I originally planned to upload a training error lora mixed sample, but this combination is also quite goodnoob - an error occurred during the training of leyon lora, and I will upload an image to demonstrate. This phenomenon is actually quite common; most lora do not meet the minimum standards for mixing. From my observations, the pass rate is about 1 percent.Here are the common manifestations of mixed Lora.Lora has not fully learned, the depth of Lora's learning is insufficient, and there are errors in Lora's learning. Additionally, some of the training annotations for Lora have issues. In short, there were many shortcomings in the early stages of Lora, and I hope that the current version of Lora has improved.But before that, I need to find a way to address the mixed dilution caused by the homogenization of prompt labeling, which seems completely impossible to achieve

Files with SHA256: 5ba813264c9e65583c8f7b2deef5cb7ab0fd5d7b011ba103f4f1d521d68ac128