<a target="_blank" rel="ugc" href="https://github.com/TkskKurumi/DiffusersFastAPI/blob/main/model_as_lora.py">https://github.com/TkskKurumi/DiffusersFastAPI/blob/main/model_as_lora.py</a>I've attempted to develop an algorithm that can calculate LoRA from subtracting two trained full model. The core is to compress the full Δw (weight difference) matrix into LoRA. I think my matrix compression algorithm is not good enough. I'm not good at linear algebra (or I don't even know what fields of knowledge is needed for the task).Compress matrix W of shape(m, n) into matul of A and B of shapes (m, rank) and (rank, n). The compress ratio according to parameter amount is rank*(m+n)/(m*n). I've found that ratio = 25% may nearly represent <a rel="ugc" href="https://civarchive.com/models/18427">Fantexi </a>subtract <a rel="ugc" href="https://civarchive.com/models/101518/rmhf">my anime model</a>. And ratio=5% with lora alpha doubled has adequate result.I've also tried the algorithm to take image as input and use 2 matrixs of shapes (width, rank) (rank, height). Following image has original 1080 pixels width and height and compressed into low rank 32.<img src="https://image.civitai.com/xG1nkqKTMzGDvpLrqFT7WA/61fb92a4-9a67-49cf-848b-2038f75caf6c/width=525/61fb92a4-9a67-49cf-848b-2038f75caf6c.jpeg" />

LoRA Extraction tests

The LoRA ≈ A - B. Compress ratio = 25%A: <a target="_blank" rel="ugc" href="https://civitai.com/models/101518/rmhf">https://civitai.com/models/101518</a>B: <a target="_blank" rel="ugc" href="https://civitai.com/models/18427">https://civitai.com/models/18427</a>

Files with SHA256: 19799e27c7aee56b8fc5e2945dff52e38945ac2d27fe5d52f93a813ca9ab31eb