Flux2KLein 9B Character Lora with Targeted blocks!! UPDATED

So in the past few weeks I have been dedicating long hours into finding optimal approaches to preserve as much of the ref latent and basically force the model to do two things; preserve the exact features and be flexible.. and it has been such a pain but I think I stumbled accidentally at many interesting features of this model and it’s architecture.. as I tinkered with every possible corner you can tinker with from conds to attn layers to all q,k,v … double and single blocks and more.. overall all I found some valuable information for people who would like to train loras and knowing what to actually target.. and I was wrong while back by publishing a map of where the character lives.. anyways here we go:

Double blocks 0-1 is just base early on where the model is just doing its thing, poses and such are beginning to form here.

Double 2-3 is where the model recognizes the colors of outfit but no outfit / character yet.

Double 4-5 is where the model locks the outfit/ body proportions but not the character’s facial features.

6-7 is where the model locks the character/outfit/features.

Singles 0-23 all just model’s style and textures no actual physical changes nor proportions or features .

And finally yes I need a break from this model.. 😂

I ran a batch for fun with these layers and the results are clean without destroying the model's knowledge or composition, it made it feel like my character was pre-trained with original flux2klein :D

      network:
        type: "lora"
        linear: 32
        linear_alpha: 32
        conv: 16
        conv_alpha: 16
        lokr_full_rank: true
        lokr_factor: -1
        network_kwargs:
          ignore_if_contains: []
          only_if_contains:
            - "double_blocks.6"
            - "double_blocks.7"
            - "single_blocks.0"
            - "single_blocks.1"
            - "single_blocks.2"
            - "single_blocks.3"
            - "single_blocks.4"
            - "single_blocks.5"
            - "single_blocks.6"
            - "single_blocks.7"
            - "single_blocks.8"
            - "single_blocks.9"
            - "single_blocks.10"

Config file for who want's to test it out : https://pastebin.com/qAP6AJia