CivArchive
    ← All articles
    Published December 18, 2024by Motoyo

    When merging Models: SDXL Alpha Block weights and where the apply, and what they affect.

    438 views9 reactions1 comments on CivitAI23 collected
    mergestool guide

    After diving into the rabbit hole of merging models, and being lost on what IN00 or OUT08 or M00 was, or what it did, or how to use it.

    I started playing with SuperMerger:
    https://github.com/hako-mikan/sd-webui-supermerger

    And between hours of sliding them randomly without being able to tell what did what, I figured out the best way is to use a VERY anime Pony, and a very realistic SDXL to merge, so I can see a big difference on sliding values on the blocks.

    Along with pestering ChatGPT with my findings, assumptions, corrections, and keeping it organized for me, and giving me feedback, this is what I found:

    Note:

    I may be wrong in this, if so feel free to correct me. I am only armchairing speculation over what it took to convince a pure anime Pony mode to make realistic pics, while keeping it Pony poses, knowledge, and proportions for people.

    It seems it processes blocks in a linear(ish) pattern of noise to the final image, each impacting the process along the way, and which model it leans towards for that step in the process of noise -> final image.

    In the example below:

    Model A is a highly anime based Pony model.
    Model B is a highly realistic SDXL model.

    Input and Early Processing (Base and IN00–IN04)

    • Step: These blocks process the initial noise input and create the foundational structure of the image.

    • Blocks: Base, IN00–IN04

    • Function:

      • Define the scene layout (composition, perspective, and object placement).

      • Handle poses and anatomy for characters or objects in the scene.

    • Impact of Weights:

      • High Model A weight: Keeps the structure anime-like (e.g., exaggerated poses, stylized proportions).

      • High Model B weight: Makes the structure more realistic (e.g., accurate human anatomy, balanced proportions).

    2. Downsampling Path (IN05–IN11)

    • Step: These blocks extract increasingly detailed features by reducing the image resolution (downsampling).

    • Blocks: IN05–IN11

    • Function:

      • Refine mid-level details, like shapes, textures, and shading.

      • Begin introducing style features (e.g., anime or realism textures, basic shading patterns).

    • Impact of Weights:

      • High Model A weight: Keeps mid-level details stylized (anime-style texturing and shading).

      • High Model B weight: Introduces realistic textures and consistent shading.

    3. Bottleneck (m00)

    • Step: This block, also called the middle block, connects the downsampling and upsampling paths.

    • Blocks: m00

    • Function:

      • Encodes the global style and coherence of the image.

      • Balances the influence of broad structure and fine details.

    • Impact of Weights:

      • High Model A weight: Preserves anime-like global style.

      • High Model B weight: Shifts the overall style toward realism.

    4. Upsampling Path (OUT00–OUT03)

    • Step: These blocks begin refining the image by adding detail as the resolution increases (upsampling).

    • Blocks: OUT00–OUT03

    • Function:

      • Handle mid-level textures and shading transitions.

      • Ensure style consistency between the structure and details.

    • Impact of Weights:

      • High Model A weight: Retains stylized textures and shading patterns.

      • High Model B weight: Adds realistic texturing and tonal depth.

    5. Fine Detail Refinement (OUT04–OUT07)

    • Step: These blocks refine the high-resolution details, like facial features, skin textures, and intricate elements.

    • Blocks: OUT04–OUT07

    • Function:

      • Enhance faces, fine textures, and other small details.

      • Add realism or stylization to critical focal points (e.g., eyes, skin, hair).

    • Impact of Weights:

      • High Model A weight: Faces and details remain anime-like (simplified or stylized).

      • High Model B weight: Faces and details become highly realistic.

    6. Final Touches (OUT08–OUT11)

    • Step: These blocks handle the final stages of image generation, adding shadows, highlights, and other finishing touches.

    • Blocks: OUT08–OUT11

    • Function:

      • Enhance lighting effects, shadows, and overall rendering quality.

      • Ensure the final image looks cohesive and polished.

    • Impact of Weights:

      • High Model A weight: Keeps stylistic shadows and artistic flair.

      • High Model B weight: Adds realistic shadows, highlights, and tonal balance.

    Summary of Block Roles in Image Generation

    Stage

    Blocks Involved

    Role

    Initial Structure

    Base, IN00–IN04

    Define composition, pose, and basic structure.

    Feature Extraction

    IN05–IN11

    Add mid-level textures, shapes, and shading.

    Global Style

    m00

    Blend and set the overall style of the image.

    Detailing

    OUT00–OUT03

    Handle mid-level textures and shading transitions.

    Fine Refinement

    OUT04–OUT07

    Add realistic or stylized high-resolution details.

    Final Rendering

    OUT08–OUT11

    Apply shadows, highlights, and polish the image.

    Impact of Weight Changes

    • Early Blocks (Base, IN00–IN04):

      • Control the scene, composition, and pose.

      • Change these to lean toward Model A for anime-style composition or Model B for realism.

    • Mid Blocks (IN05–OUT03):

      • Balance textures and shading.

      • Adjust these to emphasize stylistic or realistic textures.

    • Late Blocks (OUT04–OUT11):

      • Focus on faces, details, and lighting.

      • Favor Model B for realistic faces and polished rendering.