CivArchive
    ← All articles
    Published March 20, 2025by sirrece

    Generative AI as a signal processing problem - practical methods for improving winrates on quality

    135 views5 reactions0 comments on CivitAI3 collected
    generation guidesdxllow step inferencewinratesnegativesefficiency

    Link to the full video (all content on the Patreon is free and public):

    https://www.patreon.com/posts/124780951

    Included negative expansion helper:

    https://chatgpt.com/g/g-67db5a31fd7c81919d077b987ae5a48b-yolkhead-8-shot-generator


    Summary of the video:
    Generative AI as a Signal Processing Problem

    (Organized Transcript)

    1. Introduction: The Core Idea

    • Generative AI can be viewed through the lens of signal processing, not just as an artistic problem.

    • Artistic instincts can bias us to use familiar workflows (e.g., DAWs for musicians). This can be helpful but also limiting.

    • Node-based workflows offer superior flexibility and results compared to traditional workflows like DAWs.

    Why a Signal Processing Approach?

    • Generative AI involves tuning a "signal," much like adjusting an antenna to improve TV/radio reception.

    • Analogy: Imagine shining a flashlight onto your hand or paper:

      • Step count: How long the light shines (longer = clearer image, but too long can cause "burn-in").

      • CFG (Classifier Free Guidance): The strength/wavelength of the light. Higher CFG = brighter, sharper image, but risks artifacting.

      • Resolution: Distance between the flashlight and surface. Further = less clear, closer = sharper.

    2. Managing Common Problems in Generative AI

    Common Issues:

    • Artifacting: When results appear distorted or "overexposed" (similar to overly bright spots from the flashlight analogy).

      • Caused by:

        • High CFG

        • Too many sampling steps

        • Inappropriate resolution

    Solutions and Adjustments:

    • Adjust CFG: Lower to reduce artifacting; increase carefully to sharpen.

    • Adjust Step Count: Often, reducing steps can actually improve outcomes and reduce compute.

    • Adjust Resolution: Sometimes increasing image size (resolution) can resolve CFG-related artifacting issues without adding steps.

    Key Insight:

    • High CFG and lower step count often yield better quality results efficiently.

    • Lowering CFG and increasing steps can increase stability but decreases efficiency significantly.

    3. Sticky Negatives: A Powerful Method

    Explanation of Sticky Negatives:

    • Combines positive and negative prompts carefully.

    • Purpose: Reduces artifacting by helping AI precisely understand what to avoid (negative) and what to create (positive).

    • Anchors negatives specifically to the positives to ensure clarity in guidance.

    How to Use Sticky Negatives Effectively:

    • Create detailed and expansive negative prompts, often larger than positive prompts.

    • Rotate around core concepts, refining negatives iteratively to increase precision.

    • Negative prompts need clear descriptors of what's undesirable (e.g., “too many legs,” “bad anatomy”).

    Practical Demonstration:

    • Successfully showed clearer, more accurate images by increasing negative complexity and adjusting CFG/steps carefully.

    • Demonstrated improved image quality and reduced artifacts using minimal compute resources (few steps).

    4. Advanced Prompting Techniques: Rolling Prompts

    • Rolling Prompts: Repeating prompt phrases multiple times to leverage the AI’s context window limitations.

    • Helps the AI achieve more nuanced, multi-perspective interpretations, reducing overfitting and increasing overall quality.

    5. Extending Techniques to Music Generation (Sunno)

    Key Concept: Human Preference

    • Hypothesis: There's a universal, underlying "human preference" for certain patterns or qualities.

    • Aim: To align generative models closer to this baseline human preference by carefully choosing and structuring negatives.

    Practical Music Prompting:

    • Uses broad descriptors and carefully selected years (based on objective measures like Billboard data accuracy).

    • Avoids prompts associated with biased data (e.g., religious, state-controlled music) to avoid distorted preference signals.

    • Result: Improved realism, subtlety, and clarity in generated audio outputs.

    6. General Recommendations & Best Practices

    Signal Processing Mindset:

    • Think of generative AI as tuning a clear signal:

      • CFG = Signal Strength

      • Steps = Exposure Time

      • Resolution = Distance to Surface

    Optimizing Efficiency:

    • Fewer sampling steps often yield better results if CFG and negative prompts are carefully managed.

    • Lower CFG, very high step counts often not efficient—there’s a sweet spot balance.

    • Quickly identify artifacting and adjust settings accordingly (CFG, steps, resolution, prompt expansion).

    Negative Prompts are Crucial:

    • Do not underestimate the importance of negatives.

    • Expanding negatives precisely is often more impactful than expanding positives.

    • A comprehensive negative strategy significantly improves accuracy, reducing the need for corrective techniques (like inpainting).

    7. Summary of Critical Takeaways:

    • Generative AI as signal processing: adjusting CFG, sampling steps, and resolution to optimize clarity.

    • Sticky negatives and rolling prompts help refine and stabilize AI output.

    • Comprehensive negative prompting can significantly improve quality and efficiency.

    • Applying these methods consistently yields superior results in both image and audio generation.