CivArchive
    ← All articles
    Published June 25, 2024by driftjohnson

    Talking Faces with AI | DDLP#1

    661 views0 reactions1 comments on CivitAI4 collected
    lipsyncvideo generation guidespeech

    The First DDLP - Deep Dive Live Project!
    This is where i have an idea and run it from start to finish, demonstrating a fully applied concept to completion. Along the way i can suggest branching choices which can be taken to further enhance the final version. This will use multiple services & methods with many AI services & software on display. In the end we can see what we must pay attention to while exposing some routes to improve this concept.


    Links:
    https://claude.ai/Civitai
    https://civitai.com/
    https://www.audacityteam.org/
    https://www.kits.ai/
    https://www.hedra.com/
    https://adobe.com
    https://lumalabs.ai/dream-machine

    Sections:

    00:00:00 Demo Video
    00:00:30 Introduction to DDLP
    00:05:30 Script Concept (CLAUDE)
    00:14:08 Image Dataset (LORA)
    00:17:52 Lora Training (CIVITAI)
    00:21:28 Initial Image Gens (COMFYUI)
    00:34:10 Puppet Options discussion
    00:59:59 Speech Dataset (AUDACITY)
    01:01:56 Phoneme Speech method
    01:09:17 Speech Model Training (KITSAI)
    01:15:22 Audacity, silence injection/removal
    01:17:50 Lipsync Video gen (HEDRA)
    01:24:31 Rotoscoping (AFTER EFFECTS)
    01:31:09 Dream Machine BG scenes (LUMAAI)
    01:42:05 First Draft
    01:52:06 Results & Conclusions