Introduction
This workflow is used to create AI Tiktok dance videos, with the action of the AI avatar driven by an actual dance video.
v7 includes audio reactive background using Depthflow and RyanOnThe Inside nodes.
Models Needed
The models required are the red nodes in the workflow, explanation is in the notes (brown nodes) in the workflow. Recommend to install with ComfyUI Manager.
SD1.5 LCM: https://civarchive.com/models/81458?modelVersionId=256668
SDXL Lightning: https://civarchive.com/models/133005/juggernaut-xl?modelVersionId=920957
AnimateLCM_sd15_t2v.ckpt (https://huggingface.co/wangfuyun/AnimateLCM)
Install Using Manager:
IPAdapter Models. ip-adapter_sd15.safetensors, ip-adapter-plus_sd15.safetensors.
Clip-Vision. CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors and CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors.
AnimateDiff V3: v3_sd15_mm.ckpt, v3_sd15_adapter.ckpt.
xinsir/ControlNet++: All-in-one ControlNet (ProMax model).
control_v11f1p_sd15_depth.safetensors
depth_anything_v2_vitl.pth
Custom Nodes Needed
Install missing custom nodes using manager.
ComfyUI's ControlNet Auxiliary Preprocessors
ComfyUI Frame Interpolation
ComfyUI_IPAdapter_plus
ComfyUI-Advanced-ControlNet
AnimateDiff Evolved
ComfyUI-VideoHelperSuite
rgthree's ComfyUI Nodes
ComfyUI Essentials
KJNodes for ComfyUI
Crystools
ComfyUI-Inspyrenet-Rembg
Depthflow Nodes
RyanOnTheInside
Description
Revamp workflow to make background audio reactive
FAQ
Comments (3)
Hello. I am having a issue with the Audio Separator. I cannot find the same one and will not download from the Install missing nodes section. Where can I find it? Other than that everything else is in place.
it has been some time since i worked on this workflow. if i'm not wrong, the audio separator should be part of the audio reactive background part of the workflow. this node might have been removed by the authors.
if audio reactive is not important for you, you can try to remove that part of the workflow.
This workflow is fantastic! I was wandering is it possible to add hand refiner to this workflow. I can find some workflow to fix hand for a image, but I don't know how to fix hand in the video. thanks!