Introduction
This workflow is used to create AI Tiktok dance videos, with the action of the AI avatar driven by an actual dance video.
v7 includes audio reactive background using Depthflow and RyanOnThe Inside nodes.
Models Needed
The models required are the red nodes in the workflow, explanation is in the notes (brown nodes) in the workflow. Recommend to install with ComfyUI Manager.
SD1.5 LCM: https://civarchive.com/models/81458?modelVersionId=256668
SDXL Lightning: https://civarchive.com/models/133005/juggernaut-xl?modelVersionId=920957
AnimateLCM_sd15_t2v.ckpt (https://huggingface.co/wangfuyun/AnimateLCM)
Install Using Manager:
IPAdapter Models. ip-adapter_sd15.safetensors, ip-adapter-plus_sd15.safetensors.
Clip-Vision. CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors and CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors.
AnimateDiff V3: v3_sd15_mm.ckpt, v3_sd15_adapter.ckpt.
xinsir/ControlNet++: All-in-one ControlNet (ProMax model).
control_v11f1p_sd15_depth.safetensors
depth_anything_v2_vitl.pth
Custom Nodes Needed
Install missing custom nodes using manager.
ComfyUI's ControlNet Auxiliary Preprocessors
ComfyUI Frame Interpolation
ComfyUI_IPAdapter_plus
ComfyUI-Advanced-ControlNet
AnimateDiff Evolved
ComfyUI-VideoHelperSuite
rgthree's ComfyUI Nodes
ComfyUI Essentials
KJNodes for ComfyUI
Crystools
ComfyUI-Inspyrenet-Rembg
Depthflow Nodes
RyanOnTheInside
Description
Notice
v6 of the workflow is broken due to the update. To fix it, delete all 5 Points Editor and SAM2 Segmentation nodes, re-add them and connect them up.
Replaced Grounding Dino with Segment Anything v2.
Added individual IPAdapters for the Hair, Skin, Clothes (top) and Clothes (bottom)
Added a 2nd pass KSampler to clean up the details
FAQ
Comments (3)
Hello @PixelMuseAI , I was able to use your workflow before, but after I updated comfyui last night, this workflow can no longer run. The error message is as shown in the picture. Can you tell me how to fix it? This is very important to me because this workflow is the best workflow on the entire network. Thank you very much!
https://drive.google.com/file/d/1GIB5ms4n8shCv-IylUQ28mR4zfLyxWBI/view?usp=sharing
Detailed log:
got prompt
Failed to validate prompt for output 51:
* Sam2Segmentation 23:
- Return type mismatch between linked nodes: mask, STRING != MASK
* Sam2Segmentation 19:
- Return type mismatch between linked nodes: mask, STRING != MASK
* Sam2Segmentation 27:
- Return type mismatch between linked nodes: mask, STRING != MASK
* Sam2Segmentation 15:
- Return type mismatch between linked nodes: mask, STRING != MASK
* Sam2Segmentation 11:
- Return type mismatch between linked nodes: mask, STRING != MASK
Output will be ignored
Failed to validate prompt for output 112:
Output will be ignored
Failed to validate prompt for output 173:
Output will be ignored
Failed to validate prompt for output 54:
Output will be ignored
Failed to validate prompt for output 94:
Output will be ignored
Prompt executed in 0.30 seconds
Thanks for reporting. Workflow is indeed broken due to the update. I will release an updated workflow soon.
To fix it on your own, please delete all 5 Points Editor and SAM2 Segmentation nodes, re-add them and connect them up. This should fix the workflow.
@PixelMuseAI I fixed it, thank you very much for your advice, grateful.