Introduction
This workflow is used to create AI Tiktok dance videos, with the action of the AI avatar driven by an actual dance video.
v7 includes audio reactive background using Depthflow and RyanOnThe Inside nodes.
Models Needed
The models required are the red nodes in the workflow, explanation is in the notes (brown nodes) in the workflow. Recommend to install with ComfyUI Manager.
SD1.5 LCM: https://civarchive.com/models/81458?modelVersionId=256668
SDXL Lightning: https://civarchive.com/models/133005/juggernaut-xl?modelVersionId=920957
AnimateLCM_sd15_t2v.ckpt (https://huggingface.co/wangfuyun/AnimateLCM)
Install Using Manager:
IPAdapter Models. ip-adapter_sd15.safetensors, ip-adapter-plus_sd15.safetensors.
Clip-Vision. CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors and CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors.
AnimateDiff V3: v3_sd15_mm.ckpt, v3_sd15_adapter.ckpt.
xinsir/ControlNet++: All-in-one ControlNet (ProMax model).
control_v11f1p_sd15_depth.safetensors
depth_anything_v2_vitl.pth
Custom Nodes Needed
Install missing custom nodes using manager.
ComfyUI's ControlNet Auxiliary Preprocessors
ComfyUI Frame Interpolation
ComfyUI_IPAdapter_plus
ComfyUI-Advanced-ControlNet
AnimateDiff Evolved
ComfyUI-VideoHelperSuite
rgthree's ComfyUI Nodes
ComfyUI Essentials
KJNodes for ComfyUI
Crystools
ComfyUI-Inspyrenet-Rembg
Depthflow Nodes
RyanOnTheInside
Description
FAQ
Comments (6)
Great and simple workflow! Thanks for sharing. My animation turns out very "cartoonish" though. How did you achieve the more realistic feeling of it?
The few things that come to mind are:
1) Use a realistic checkpoint, e.g. epiCPhotoGasm, Realistic Vision, epiCRealism
2) Use the same realistic photo checkpoint to generate the IP Adapter Image
3) Use "photo" in your positive prompt and "painting, drawing" in your negative prompt
4) Try to vary the IP Adapter present and weight, either lower or increase and check the outcome
5) Try to randomise your seed
6) Try to vary your cfg scale beween 1 - 2
Hope these help in your generations.
@benjamintay85779 thanks a lot for the detailed explanation! Will try the different methods you mention and share the outcome here when I'm done 🙂
I've updated the workflow (V2) to use animate LCM as it found that it gives more realistic generations. You can download the updated workflow to check if it gives you something closer to what you're looking for.
Great workflow, if you have an avatar with a while background, you can use an image in the 'blank' image location, for interesting effects (as in my upload)
Thanks. Yes, this is a great idea. The blank image can be changed to something that you want to appear in the video. Thanks for sharing the idea and your video.