Version to Downloadv3.1 is a minor update to this workflow for users who have 16GB of VRAM. It generates the whole dance video in a single workflow.v4.0 of this workflow splits the workflow up into 5 parts to limit the VRAM usage to 12GB. Hopefully this opens the workflow up to a bigger group of users.ComfyUI VersionThis workflow was tested on ComfyUI v0.21 and should not have package conflicts when installing custom nodes from manager: <a target="_blank" rel="ugc" href="https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.2.1">https://github.com/comfyanonymous/ComfyUI/releases/tag/v0.2.1</a>IntroductionThis workflow uses Mimic Motion to generate the motion of the character and animate diff to create a 16 frame looping video of the background. Grounding Dino is then used to composite the character and background together. Best used with a driving video where the camera is static and all the motion is done by the dancer.The motivation is to leverage the strengths of both models to create as consistent a video as possible.LimitationsIf you watch the video above, you will realise that the dancer's interaction with her hair is not animated properly. The is due to the fact that Mimic Motion takes as input Open Pose information. If the depth information is used as well, the interaction with her hair would be animated properly.NodesNodes are colour coded:<ul><li>Red: These models must be downloaded and placed in the appropriate folders. After downloading, refresh ComfyUI and ensure the correct models are selected.</li><li>Green: These nodes are models that will be automatically downloaded</li><li>Blue: These are user inputs, either a video with the original dance moves or a prompt / setting.</li><li>Brown: These are comments that should explain what is going on with the workflow.</li></ul>Please use ComfyUI manager to install the missing nodes. These are the custom nodes that are used by the workflow:<ul><li>ComfyUI Impact Pack</li><li>ComfyUI's ControlNet Auxiliary Preprocessors</li><li>ComfyUI Frame Interpolation</li><li>AnimateDiff Evolved</li><li>ComfyUI-VideoHelperSuite</li><li>ReActor Node for ComfyUI</li><li>rgthree's ComfyUI Nodes, ComfyUI Essentials</li><li>ComfyUI-MimicMotionWrapper</li><li>segment anything</li><li>Comfyui lama remover</li></ul>ModelsSome models automatically download, some models are downloaded using ComfyUI manager and others have to be downloaded and placed in their respective folders manually.<ul><li>AnimateLCM: <a target="_blank" rel="ugc" href="https://huggingface.co/wangfuyun/AnimateLCM">https://huggingface.co/wangfuyun/AnimateLCM</a> Put AnimateLCM_sd15_t2v.ckpt in "models/animatediff_models" and put AnimateLCM_sd15_t2v_lora.safetensors in "models/loras"</li><li>SD1.5 Checkpoint: you can use your favourite depending on the type of video you are generating (realistic / anime). I use Realistic Vision as I am aiming for a realistic generation.</li><li>control_v11p_sd15_openpose.safetensors. Download with ComfyUI manager</li><li>MimicMotion, Segment Anything, Grounding Dino, ReActor models should download automatically</li><li>Please follow the instructions in the Troubleshooting section (<a target="_blank" rel="ugc" href="https://github.com/Gourieff/comfyui-reactor-node">https://github.com/Gourieff/comfyui-reactor-node</a>) if you are having troubles installing insightface</li></ul>

Tik Tok Dance Workflow (Mimic Motion + Animate Diff)

Minor changes to the workflow<ul><li>Updated the calculation of frames to use when less than 72 frames to frame_count + 1, this fixes a bug that uses double the sampling steps because there is 1 additional frame for the reference image.</li><li>Updated the Mimic Motion Scheduler to use Align Your Steps as per this issue: <a target="_blank" rel="ugc" href="https://github.com/kijai/ComfyUI-MimicMotionWrapper/issues/8">https://github.com/kijai/ComfyUI-MimicMotionWrapper/issues/8</a> and reduced the sampling steps to 10. </li><li>Introduced Condition Zero Out for the AnimateLCM conditioning as we are using CFG 1.0 which ignores the negative prompt.</li></ul>

Files with SHA256: edb2b0f5462860ad7b81a79fb90a9cdda3ccaea68ebe165a74aa07b989ec7a79