IntroductionThis workflow uses Segment Anything v2 to segment out the part of the original video to be replaced. Then the Set Latent Noise Mask node is used before passing the VAE encoded video into the KSampler.You will provide a reference image to be used with IPAdapter and also a prompt to guide the generation of the new object. The depth controlnet is used to give some guidance on what the shape of the replaced object should be.Models<ol><li>SD1.5 Checkpoint: I use a LCM checkpoint so I do not need to use the AnimateLCM LoRA. You can use your favourite SD1.5 checkpoint. <a target="_blank" rel="ugc" href="https://civarchive.com/models/4384?modelVersionId=252914">https://civarchive.com/models/4384?modelVersionId=252914</a></li><li>AnimateLCM_sd15_t2v.ckpt: Download from <a target="_blank" rel="ugc" href="https://huggingface.co/wangfuyun/AnimateLCM/tree/main">https://huggingface.co/wangfuyun/AnimateLCM/tree/main</a> and put in models/animatediff_models</li><li>control_v11f1p_sd15_depth_fp16.safetensors and depth_anything_v2_vitl.pth. Download using Manager.</li><li>IPAdapter PLUS: download ip-adapter-plus_sd15.safetensors using ComfyUI manager. Also download ViT-H and ViT-G clip_vision models using the ComfyUI manager. </li><li>SAM2 models should automatically download</li></ol>Custom NodesInstall with Manager:<ol><li>ComfyUI-VideoHelperSuite</li><li>Crystools</li><li>KJNodes for ComfyUI</li><li>ComfyUI's ControlNet Auxiliary Preprocessors</li><li>ComfyUI-Advanced-ControlNet</li><li>AnimateDiff Evolved</li><li>ComfyUI_IPAdapter_plus</li></ol>

Files with SHA256: ca8b5727915943d484c04387de80083af3a6df7027d9f59eeb30f2a0f4bdbe88