--- the color correct node doesn't seem to work anymore, please leave a comment if you have a solution ---
Simplified Image 2 Video for a video made from 1 image, adaptation of Ipiv's Morph img2vid workflow.
Since I made some videos from single images of mainly waves I get a lot of questions in comments, on articles and via private messages about how this, how that, so now I can send them here! =)
The original image is 896x1152, which was a recommended portrait resolution for SDXL, I don't want to use upscaling, cause that takes forever, but also not end up with a small video, so I used half of that (576x448) as latent image.
Download the json file, install the missing nodes via the manager and go!
Required files and their folders
animatediff_models -> AnimateLCM_sd15_t2v.ckpt
animatediff_motions -> AnimatedDiffusion-Blank.mp4 (link)
loras -> AnimateLCM_sd15_t2v_lora.safetensors
loras -> Hyper-SD15-8steps-lora.safetensors (you can also use LoRA above twice)
checkpoints -> juggernaut_reborn.safetensors (or another SD1.5 model)
vae -> vaeFtMse840000EmaPruned_vae.safetensors
controlnet -> control_v1p_sd15_qrcode_monster.safetensors
IPAdapter PLUS (important for consistancy!)
ipadapter -> ip-adapter-plus_sd15.safetensors
/ComfyUI/models/clip_vision folder
Download and rename to "CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors"
https://huggingface.co/h94/IP-Adapter/resolve/main/models/image_encoder/model.safetensors
Download and rename to "CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors"
https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/image_encoder/model.safetensors
The prompt
Unlike the note says in the workflow, the prompt makes a difference!
I hope this will get you on the way, more info in the notes in from the original Ipiv workflow!
Description
First release
FAQ
Comments (5)
Muchas Gracias Amigo!
Do I understand correctly that load video(path) contains just a white background? I'm asking this because only you can see this mask; it's not included in the standard comfyui build. So, if it is really a white background there, then CreateFadeMask does not make sense since white on white will be white.
For one image the whole mask thing could probably be removed also, but I just wanted to make a quick version, not to answer the same question every day. Now you can still see how it is done, or use the original version, but with these params.
@JayNL perhaps I wrote it difficult. In your workflow there is no loaded mask in "load video(path)", since instead of a link there is a path to the "comfyui" folder. Apart from this, "CreateFadeMask" does nothing in your workflow, if I correctly assumed that there is an empty white mask in "load video(path)". I have no idea what the same questions are being asked to you, but when you post such lazy workflow they are inevitable.
I dont understand the logic to use QRcode monster for control net