Howto

Adjust prompt. Generate.

Description

This is old stuff. AnimateDiff was one of the first Video AI solutions for offline needs. And this is the most basic workflow for it. The base for other AnimateDiff workflows.

This workflow here is about prompt traveling with AnimateDiff. You choose different prompts for different keyframes. And in the video the prompts morphs into each other.

There are two workflows available. One for AnimateDiff 2 and 3. And one for the AnimateLCM, which is based at AnimateDiff 2.

AnimateDiff and Stable Diffusion 1.5, which are used in conjunction, are trained at 512 pixels in size. So you better won't go this much higher in creation size. The result will become inconsistent. What you can try is to upscale afterwards.

Both workflows comes with upscaling. But i recommend to upscale in an extra workflow. By a second Ksampler. See this article here:

https://www.tomgoodnoise.de/index.php/video-upscaling-in-comfyui/

Time

The example with a length of six seconds and a resolution of 1280 x 720 (simple upscaling with AI) finished in two and a half minutes at an 4060 TI with 16gb Vram. It does not use the best upscaling method though.

Requirements

These workflows were generated with 8 gb vram back in the days. However, i ran too often into an OOM. And when you want to do a proper upscaling then i highly suggest a card with at least 12 gb vram.