Single Image with Controllable Video to Control Camera movement in 6 directions: left, right, up, down, zoom_in, and zoom_out

The LoRa was trained to control camera movement in 6 directions: left, right, up, down, zoom_in, zoom_out.

Start prompt with text like this:

'Сamera moves to the {}...',
'Сamera is moving to the {}...',
'{} camera movement...',
'{} camera turn...',

This LoRA lora_r:128, alpha:1 The formula for lora_scale is: lora_r / alpha

The larger the lora_scale, the faster the rotation speed

The model is released under a permissive CogVideoX license.

To get started with ComfyUI;

1. Update to the latest version of ComfyUI

2. Download CogVideoX model weights into models/diffusion_models folder

3. Make sure a text encoder [1] is in your models/clip folder

4. Download the VAE to: ComfyUI/models/vae

5. Download the LoRA to ComfyUI/models/CogVideo/loras

CogVideoX has kijai ComfyUI support, and will run on 20GB+ or 9GB VRAM.

Github: https://github.com/THUDM/CogVideo

HuggingFace: https://huggingface.co/NimVideo/cogvideox1.5-5b-prompt-camera-motion