Model Description
This repository contains a collection of basic camera motion LoRAs finetuned on Animatediff v2 models. The training process utilized Kijai's AnimateDiff Motion Director ComfyUI nodes. These LoRAs have been tested exclusively in ComfyUI with Kosinkadink's AnimateDiff-Evolved nodes, however they may be compatible with other AnimateDiff implementations.
How to use
copy the motion lora files into your "ComfyUI\models\animatediff_motion_lora" folder
Use the "Load Animatediff LoRA" node
General recommendation
These are finetuned on a v2 model. They work also with AnimateLCM but don't work with v3 models
They work with SD 1.5 checkpoints
Animatediff v3 adapter LoRA is recommended regardless they are v2 models
If you want more motion try increasing the scale multival
I recommend using the above Base AR (Aspect ratio) for inference however they might work with other ARs too
Try playing with the strength of
motion LoRA
scale multival
v3_adapter lora
like increasing the scale multival and lowering the lora strength.
I recommend using IP-Adapter, however it reduces the effect of the mLORAs, so play with the strength. "Ease in out" weight type is a good starting point.
If you chain multiple motion LoRAs, I recommend lowering their strength, otherwise, the quality of the result will be worse
Example workflow
You will find an example workflow here: https://civarchive.com/articles/6702
With this you will be able to recreate this animation: https://civarchive.com/images/24310950
Details
Zooming in (16 frame)
recommended strength: 0.7-1.0
Zooming in (32 frame)
this is an experimental motion LoRA trained on 32 frames so it can be used with higher context length than 16.
recommended strength: 0.7-0.85
Zooming out
recommended strength: 0.7-1.0
Lateral (horizontal) left
recommended strength: 0.7-1.0
Lateral (horizontal) right
recommended strength: 0.7-1.0
Crane (vertical) up
recommended strength: 0.7-1.0
Crane (vertical) down
recommended strength: 0.7-1.0
Description
FAQ
Comments (12)
Amazing, thank you for sharing! any recommened trigger words? Just the lora names i guess?
For motion LoRAs no trigger word is necessary. Obviously there are scenes that are biased towards certain motions, because of that the prompts can influence the strength of the motion LoRA but trigger word is not needed. If you load the Lora into the Animatediff LoRA loader in ComfyUI it will work right away.
I can't make this work :/ No matter what I do (even using your recommend settings) I see no movement in the output.
are you sure you use Animatediff v2 motion model? It won't work with animatediff v3
try it with this example workflow: https://civitai.com/articles/6702%EF%BF%BCWith
@Cseti yes I followed your instructions.
@Cseti Just checked your workflow. I use this one: https://civitai.com/models/335070/simple-lcm-img2vid-workflow-or-comfyui-or-v3. Could it be the cause? Tested with yours and I have no motion either. Did not touch any settings.
@PinnochioProut which motion LoRA have you tried? the "Zooming in (32fr)" maybe? Try it with the ones with (16fr) in their name
@Cseti I tried only the 16fr. Its like it ignores Lora completly. Im using ComfyUI v0.2.2 btw.
@PinnochioProut I recommend checking all model loader nodes one by one. Sometimes it is replacing the models with wrong ones. I have no idea what else could it be. Also update everything to the latest.
@CsetiI did more testing in the afternoon and realized I get it to work with some images and not some others. So I believe the start image has an impact. I can't find out what causes it work sometimes or not though. So far thats the only commun point :/ I will make it work eventually. Thank you for assisting and trying to help!!!!
@PinnochioProut you can also play with the "motion scale" (you get more motion by increasing it), the "v3 adapter LoRA" strength (you get more motion by decreasing it).