2 test loras for CogVideo for 1000 and 2000 training steps
Training with default settings without test videos using https://github.com/a-r-r-o-w/cogvideox-factory
Training each 1000 steps took about 2 hours
These LoRa can be used with the CogVideoX-5B-I2V video generation model (For example, you can use comfyui https://github.com/kijai/ComfyUI-CogVideoXWrapper)
Description
FAQ
Comments (12)
Wonderful
its only for comfy? or can we do it also in SD Forge XL
So far mostly only for ComfyUI(Maybe someone will make another interface for video models in the future). ComfyUI is not so difficult to use. Basically you can just download ComfyUI and ComfyUI-Manager(With the help of manager you can download nodes that are not enough to start workflow). In general, I advise you to study this interface, because it is usually the fastest way to get more or less convenient interfaces for neural networks.
Example startup scenario:
1. Download ComfyUI
2. Install ComfyUI-Manager
3. Download the standard workflow for I2V generation with CogVideo (https://github.com/kijai/ComfyUI-CogVideoXWrapper/blob/main/examples/cogvideox_I2V_example_01.json).
4. Install the required dependencies
5. Place LoRa in the path ComfyUI\models\CogVideo\loras
6. Connect the CogVideo LoraSelect module to the model loader and select the desired LoRa
To run CogVideo-5B you need a video card with 12~16GB VRAM minimum (Ideally 24GB+).
I myself use 4070 ti super with 16 GB Vram and with fp8 and 50 steps generation takes about 10 minutes
Oh and yes if anyone wants to run this LoRa, here is an example of the hint I usually use:
positive:
Paizuri. In the anime style video, a playful young woman looks at the viewer. She confidently holds her full, round breasts together, emphasizing their size and prominent dark pink nipples. Between her breasts, she squeezes a long penis, rubbing it between her breasts in a teasing display of paizuri. Live2D.
negative:
The video is not of a high quality, it has a low resolution. Watermark present in each frame. Strange motion trajectory. Static image. Static video. Slow-mo. Camera movement. Blur. Change of frame.
Working very fine! Thanks for it!
Looking forward for more Loras or tips on how you performed the training!
I am now building a dataset for LoRa that will generate "shikairo days dance (meme)".
Basically I used the default cogvideox-factory settings (Changed paths and turned off test videos for faster learning).
This LoRa used 32 videos with a frame rate of 8 and a total of 49 frames
Initially I wanted to train the model for 3000 steps but some circumstances happened that made me stop at 2000 steps. As I have already written on H100 one thousand steps is trained for about 2 hours.
In theory you can also train on 4090 or 3090 (with 24 GB VRAM) but I'm not sure what to do for that. Maybe the default settings are already designed for this but I'm not sure.
@Cixiao I might give it a try with my 3090. It'll take a while for sure. The most difficult step is gathering a quality dataset probably, not sure if editing with After Effects or Premiere would be the way to go.
I've uploaded some tests using this Paizuri Lora but Civitai doesn't allow me to link the resource :(
Its awesome!
How much vram is required to generate a video like this, and how long it will take on 10gb for example?
And how hard it is to train such loras?
Im a noob but out of my 16gb it generally uses around 12-23gb but there's ways to lower it.
if I use the default 240p resolution or whatever it takes about 1 minute to generate on my 4080s but then using 768x1024 can take upto 10 minutes (once again im a noob)
I wish to have a lora like this but for idle animation, you know like just standing and slight movements of clothes and hair etc, and blinking
I couldn't figure out how to get anything running since I'm incompetent and my specs are horrible but this looks really good
Hello, one question, what would be the ideal configuration for a 12GB 3060?