<My first lora on here.>
Works well with I2V or T2V.
I trained on the movie clips (36 clips trained at 768 res from cropped 4K footage) as well as still frames and some AI images for the...ones that weren't in the films.
Say: "wearing a blue police uniform"
"with floppy ears" will help fix that if you get 4 ears or if you prompt her wearing a hat.
"wearing a blue hat with orange vest"
*rank 64. 7000 steps
Wan2GP was used for all examples generated with Distilled LTX2.3.
Description
3000 steps trained
I recommend https://civitai.com/models/533302/judy-hopps-on-model for start images
FAQ
Comments (6)
Thats incredible to get results like that in only 3000 steps. Can you share all your AI toolkit settings - copy paste the config file and community can start producing more well-trained loras like this.
Also the images and number included in the run, seems like they made a huge difference.
This is basically the important parts. videos trained at 768 res in Ostris AI Toolkit. if you want the workflow I used, watch Ostris's newest LTX2.3 guide on YouTube. I used his exact technique. The AI Toolkit is his creation. content_or_style says weighted now but before I used high noise for the first 3000 steps. I am continuing now with balanced.
_____________________________________________
"type": "diffusion_trainer",
"training_folder": "C:\\ai-toolkit\\output",
"sqlite_db_path": "C:\\ai-toolkit\\aitk_db.db",
"device": "cuda",
"trigger_word": null,
"performance_log_every": 10,
"network": {
"type": "lora",
"linear": 64,
"linear_alpha": 64,
"conv": 16,
"conv_alpha": 16,
"lokr_full_rank": true,
"lokr_factor": -1,
"network_kwargs": {
"ignore_if_contains": []
}
"batch_size": 1,
"bypass_guidance_embedding": false,
"steps": 6000,
"gradient_accumulation": 1,
"train_unet": true,
"train_text_encoder": false,
"gradient_checkpointing": true,
"noise_scheduler": "flowmatch",
"optimizer": "adamw8bit",
"timestep_type": "weighted",
"content_or_style": "balanced",
"optimizer_params": {
"weight_decay": 0.0001
},
"unload_text_encoder": false,
"cache_text_embeddings": true,
"lr": 0.0001,
"ema_config": {
"use_ema": false,
"ema_decay": 0.99
I've been playing with settings for awhile with test loras and LTX pretty much requires rank 64+ unlike other models. It performs MUCH worse at rank 32 or less. That is probably most of people's issues with it.
@Ada321 rank 64 gives so much better detail. But most of my issues were bad captioning techniques. At first I couldn't get any results here without also including CGI anthropomorphic rabbit in addition to the name. Finally got good captioning
Looks amazing, thanks.
Did you use a standard template or could you share your Workflow please ? :)
for this I used the same thing Ostris did in his YouTube video training using AI Toolkit for LTX 2.3.
3000 steps using weighted timestep at high noise, then 3000 steps at balanced noise.