<My first lora on here.>
Works well with I2V or T2V.
I recommend using strength 1;0.7 if you want to get more generalization (less random bunnies in the crowds, or to have other voices other than just hers)
Version 2.0:
this version has been trained with regularization--no more rabbit clones! you can utilize more characters or scenes than before. it works well with some loras, but not as many as my last version. (crank up the strength and it sometimes helps with poor style lora combinations. works well with the fight lora :) )
must use "jhp_rbt anthropomorphic rabbit"
actually, "Judy Hopps CGI anthropomorphic gray rabbit" works too
Version 1.5:
I trained on fewer clips (all movie based, no AI) than before to get Nick the eff out of my lora. Rank 64 stepped down to 48. 2800 steps using a very good new method. (I ditched AI images because they hurt the likeness. You can make her nude other ways probably. Mostly I2V)
Version 1.0:
also movie only trained, no AI, for 7000 steps using an older method.
You might see some random Nicks in there.
Version 0.9/0.5:
Trained with movie clips and AI nudes. Works with I2V and T2V. Just prompt her to be nude and it should work, otherwise her iconic outfits should work.
Say: "wearing a blue police uniform"
"with floppy ears" will help fix that if you get 4 ears or if you prompt her wearing a hat.
"wearing a blue hat with orange vest"
new training method used from this post:
Wan2GP was used for all examples generated with Distilled LTX2.3.
Description
3000 steps trained
I recommend https://civitai.com/models/533302/judy-hopps-on-model for start images
FAQ
Comments (10)
oh yeah noticing the clips that weren't in the movie, looking good
I used Judy Hopps (On Model) to make I2V start images. That is the best one I could find. great quality. I had linked it but the link was removed from my post for some reason?
the animation looks pretty smooth, are you generating on 24fps or 50fps?
24 fps. I generated the I2V using Distilled in Wan2GP
@kronos1959777 that's interesting, what sampler do you use and what resolution. thanks for sharing.
@tazmannner379 sampler might be euler? That is the default for dev model at least in wan2gp. Only sampled these at 720p. I will do 1080p for my next checkpoint to show off the detail better. Some of those examples are i2v using the referenced lora (NoobAI lora plus yiffymix checkpoint i think, see the link) to get start images.
Thats incredible to get results like that in only 3000 steps. Can you share all your AI toolkit settings - copy paste the config file and community can start producing more well-trained loras like this.
Also the images and number included in the run, seems like they made a huge difference.
This is basically the important parts. videos trained at 768 res in Ostris AI Toolkit. if you want the workflow I used, watch Ostris's newest LTX2.3 guide on YouTube. I used his exact technique. The AI Toolkit is his creation. content_or_style says weighted now but before I used high noise for the first 3000 steps. I am continuing now with balanced.
_____________________________________________
"type": "diffusion_trainer",
"training_folder": "C:\\ai-toolkit\\output",
"sqlite_db_path": "C:\\ai-toolkit\\aitk_db.db",
"device": "cuda",
"trigger_word": null,
"performance_log_every": 10,
"network": {
"type": "lora",
"linear": 64,
"linear_alpha": 64,
"conv": 16,
"conv_alpha": 16,
"lokr_full_rank": true,
"lokr_factor": -1,
"network_kwargs": {
"ignore_if_contains": []
}
"batch_size": 1,
"bypass_guidance_embedding": false,
"steps": 6000,
"gradient_accumulation": 1,
"train_unet": true,
"train_text_encoder": false,
"gradient_checkpointing": true,
"noise_scheduler": "flowmatch",
"optimizer": "adamw8bit",
"timestep_type": "weighted",
"content_or_style": "balanced",
"optimizer_params": {
"weight_decay": 0.0001
},
"unload_text_encoder": false,
"cache_text_embeddings": true,
"lr": 0.0001,
"ema_config": {
"use_ema": false,
"ema_decay": 0.99
I've been playing with settings for awhile with test loras and LTX pretty much requires rank 64+ unlike other models. It performs MUCH worse at rank 32 or less. That is probably most of people's issues with it.
@Ada321 rank 64 gives so much better detail. But most of my issues were bad captioning techniques. At first I couldn't get any results here without also including CGI anthropomorphic rabbit in addition to the name. Finally got good captioning