This LoRA model is in testing stage, it should not be anticipated to get highly amazing results.
This is my first LoRA.
Notes:
The dataset contains few features related to Lumi, thus this model may behaves not well in generating images related to it.
Entire dataset was tagged with only natural language and I did not test if tag-string prompt works well.
If there is any error, please let me know.
Description
Re-trained with 141 images.
Training parameters:
Max train steps：2000
Epochs：29
Batches per epoch：141
Batch size：1
Gradient accumulation steps：2
Effective batch size：2
Learning rate：5e-4
LR scheduler：cosine
LR warmup steps：100
LR cycles：1
Optimizer：AdaMuonLoRA
Max grad norm：1.0
Mixed precision：bf16
Loss type：l2 (MSE)
Weighting scheme：uniform
Min-SNR gamma：none
Noise offset：none
Multi-resolution noise：none
Timestep sampling：sigmoid, scale 1.2
Zero terminal SNR：no
Gradient checkpointing：Yes
Attention implementation：SDPA
Latent caching：to disk
Text-encoder output caching：Yes
VAE：chunk size 64, 2-D mode, cache disabled
Network (LoRA) parameters
Module：networks.lora_anima
Network dim：64
Network alpha：64
Network dropout：none
Rank dropout / module dropout：none
Weight-norm scaling：none
Train UNet only：Yes
Train text encoder：No
Train LLM adapter：No train_llm_adapter=False, llm_adapter_lr=0)
Trainable parameters：91,750,400 (560 matrices)