Prompt example:
csetiarcane animation style. The scene reveals a woman lounging on a sunlit balcony, her form bathed in the warm glow of the setting sun. She wears a white, off-shoulder blouse that falls loosely, revealing one bare shoulder and collarbone, paired with a flowing skirt that drapes over her thighs. Her long, wavy blonde hair catches the sunlight, creating a halo effect as she tilts her head back and sips from a glass of wine. The soft breeze teases her hair and the hem of her skirt as she gazes dreamily toward the horizon.
ID Token / Trigger word(s):
Use this in your prompt helps providing the style.
csetiarcane animation style
Trainig details:
It was trained on images and videos
LR: 2e-5
Optimizer: adamw
epochs: 20
steps: 6660
dataset: 135 videos and 135 images
repeats: 5
rank: 128
batch size: 1
gradient accumulation steps: 4
Important Notes:
This LoRA is created as part of a fan project for research purposes only and is not intended for commercial use. It is based on the TV series called Arcane which are protected by copyright. Users utilize the model at their own risk. Users are obligated to comply with copyright laws and applicable regulations. The model has been developed for non-commercial purposes, and it is not my intention to infringe on any copyright. I assume no responsibility for any damages or legal consequences arising from the use of the model.
Acknowledgment:
Thanks to the Tencent team for making this great model available
Thanks to tdrussel for the diffusion-pipe that helps us making these LoRAs.
Thanks to Kijai for his great ComfyUI integration
Thanks to POM for providing the computing resources.
Description
FAQ
Comments (27)
why is the lora so big :o
Because of the rank value that is 128
@Cseti Ooooer thank u, is there a way to do a smaller rank training or does it decrease the quality a lot? I do 3D art and I'm interested in doing a style LoRA when Kohya finishes up the training thing.
@thefoodmage you can use lower rank but will need more time to train. My Jinx lora was trained with rank 32 and turned out quite well but needed around 2 days of training
@Cseti 2 days? what are you training with? the few characters lora's ive done, one shared here, trained in 45 minutes on rank 64
@mrreclusive3545 45 minutes wow. maybe with images only? The Jinx lora was trained only on videos and if that is the case it take much more time. This one was trained on both images and videos so needed less time. I never tried image-only training as I found them too static. Also when I tried lower LR (like 1e-4) to speed up the training I felt like the the quality was much worse.
@Cseti yeah images only, don't have much static issue as I use the block editor in the hunyuan video wrapper, with only double blocks 10 through 19 it works flawlessly, I've been working on video training as well, and yes. those take a butt load of time, but thats for motion and not characters.
@Cseti Hold on, is higher rank faster to train? :)
@illinar afaik higher rank can affect learning speed.
@Cseti By increasing it, as far as I know. That's why it is much slower to fine tune the whole model than it is a small part of it.
@illinar higher rank indirectly speed up the training as it defines how many parallel information the model can handle. So basically it wont increase the training speed (s/it) but increases how much data the model will handle/store so most probably you will get better results at lower iterations but also the model will overfit sooner.
@Cseti Are you planning to retrain it at a lower rank to give users options for lower weight/quality? I'd be curious to see the differences
Did this fit on 24 GB VRAM? I think not. Good job!
You mean the training? If you lower the rank to 64 i think it would
@Cseti I only got to 32 Rank with 512 x 512 max pixeles (sum to 262K max) with 23 or so.
@LDWorksDavid I'm training 768x768 with images only with 12gb following this guide https://civitai.com/articles/10335
Now sure how good the results are yet, one comment said diffusion pipe is better than this hmmmm
@LDWorksDavid i used videos with bucket of 256 and images with 1024. It used 26 gb vram with rank 128 so i think it would fit into 24 with rank 64
@Cseti using diffusion pipe? anyway cool idea, I think this lora is a beginning of great things, finally seeing some more variety of loras now edit: ah yeah I see the diffusion pipe part you wrote
@Redbird I use diffusion pipe with video shots for train yup.
@Cseti Hello! what video length do you use? There's a frame_buckets settings for video in diffusion pipe I don't understant
@jfsarazin933 If I remember correctly, I used clips between 33-81 frames. Frame_buckets means that it will cut your videos to that frame count. For example, if frame_bucket=[33, 81], then it will shorten your videos between 33-81 frames to 33 frames, and videos above 81 frames to 81 frames.
@Cseti Thanks for your answer! So if I've already cut my videos to show only one quick action (max 80 frames) I should not use this feature? If Don't want to use it, I set it to [1000] in example?
@jfsarazin933 If you set it to 1000 it will drop all your videos. The numbers in the frame bucket need to be smaller or equal than your videos' frame count. It will cut them to the closest value in the bucket. If I remember correctly, you can specify values in the bucket that match '4k+1'.
@Cseti and if we used different number of frames videos, we will need to put the lowest one. Right?
@LDWorksDavid if you put the lowers one then all of your videos will be cut to that frame length. You can put multiple values there so you can put all of your videos' frame length like this [33, 49, 81, 161]. in this case the vides longer than 161 frames will be cut to 161 frames, the ones between 81 and 161 will be cut to 81 and so on.
@Cseti My bad, I meant the highest of the frame lenght. Then for ex. If I have 20 videos should I put every frame lenght on the frame bucket? Or just the get the highest one? Or what do you recommend here?
BTW how much was the resolution you used for the videos?
Please make style of Theobrobine animations. His 2s animations is very nice and fluid, high quality
Details
Available On (1 platform)
Same model published on other platforms. May have additional downloads or version variants.