This is my first attempt at a lora. Trying to train hunyuan video on exotic dance moves from one of my belly dance videos. Trying to slow down hip movement and add a little more exotic feel compared to existing dance loras. I hope you enjoy it. I haven't done a lot of testing with other loras. Lora was also trained with 'sexy dance, erotic dance, exotic dance, dance' keywords. Strength 0.6 to 0.8. has trouble keeping on clothes. Any higher starts to resemble training model used.
NOTE: if you do use any other 'dance' related words in your prompt, hunyuan tends to bring in other 'silly' dance behavior.
I used this process for local training:
https://civarchive.com/articles/9798/training-a-lora-for-hunyuan-video-on-windows
vers 1.0 wan - 20x50 frame clips, 160x160 pixels, 200 epochs. Trained with Wan2.1-T2V-14B, also works with I2V model. Might need to add 'dancing' to motivate the lora..
vers 2.0 - 27x65 frame clips, 256x256 pixels, 24 epochs. training step was 60% faster. better hip movement than v1.0. blurred faces for better lora interaction. added 3 additional models for variety. 2 models were full body. semi-verbose captioning. for some reason this vers gets a lot of spinning action.
vers 1.0 - 47x55 frame clips, 384x384 pixels, 10 epochs. training was pretty good learning body movement i was trying to capture.
Description
Initial version
FAQ
Comments (37)
I think you have a mismatch on your number of frames generated and your FPS on your video combine. That can cause that slow output on the final video.
that slow output was what i was training for. the other loras were too 'spazzy' for me. here's an ebsynth version of what i was trying to capture... https://civitai.com/images/50237696
double checked... everything is at 24fps.
any additional feedback or suggestions are appreciated. thx.
I see what you mean now, currently testing it. I'll drop some gallery videos when done! Good job!
@TheAIDoctor thx. hope you like it... :)
I love the idea of this Lora. I tried running this with various settings and having difficulty getting smooth movements. I get a lot of flickering of the clothing, fingers and hands. body movements are good but limbs/cloth goes crazy. Not sure if i just need to dial in settings or not but my success rate is lik 10% currently. I am uploading 2 videos, one with blue belly dancer outfit was only good result i got in t2v. I then took one in blue hair and ran it about 5 passes in v2v with various settings until i got a result i liked. Really cool when working but currently difficult to use unless i am using wrong settings it's success rate is low.
you can drag one of the videos. it should have all workflow settings. it's my first lora, so i may have over trained / under trained. see if you can reproduce one of my videos...
i dragged in your workflow. i didn't see trigger word in prompt 'b3ll13-d8nc3r'. am i missing something? if not, you get hunyuan spaz dancing... :)
@tedbiv Oh thanks. I thought i only needs words like sexy dance, erotic dance, exotic dance, dance from your description. didn't realize i needed that trigger word too. I can give it another try later
@zengrath those other words were used in training, but 'b3ll13-d8nc3r' will activate lora and not have hunyuan 'dance' content influence video.
@tedbiv I tried it again, same prompt but added trigger word before and at end of my prompt. just same exact situation as without trigger word. her hands are going absolutely crazy and i tried various flow shift values, guidance, i tried lora strength from .6 up to .9. The fingers and fans flicker like The Flash. haha. Sorry just not working out for me.
@zengrath yeah, i'm seeing same issue with your workflow. i just changed your lora blocks setting from double block to all. i don't have that setting... stay tuned.
@zengrath almost like my lora has no affect on the video?
@zengrath i just copied your prompt into my workflow and ran it. i'll post it shortly. it turned out really sexy... i'm going to go through the different settings to see what's going on. one thing i noticed is you use clip_l in clip1 name1. i use clip-vit-large-patch14. i'll keep comparing...
@zengrath posting your prompt my wf. not a great video, but i can see my lora, and the girl looks almost identical to the model i used.
@tedbiv No problem. Maybe it's something to do with something in my workflow not liking this lora. I like this workflow for it's flexibility and found settings that work on my PC for produce high quality videos. But i don't know. I did try change CLIP and your other suggestions. It still is very jittery and glitchy despite high resolution/steps, increasing flow shift sometimes helps but overall still jittery. I could try your workflow just to see if it works better. i just wanted to report it. but not a big deal. I'm just trying to learn as well. Rare with right combination of seed and prompt i get a result that isn't too bad, then when refined enough gives me a smoother video.
@zengrath i'll keep poking. i'll let you know if i find something.
@tedbiv No problem. I tried testing by using your workflow, didn't change a thing except put the trigger word and copied my prompt over just to keep test same. still incredibly jittery and flickering, like belt flickers all over vanishing and appearing and changing shape, clothes flicker, fingers and arms, etc. Maybe it's just real sensitive to the prompts i am using, i generated 10 prompts to try of various places and styles and most have same issue.
your work flow and mine are backwards from each other... :). i do 6 steps at 1.6 speed to determine if subject is good, then do 20 steps at 1.6 speed to finish. i found the biggest change to your flow would be resolution 400x720 and increase frames to >100 i can see my lora showing through. i trained on 384x384 videos but i wouldn't have expected it to manifest like that...
@zengrath you have to get a good stable video in the first window with 6 steps before going to the second stage of 20 steps. changing the seed till you get a good start. see my comment on your workflow resolution?
also try 1.6 speed instead of 2.1
@tedbiv tried all that. changed resolution, tried lowering steps to 6 or slightly higher, tried enabling and disabling fast lora on my workflow i am using, tried again playing with guide strength and flow shift. 1.6 speed on tea sampler. It's same issue, mainly of her looking light she's moving at 10x speed. Essentially I would have to try over and over and over again to get something usable which is even more difficult at lower res and steps since it gets even worst quality results. I actually had better success with my original settings and just being patient to generate a dozen times in first step before upscaling until i got a good one wort upscaling and refining and even then usually takes a V2V pass to refine it more. It's fine. i just saying, to me not worth headache of a lora that takes that much time to get a result that isn't completely broken. I've been using a ton of character lora's here to make various sample lora's and i get a good result 95% of time with my settings. I've tried some NSFW lora's and i usually get a okay result around 50% of time but those require good specific prompting to get higher. This one feels like 10% if i am lucky success and then takes a lot of refining on top of that. i don't think issue is on my end at this point but a difference in patience, where someone may be willing to generate 20 tries until get something usable. i'd rather not.
i'm currently retraining this lora. i have made some changes suggested by other creators. added 3 additional models. 2 clips of full body and 2 clips of 3/4 upper body. 4 different girls altogether. hopefully it will perform better... still learning.
@tedbiv Sounds good, i kinda like to see more dancing lora's and things like that so be cool to see what you come up with. Even popular tiktok dances or things like that would be fun to see. although perhaps there is some kind of control net situation that could do job in a vid to vid situation, i don't know.
@zengrath i'm trying to stick with more erotic/exotic dancing. more slow hip movement, lots of body/breast caressing. sort of a slow striptease kind of motions.
just uploaded vers 2.0 this image was with your workflow. i lowered height to 720 cause it pegged my card. https://civitai.com/images/60758306
Wow, that was hard to get working...
For those who want realism this LoRA works better with:
[blocks_type = double_blocks] from your LoRA Loader, if you try with [blocks_type = all] (Like me) will give you cartoonish all the time...
Oh! and guidance should be high (9.0+)
Thanks for the share and for the work on this!
What was the text you put in the dataset text file? As i'm attempting to make a dance "move" video lora for the first time, and having struggle to get the move itself. Visually, i get the character it was trained on, not so much the dance itself. Using 51 frames being 608x1080
i'm retraining it now. got some pointers from another creator. i'm using 27x72sec. original clips were 1024x1024. dataset.toml i set resolution to [244] script rounded up to [255], frame_buckets[1, 65]. now each step in training takes 10.5 secs. instead of 30secs. i tested a checkpoint at 12 epochs last night and the hip motion was really good. here's a sample of one of the text files. --- b3ll13-d8nc3r, erotic dancer, sexy dancer, exotic dancer, belly dancer, sensual dance, woman with a censored and blurred face, slender blonde woman with shoulder-length hair, wearing a brown crop top and a long, flowing blue skirt. barefoot on a white balcony, stretching her arms above her head, with her back to the camera. The background features a scenic view, 1girl, solo, dark skin, bracelet, midriff,
it seems less clips/lower resolution is faster and seems to train equally well. i'll find out when it's done. running 25 epochs.
used this tool on clip segments, then edited out crap i didn't care about training... https://civitai.com/models/1129181/hunyuan-video-lora-training-toolkit
here's another sample from a different clip/model --- b3ll13-d8nc3r, erotic dancer, sexy dancer, exotic dancer, belly dancer, sensual dance, woman with a censored and blurred face, nude, athletic woman with a slim, toned physique and light skin. She stands confidently with arms raised, hands clasped behind her head. Her breasts are small and perky, and she has a visible, neatly trimmed pubic area. The background is dark, with a single blue spotlight,1girl, solo, short hair, navel, jewelry, standing, nude, armpits, necklace, arm up, loli, nipples, female pubic hair,
@tedbiv Gotcha! And does the training work better if using regular old SDXL/1.5 keywords descriptors instead of describing the scene as if reading from a director's notes like hunyuan video requires the prompting to be?
What's the crap you didn't care about training? As i just trained with it stock, as i don't know what half the settings benefits/doesn't benefit with.
here's an example caption with what i would remove in [ ] --- str4p0n, 2girls, nude, black harness, strap-on, sex toy, dildo, object insertion, vaginal insertion, vaginal object insertion, anal object insertion, [This photograph captures an intimate moment between] two nude women on a bed. The foreground woman, with long brown hair, is positioned on top, gently holding the other woman's breasts. The woman on the bottom has [a slender build, pale skin,] and dark hair, lying on her back with her legs spread, exposing her genitals. [The background features white and purple bedding. A red "HUSH" watermark is visible in the lower right corner.,long hair, breasts, short hair, multiple girls, brown hair, black hair, navel, 2girls,] nipples, [closed eyes, nude,] small breasts, pussy, spread legs, on back, yuri, [vaginal, flat chest, completely nude, uncensored, makeup, grabbing,] grabbing another's breast, [letterboxed,] sex toy, [realistic, dildo, strap-on, open mouth,] dark skin, nipple stimulation, [nipple tweak, vaginal object insertion]
mostly background and scene descriptions that don't add to lora definition.
@Therma i think the sdxl style descriptions work better, unless you're describing an action/movement. i tried a lora with very verbose captions and i didn't like how it turned out. ymmv.