~Magic Wand for HunyuanVideo(and now with experimental MagicWan2.1)~
Heya! I've updated the Wan version with my much better training settings that I've developed! It should be /much/ better now and I improved the captions to support "wide shot/medium shot/medium close up/close up" like my other models do. From my tests the overall quality, motion and everything is improved significantly. If you have any issues with blurred faces or face out of frame please add "blurred face" or "face out of frame" to your negative! Enjoy! Training was exactly like my WanNipplePlay model: https://civarchive.com/models/1590451/both-hands-sensual-nipple-play-selfother?modelVersionId=1799759
Experimental MagicWan: Latest version "MagicWan" for Wan2.1 is up! It's experimental and my first attempt at training a LoRA for Wan. I used the same settings and dataset I'd used for 1.5 Hunyuan but lower res. By 3600 steps it was COOKED, but epoch 56 around 2000 steps seems usable though it can definitely be improved! Feedback is welcome and I will definitely be attempting to improve it. Oh and please let me know in the comments if there is any interest in a version for the 1.3B model, I don't use it personally but I'd be happy to train a version for it if it's desired!
MagicWand 1.5: New version is up! Featuring much better quality overall, it was trained on a 25% larger dataset at 8e-5 for 3600 steps. It can prompt for corded/cordless (specify either before the color but note that non-white corded models are very limited in dataset and in porn in general so ymmv there) and a bit better for the motion of the vibrator(can append either "vigorously rubbing it up and down" or "grinding herself against it/she is grinding herself against it" to the trigger, results may be seed dependent for grinding especially). For instance:
"a nude young woman is reclining on a bed while her girlfriend uses a cordless white MgcWndVb to stimulate her vagina, vigorously rubbing it up and down. She has messy blonde hair and large breasts. Her girlfriend has red hair and is wearing black lingerie. Their faces are close together and they are looking directly into each other's eyes as they share this intense moment of intimacy"
It should also handle prompts for couples better. I noticed that "her vagina is wet" doesn't work as well as it did in 1.0 though and instead... kinda makes her squirt a bit sometimes? More testing is needed there.
MagicWand 1.0:
Good vibrations are coming! Hey all, this is my second LoRA for HunyuanVideo and it turned out /really/ well in my opinion. It produces videos of women either using a magic wand style vibrator on themselves or having one used on them by another man/woman. It was trained at Rank 16 with blurred faces(captioned "blurred out face") so it should be highly compatible with your character LoRA. It learned the motion well enough you can often see the woman's vagina undulating with the vibrations of the wand!
Captions/Prompting:
"a nude woman is reclining on a kitchen chair with her legs spread while a man standing behind her uses a purple MgcWndVb to stimulate her vagina, rubbing it up and down. She has dark brown hair, small breasts, and her vagina is wet. He has gray hair and is wearing a dark gray shirt. Behind them is a large bed with a black and white painting hanging above it and on the left appears to be a sliding glass door and another painting"
"a nude woman with dark hair is using a black MgcWndVb to stimulate her vagina. She arches her back and bends forward as she begins having an orgasm. A tattoo is visible on her hip and her bellybutton is pierced."
The trigger should be "using a {black/white/purple/pink} MgcWndVb to stimulate her vagina." Several of the data points also showed the woman wet with arousal and were tagged "Her vagina is wet" so that's promptable too, as is the motion of the wand to a degree. It should be capable of creating solo female, female/female, and male/female scenes. The dataset was quite varied and included various positions, ages of women, ethnicities, aspect ratios etc so try it out I hoped to make it versatile!
Training notes:
I've learned that high resolutions are not needed for training Hunyuan(Sauce: https://civarchive.com/articles/11942/training-a-lora-the-right-way), at least for motion LoRAs. This LoRA was trained on 15 videos that were preprocessed to six seconds long at 24 fps. A simple python script was used to employ Yolov8X-face detection and then apply a heavy Gaussian blur to faces in order to maintain face agnostic behavior(captioned accordingly). These were then VAE encoded to a combination of 424x240@129f and 640x360@41f and trained in about 12 hours on my 4070TI Super with Musubi Tuner. LR was 1.2e-4 with a LoraPlus multiplier of 4 for 2400 steps using CAME optimizer and constant with warmup scheduler and 100 warmup steps.
Final Notes:
Ever used one of these things yourself or had one used on you? If not, you should try! It will break your mind in the very best way 😉
Oh also, I've got a few utility scripts I use when creating video datasets for like chunking up videos, normalizing frame rate, blurring faces, etc. Nothing special, don't expect too much; they are just simple CLI affairs but if you are like me and use a largely CLI workflow, they could be helpful: https://github.com/Sarania/videoprocessingscripts They've been written and tested on Linux but should work on Windows too. Depends on Python3, ffmpeg, opencv, and ultralytics. Feel free to use them or not I just thought they might be useful especially the yolov blurring one!
Description
This revision allows for prompting corded/cordless, allows better control of the vibrator motion, does better with couples(especially girl/girl), and is all around an improvement. It was trained at a lower LR of 8e-5 for 3600 steps.
Resolution: 1280x720 Architecture: hunyuan-video/lora
Network Dim/Rank: 16.0 Alpha: 1.0 dtype: BF16
Module: networks.lora : {'loraplus_lr_ratio': '4'}
Learning Rate (LR): 8e-05
Optimizer: came_pytorch.CAME.CAME(weight_decay=0.01,eps=(1e-30, 1e-16),betas=(0.9, 0.999, 0.9999))
Scheduler: constant_with_warmup Warmup steps: 100
Epoch: 30 Batches per epoch: 120 Gradient accumulation steps: 1
Timestep sampling: Shift Discrete Flow Shift: 7.0
FAQ
Comments (23)
seems to be working a lot better for me anywy, gj man keep em cuming!
I'm glad the new version is working better! It's still trained at the same res as I couldn't really find any high resolution images to include like I'd mentioned, but I improved my handling of the censored faces and captioning and added 5 new video clips to the original 15! Then I lowered the base LR to 8e-5 (still with LoraPlus 4x) and trained for 50% longer. I'm VERY pleased with the results!
Oh one thing I didn't mention in my other post about my workflow is, I tend to use hidden_state_skip_layer=1 in HunyuanVideoWrapper. I find that often the model tries to "over-interpret" complex prompts, for instance creating another person for every mention of "woman" even if I'm only referencing a woman I've already referenced, or just... taking the prompt a /wee/ bit too literally. But using skip layer helps "zoom out" it's understanding just a bit to a more natural level in my experience. I usually leave apply_final_norm to false, but if I'm having trouble getting strict enough prompt adherence I will switch it to True. Also if I'm writing a REALLY complex prompt I might even set skip layer to 2, which is actually the default in HVW but personally that's just a bit too "zoomed out" for me usually. I don't know a way to replicate this effect in native, using clip_skip nodes doesn't seem to achieve it.
In case you haven't tried this, I've been incorporating close-ups in a separate, fewer repeat, folder and captioning them clearly as 'part of the concept' and I feel like this has drastically improved detail rendering. food for thought :)
great work. do you mind uploading your relevant toml somewhere? would love to get a direct reference
I don't mind at all but specifically what are you wanting to see? Like my dataset.toml for the most recent version was:
[general]
caption_extension = ".txt"
enable_bucket = true
bucket_no_upscale = false
batch_size = 1
[[datasets]]
video_directory = "/home/blyss/projects/art/extra/dataset/MagicWand"
cache_directory = "/home/blyss/projects/art/extra/dataset/MagicWand/cache0"
num_repeats = 3
resolution = [424, 240]
target_frames = [129]
frame_extraction = "head"
[[datasets]]
video_directory = "/home/blyss/projects/art/extra/dataset/MagicWand"
cache_directory = "/home/blyss/projects/art/extra/dataset/MagicWand/cache1"
resolution = [640, 360]
target_frames = [41]
frame_extraction = "uniform"
frame_sample = 3
Anything else, or any questions, let me know!
@blyss thanks . I just wanted to full file because whilst you do give all the info in your description I'm always afraid that I'll miss a small setting somewhere . can you post your config.toml as well please?
@QualityControl config.toml? I'm not sure what you mean. If you mean like my training command, I use a custom shell script to wrap Musubi so I can't really provide that outright, but all the relevant info is in the version details box:
Resolution: 1280x720 Architecture: hunyuan-video/lora
Network Dim/Rank: 16.0 Alpha: 1.0 dtype: BF16
Module: networks.lora : {'loraplus_lr_ratio': '4'}
Learning Rate (LR): 8e-05
Optimizer: came_pytorch.CAME.CAME(weight_decay=0.01,eps=(1e-30, 1e-16),betas=(0.9, 0.999, 0.9999))
Scheduler: constant_with_warmup Warmup steps: 100
Epoch: 30 Batches per epoch: 120 Gradient accumulation steps: 1
Timestep sampling: Shift Discrete Flow Shift: 7.0
Otherwise I apologize I'm not familiar with a "config.toml"
@blyss ah. ok. I'm used to training with diffusion-pipe that has both a dataset.toml and a config.toml. . . just files that contain all these things you've posted already. Thanks for the info. I'm halfway through installing mitsubi trainer to try it as an alternative
@QualityControl Oh I forgot that's how diffusion-pipe was I haven't used it in a while. But just a heads up you'll prolly wanna use a higher LR in Musubi than you would in diffusion-pipe! This is because Musubi supports Network Alpha which is a way of modifying the LoRA weights during training to preserve very small changes, and the default settings for that will cause you to need a bit higher LR. Default in diffusion-pipe is 2e-5, I've found between 8e-5 to 1e-4 to be a good starting point in Musubi especially if you use LoraPlus: "--network_args loraplus_lr_ratio=4" to set a ratio of 4 for instance. This helps not only converge faster, but tighter: https://arxiv.org/abs/2402.12354
this would be magic on wan...d
Wan just got training in Musubi ( https://github.com/kohya-ss/musubi-tuner/commit/6abca83c68eb20b0fd55748ec985fae1b7146067 - 7 hours ago as of writing!) I'm personally still using Hunyuan for now as I tested Wan and it wasn't as much to my liking(not bad, just not better enough to justify the heavier nature of it), but I'm happy to train my existing datasets for it when I get the time! My poor 4070 TI Super doesn't ever rest lmao. In addition to generative artwork and training for that artwork I also do a lot of stuff locally with LLMs. But yeah when I get around to it for sure, shouldn't be too long!
@blyss From what I had read on diffuser-pipe, it took the same settings and prompts entirely. My hope is its as easy as just setting and forgetting...
Nevermind the reply I deleted; I mixed two comments in my head and replied to that LMAO! Yes I've read training them is VERY similar!
FYI I'm waiting for Musubi's Wan stuff to stabilize a bit before I try to train with that, I don't feel like fighting the early access issues XD I will do Wan versions though just give it a week or so!
There's your Magic Wan! This first version isn't perfect but it's usable. I'm just now getting started with training Wan so there will definitely be updates!
@blyss lets gooooo
@makiaevelio543 In my personal opinion, Hunyuan beats Wan for T2V the quality is similar but Hunyuan's embedded guidance is much lighter. Wan crushes for I2V though, Hunyuan doesn't hold a candle the results I've gotten from Wan I2V are /stunning/. I'm not sure about the feasibility of training the I2V model though I'm already really pushing my GPU to the limit to train Wan T2V. When I bought it 7 or so months ago after saving for about 9 months, I thought 16GB would be good for quite a while heh... then Flux happened, then Hunyuan, then Wan... thank goodness for block swap and 64GB sysram heh.
@blyss I appreciate how little body-horror I get out of WAN compared to Hunyuan, although the time per generation doesn't make it feel worthwhile if you've found a consistent prompt. I'm not sure about the I2V, but I have seen many of the loras here work on both -- so I wouldnt be surprised if this did as well. It works well in normal T2V!
@makiaevelio543 I don't get that much bodyhorror with Hunyuan but I do tend to keep my embedded CFG low like I max out around 10 and usually more like 6-8. Real CFG does usually produce tighter adherence to prompts and better trainability and I am seeing both of those effects with Wan. It's good that both exist and it's easy enough to support both, it's making the dataset/captions that's the rough part but once you've got one yeah.
@makiaevelio543 I've got my times per gen on Wan down to around 20 minutes for T2V at 480p resolutions, 81 frames /without/ teacache and with a full 50 steps. That's on par with my Hunyuan inference times! I'm doing it in a custom fork of Musubi Tuner that supports fp16 accumulation from pytorch nightly, along with kohya's combined implementation of fp8_fast and fp8_scaled and torch.compile. Normally fp8_fast can't be used without quality degradation according to kijai, but kohya discovered that if we use fp8_scaled as well it offsets that effect(mostly, there's still some small issues that can hopefully be ironed out. No fp8 fast for same gen = 26min) Oh and if you are bothered by the lower native framerate of 16 in Wan(as I was), utilize this https://github.com/kijai/ComfyUI-GIMM-VFI and be bothered no more! I use that to interpolate all my outputs to 32 fps and it's pretty much flawless at doing so IMO. Wan has grown on me, and there is hope to tame the beast!
Future request : PremiumBukkake Lora for hunyuan (will tip alot of buzz for this)
I'll be honest that's not one I'm personally likely to do! It's just not something I have any personal interest in so I wouldn't be able to collect a good dataset or know what good results were. My next models will likely be focused on some kind of kinky shenanigans but I've been working on upping my generation game for a bit first.