Hello,
I would like to share with my latest experiment, My goal was to see if it was possible to create a style lora for Hunyuan video that will apply a better cinematic style to my videos. Even though Hunyuan creates excellent videos it still have difficulties creating that gritty classic movie style we all enjoy.
This lora was trained on a selection of movie stills from the late 70s to 90s, it tends to want to create more talkative humans though, and lacks a lot of flow for more action driven prompts but it does the job if you want deeper colors and realistic humans.
This lora also isn't great at doing non-humans aka monsters, beasts, robots, it will just make them human. So lower the strength to [0.6 - 0.7] if you want a better range of characters
The demo video was made with my trusty but slow 4060TI, I didn't cherry pick the best generations, because I like showing the flaws a Lora can have, I wrote a short descriptive prompt, generated the videos and put them in my sequence, also the video resolution is 768x384, running 69 frames for speed, because I'm GPU poor so I'm curious what you peeps will be able to run on faster cards.
The training dataset was around 280 images, trained for 50 epochs (18K+ steps) on a cloud based L40 GPU.
Trigger word:
cinematik
Example prompt:
-- Marcus, a phisically lanky 30 years old male with short buzz haircut, belgian facial features, dark military jacket, gree camo pants, walking trough the street of a dilapidate serbian warzone, cinematik, cinematic 8k high quality video
-- Roberta, 35 years old nigerian woman, with hair hidden under a ninja mask, she's overweight lady wearing a black ninja suit, she's moving through shadows, her stealth is evident as she blends perfectly with the background, the background a bank with the vault door openue short, cinematik, cinematic 8k high quality video
-- Johan, a 20 year old male with blonde hair, african descent and congolese facial features, black tshirt and blue short, he's setting fire to a wooden hut in his african village at night, the flames are rising up to the sky, cinematik, cinematic 8k high quality video
-- Duke, a cybernetic policeman, wearing a tight leather tron like suit, he is wearing a black reflective helmets, he is riding through cyberpunk building at very high speed, the lights of the streets and neon signs is reflected, the camera is panning toward his closed helmet visor, cinematik, cinematic 8k high quality video
Description
FAQ
Comments (11)
Great! Tell me more about the training.
What would you like to know? I made it using diffusion pipe
@Drno amazing lora! Can tell what sort of captioning you used for the training images. Thanks
This is an interesting concept. I have been experimenting with style and I have also experienced more "talkative" models, wonder what causes this?
When experimenting with this, does it need to be in 16:9 or is it trained on different aspect ratios as well?
@Suppressor I still wonder why they are still talkative,
I also haven't tested any other formats than 16;9, I'll run some tests now
@Suppressor Hey, I've been testing 9:16 format videos, and it works quite well
@Drno Yeah, it's strange when they do that.
@Drno Good deal, I am excited to work with it this weekend.
unfortunately, this lora seems to generate text over the scene like subtitles or credits which tend to spoil otherwise nice sequences
Hey sorry to hear that, I have generated around 100+ videos with it and have yet to see any text of any kind, could you share the prompt that leads to this issue?
Tip, try to keep the prompt simple,
example:
Nosferatu prince of hell his body is in direct sunlight and it is melting away due to contact with the sun, his body explodes and is vaporized,cinematik, cinematic 8k high quality video,