(LTX-2 Lora) - Rally (Race) Driver POV (Video)
Please drive responsible, wear a seatbelt and follow your local laws and rules.
Create GoPro / POV videos of (real) Rally Driving.
Use "r4llyp0v" as a prompt word
Use Lora Strength (!)
T2v around 0.87
I2V around 0.65
Works best when using T2V on LTX-2 DEV (FP8), but should be feasible for I2V too
Pro tip: Capture yourself in your car from a POV perspective :)
Just note that it will require or force a round steering wheel
Trained on 640x360, 832x480 and 1280x720 - 7 seconds videos
Seems to work for clips up to 30 seconds (see preview videos)
Feel free to post your results!
Read more below for prompt examples
Disclaimer: This is my first LTX-2 Lora and may not properly follow turns as prompted. It's just a concept lora. Speedometers may not properly update.
Prompt example:
r4llyp0v, guardrails, forest, rainy, co-driver, windshield wipers wiping, fast driving, mud, Scottish voice accent
Timestamp 00:00 - 00:10
Hands wearing handgloves are placed on the steering wheel.
Speech "Launch! Okay...... let me check my notes.. Right six!"
Timestamp 00:10 - 00:20
Taking a sudden right turn.
Speech "Okay so, be careful in this stage, we don't want any incidents like the other teams have"
Timestamp 00:20 - 00:30
A left turn is coming up.
Speech: "Fifty, don't cut. Danger up ahead. One hundred yards, right six into left four over crest"Was trained on captions like:
r4llyp0v, left right turn, forest, speech "sixty ten left clip after in ninety flat nine right cant forty nine left plus over crest"r4llyp0v, forest, people, village, co-driver, right turnDescription
Initial version
FAQ
Comments (11)
Do one each for
-> motorcycle
-> speedboat
-> drone
Thanks for your suggestion. Aside from not being trained on those for T2V, technically, you probably could do I2V with a motorcycle and speedboat (although it may transform a steering wheel).
(look at examplew below)
Drone is one I will probably look into ;)
@JonXL Alright. Please do so with adjustable speed and ground level distance maybe? This is going to be very useful to construct maps and model cities.
@aungkhant0911 I'll try to keep adjustments in mind when gathering the dataset and captions, note however that LTX-2 lora training is brand new and LTX-2 has it's limits.
Man, this is awesome. Now we just need a Formula 1 LoRa
Interesting idea!
Although it tries to force a round steering wheel, you could try using this with I2V with an image of a F1 POV car. (maybe playing with lora strength could keep the original wheel)
I have so many cool ideas I'd like to make lora's for, stay tuned :)
holy shit thats RAC rally