This Lora was trained on Japanese language audio clips of Kasumi Yoshizawa from Persona 5 Royal. I tried to include a wide range of her emotions in the dataset (shy, nervous, confident, relaxed, happy, excited, loving).
The dataset was captioned in English & transcribed in Japanese, examples below. All of the words spoken in the dataset are Japanese. You can give her English dialogue and she will say it, but I assume it will sound different from her official English voice actor.
Sample training captions & phrases from dataset:
p5yoshizawa, Yoshizawa is confidently saying "それでいつか、成長した私を先輩に見てほしいんです！"
p5yoshizawa, Yoshiawa is seriously saying "どうしても話しておかなきゃいけないことがあって"
p5yoshizawa, Yoshizawa from Persona 5 is dejectedly saying "先輩のおかげもあって最近は調子が戻ってきた気がしたんですけど"Description
FAQ
Comments (3)
这个需要lora吗？不是一个参考音频就能解决么？
Sure, you can use reference audio, but a Lora works better for a few reasons:
1. Reference audio is limited to 15s total across all samples, which might be too short. With a lora, I was able to train on 180+ seconds of speech data that covers a wide range of emotions.
2. Unless you use a refmod, reference audio needs to be re-computed into latent space for every generation, which costs time. A lora is already "compiled", so it is faster to use.
3. I find that the fl2va model produces higher quality output than the ref2va model, and you can't use reference audio with fl2va.
Awesome, thanks for sharing! Would love to see an English VO version as well