The Workflow was setup to have a clean "GUI" showing only parameters that matter, so you might want to toggle off Link visibility, like in above screenshot.
Features of this workflow:
- Standard song creation (Text to Song) or Cover song creation with input song as reference (reference to cover-song)
- Tags creation with Ollama or Style Presets or own tags
- Lyrics generation with either:
Ollama
Re-writer (rewrites Lyrics to match the singing structure of the input lyrics)
Song Analyzer (analyses Input Song and extracts Lyrics as text)
Own lyrics
- creates matching Song/filename with Ollama
download files:
https://huggingface.co/Comfy-Org/YuE2/tree/main
recommended Ollama models for Tags/Lyrics/Songname/Re-write:
https://ollama.com/4skl/gemma4-e4b-mtp
for Tags with references to existing artist/bands/songs and for Re-write lyrics:
https://ollama.com/kwmcglon/gemma4-31b-abliterated
Save Location:
📂 ComfyUI/
└── 📂 models/
├── 📂 audio_encoders/
│ └── sheetsage2_bf16.safetensors
└── 📂 checkpoints/
└── yue2_3b_bf16.safetensors
Custom nodes used:
Ollama help:
Install Ollama from https://ollama.com/
download a model: Go to a model page, chose a model , then hit the copy button, i.e. https://ollama.com/mirage335/Llama-3-NeuralDaredevil-8B-abliterated-virtuoso
open terminal and paste the model name, i.e.: ollama run huihui_ai/qwen3-vl-abliterated
model will be downloaded and can be selected in green comfy node "Ollama Connectivity". Hit "Reconnect" to refresh.
Description
Yue2 Text/Reference to Song or Cover
FAQ
Comments (5)
Some Infos/Tips:
Parameters worth trying to play with:
- Switch ABC Planning Off (in Yu2 Standard group)
- toggle mode between "full" and "melody" in Yue2Generate Music" node within processing
- change CFG in ksampler node within processing
In case you use the Re-Write lyric feature and face slow processing afterwards, change the paramater "keep_alive" in green "ollama connectivity node" to zero, so Ollama releases Vram after use.
Consider the Song Analyzer (to extract lyrics from audio) to download a model (about 1.7gb) on first start .
update comfyui to >= 0.37, as they implemented some bugfix related to cover song behaviour (sheetsage2 Audioencoder)
to use the SongAnalyzer to generate lyrics from audio, it is required to install faster whisper, from you python embeded folder use this command.
python_embeded\python.exe -m pip install faster-whisper(see also github repo: https://github.com/TheLocalLab/ComfyUI-SongScribe )
So far I've tried 6 workflows I've found to use Yue2 and yours is the best, I'm using it with gemma-4-12B-it-heretic-GGUF and it works well too, will need further testing to compare it to the one your recommend. I havent tried/downloaded the lora you used, what is it for? Do you know about the instrumental lora? https://huggingface.co/Mothersuperior/YuE2-instrumental-cot-full-loras Can we use it with this workflow or how do we implement it?
@Sumifuha thanks and glad it works for you.
Your gemma4 12b should work nearly the same as the gemm4-e4b I have recommended for that tasks, but for me it was faster. The large gemma4-31b has more knowledge about actual songs/bands/artist and delivers better tags if you reference them, but it is slower.
The instrumental lora you have linked was the first one I´ve tested.
For insturmental: Write "[Instrumental] in lyrics and use Ollama (lyrics) to create instrumental tags (or add own), otherwise output might be very repetitive and boring. Load that instrumental Lora with clip strength 0.5-0.75 into one of the 2 lora nodes. (try with Lora on/off).
Tbh the model is not very creative on instrumentals, guess it wants lyrics to sing along :).
@tremolo28 Thank you so much for your fast reply and guidance I'm having so much fun making music, Ive been waiting for something like this. =)
@Sumifuha If you get tired from Yue2, try my AceStep workflow, with Loras avail. I think its a bit more versatile.