So a new model was released when I wasn't looking. Thanks for letting me know, you douche. It's really good. Really good, much better than 1.5. Must be the textile recoder thing. I've heard that those are good. Lately I've been using MiniMax Music, with Ace relegated to remixing (which MM can't do). But I am pleasantly surprised at the quality of the new model.
There is a full model and a turbo model (links in the WF). I'm not sure why, but the remix group does not work well with the full model, so use turbo for that. I have to fiddle with the settings to see what it doesn't like.
Steps+CFG are handled by a Turbo? switch in the loader subgraph. This toggle does not switch the models, that would just be an annoyance if you don't want to download both of them. I know how you hate red nodes. Turbo is the default.
I ditched the calculation logic - if you go over to the MM music template and punch in your lyrics- it will spit out a song length that fits them (if it's within the cap you set). That works much better for finding the optimal length.
The remix/cover section is totally redone - I've imported my cover workflow as a group. Uses ROTI nodes, which you should already have. If you don't there is definitely something wrong with you.
If rgthree is giving your comfy front end fits, you can delete the bypasser and manually bypass the unused mode group.
Old v2 description:
So the new model is here, 1.5 Turbo. It's pretty great. It has its idiosyncrasies, like the previous version, they're just different. The very latest comfyui update has native Ace-Step nodes, so all you really need is to update comfy. I've got a bunch of custom nodes in the new workflow, but none of them are required for the actual encoding/sampling. It's all logic. I could not switch the latents in the old version, but I can with these, so there is a switch for toggling between new empty latent and inpainting (remixing). There is also a switch if you want to compare models. The AIO is a checkpoint, and goes just where you think it does. The other way to do it is with the diffusion model, two text encoders (dual-clip loader) and a VAE. If you just want one method, delete the loaders you don't need and the switch.
The new encoding node has many more features. Since it now has a BPM widget, figuring out the appropriate word count can be tricky. So I automated all of that annoying stuff. Just enter your duration and your BPM and after running you will get the calculations that tell you (based on your lyrics and the widget values you set) the total number of bars in the song, what the optimal word count is, what your actual count is, and the difference between the two. This can help you dial it in. A lot of artifacts people get are coming from a mismatch between their duration and the prompt. It's not the model- it's you. You suck, stop it.
I included a slider in the logic section to adjust the vocal speed as part of the calculated values. For example, a really fast rapper vs a melismatic diva.
A note on the time signature - because the input to the signature is a combo, I had difficulty connecting it to the logic (and it most definitely needs to be) so there is a separate INT node for the logic. If you change the time signature on the encoder, make sure to change it on the INT node. It's right next to it. This will ensure that your calculations are correct.
If you don't have any experience with TTS and have no idea what temperature, p and k values are, don't mess with them too much. It's a bit like CFG, they target specific ranges and limits on how much freedom the model has. Best bet is to leave them alone, CFG too, and focus mainly on the prompting. The model is now better at natural-language prompting in the style section. Don't go by my default text, that's just carried over from the previous workflow. It can be written out descriptively now instead of being parsed into simple tokens.
Euler works pretty good as a base. Others work as well, dpmpp_2m gives interesting results. Just be warned that it has no idea what a key is. Your output will be a multimodal wet dream. It's pretty cool if that effect is what you want.
Original v1 description:
This is pretty much Ace-Step right out of the box. It's there in the templates. I post this what for to remind you that it exists. Silent movies are boring. And none of you even have the decency to include intertitles. Admittedly, they'd have to be mostly onomatopoeia, but you could try. Here is a friendly reminder that you can make awful music just as easily as you make awful pictures. Easier actually. Way, way easier. Just use it. You don't even need to download this workflow. It's right there. End of PSA.
Description
There can be only one.
FAQ
Comments (8)
I'm getting this error "KSampler
Tensors must have same number of dimensions: got 3 and 4". Standard wf works.
Can you be more specific? Are you using inpainting? Dimension errors happen when there is a mismatch between what the model expects and what you're feeding it. Like errors that happen when you have an alpha channel and try to denoise the image- it doesn't know what the hell to do with the extra tensor. Does it happen with the template from comfyui as well? If you tell me the state of the settings that differ from the default state I can give you a clear answer.
Hello, I get the error if I try to run the workflow with all the default settings, the only think I did was selecting the model in the load checkpoint node, as the name was slightly different, maybe I was using the wrong model? Mine is is called "ace_step_1.5_turbo_aio.safetensors" ,no impainting. My system is 3080ti 12Gb VRAM and 32Gb RAM. Here is the full log https://pastebin.com/W9f8vkVR
@caligola99 Ah, there you go. You've got the brand new model. I didn't know it was out, looks like it's only a few days old. The entire architecture is different, text encoding, DiT, all of it. I'll have to check it out, I'll post a build if it's any good. If you want to use this setup you will need the 1.3 model. https://huggingface.co/ACE-Step/ACE-Step-v1-3.5B/tree/main/ace_step_transformer. Rename the safetensors file to 1.3 so you know what the hell it is. Thanks for bringing the new model to my attention, it looks awesome. Hopefully it sounds awesome too.
Yeah sorry for wasting your time, should've checked the workflow release date, it was a few days before 1.5 got released. I tried with the 1.5 with the standard workflow and my results are...not that great to be honest, could be a prompt skill issue but what I find really bad is the actual quality of the sound, it's like they have bass loudness pumped to the max, very unpleasant to the ear.
No worries, I know how it goes. This stuff can be really, really confusing even after you've been using it for years, and diving in to new stuff is just the same headache all over again. In my WF, that horrible overdriving clipping vocal effect is alleviated with the Tone Map Reinhold node. Not sure how it's implemented in the newer one, but I've definitely had that happen. Usually when the vocals are male. That's why I set it to .75 in the default. You also have to be very careful about the duration you set and the word count of the lyrics. If you try to squash too many words in, this effect loves to show up. And if you have too little text, you get a lot of repeats and garbled nonsense. You get a feel for it after a while. When it works, it's really, really, really good.
@caligola99 Posted my take on the new model. Works great for me. Haven't experienced the crunchy vocals at all. The node that will address it if it does show up is now 'Aura Flow.'
I tried the one "For Pussies" as I already had the all in one model, the quality seems already better than the default workflow, this model has huge potential for sure, I really hope this gains some momentum, I’d love to see plenty of Loras to play with.