The Workflow was setup to have a clean "GUI" showing only parameters that matter, so you might want to toggle off Link visibility, like in above screenshot.
V1.8 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama. Text to Audio/Song (examples below).
small update on Lora handling to better include Lora trigger words/phrases
Check this link with 1.5.Turbo XL Loras by Ryanontheinside: https://huggingface.co/ryanontheinside/models
1.5 XL Loras can be loaded on any XL Slot (Turbo, SFT or merges), it improve the sound very well, like more authentic guitars for rock&metal, etc. Also song structures follow the genre better.
Loras require a trigger word that can be found on the main page per Lora of above link. Use the trigger word in the new node "Pretext" within the Input section of the workflow, added around 25 of those triggers to a Note in the WF for copy&paste.
See post in discussion below for more info.
Key features:
Can use any Song, Artist as reference or any other description to generate tags and lyrics.
Will output up to 4 songs, each by Turbo, SFT, Turbo XL and SFT XL model (or any merge or Base model).
Keyscales, bpm and song duration can be randomized.
able to use dynamic prompts.
creates suitable songtitle and filenames with Ollama.
saves songs as MP3 with tags like lyrics, artist, genre, etc.
V1.7 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama. Text to Audio/Song.
replaced the "save Mp3" nodes to allow ID3 tags to include data like artist name, bpm, genre, lyrics, etc.
requires custom node: https://github.com/mattjohnpowell/comfyui-audio-expo
added a feature to save all relevant data (tags, lyrics, etc.) as a separate text (.txt) file
audio render processing remains unchanged to previous version
V1.6 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama.
updated the settings for XL models and added a 3rd System Prompt for tags to chose, with more descriptive song descriptions.
1.5 XL SFT pipeline now has an "Adaptive Projected Guidance" node and negative prompt.
** See below some tips which model and settings to start with.
V1.5 Ace Step 1.5. Turbo and SFT normal and XL model with Ollama:
setup to create up to 4 tracks in a run, 2x Ace1.5 and 2x Ace1.5 XL, each with Turbo and SFT model, to compare (can be individually switched on/off)
VAE changed to tiled Audio VAE decode, uses less Vram.
V1.2 Ace Step 1.5. Turbo and SFT model with Ollama Text to Audio/Song
small update to GUI, system prompts and SFT sampler "engine"
V1.0 Ace Step 1.5. Turbo and SFT model with Ollama Text to Audio/Song
Ace Step uses TAGS and LYRICS to create a song. These can be generated by Ollama or by own prompts.
Download Files:
Ace Step 1.5 TURBO model: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/diffusion_models
Ace Step 1.5 SFT model: https://huggingface.co/ACE-Step/acestep-v15-sft/tree/main (download model.safetensor and rename it)
Ace Step 1.5 XL Models: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/diffusion_models
Ace Step 1.5 XL Model merges (i.e. merge of turbo with SFT): https://huggingface.co/Aryanne/acestep-v15-test-merges/tree/main/xl
Textencoder: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/text_encoders (Qwen_0.6b and Qwen_4b required, 1.7b is a smaller alternative to 4b)
VAE: https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/tree/main/split_files/vae
alternative VAE: https://huggingface.co/megagrump/Ace-Step-1.5-ScragVAE-ComfyUI
Ollama Models, required for tags, lyrics and songtitle, you can choose 1,2 or 3 different models, tags and lyrics might need a bigger model >7b, songtitle can use a smaller model:
https://ollama.com/mirage335/Llama-3-NeuralDaredevil-8B-abliterated-virtuoso (allround model, fast, usable for tags, lyrics and songtitle, recommended)
https://ollama.com/huihui_ai/qwen3-vl-abliterated (for tags and lyrics, able to use thinking)
https://ollama.com/artifish/llama3.2-uncensored (small and fast for songtitle and tags)
Alternative Turbo Models and merges (normal, non XL) :
Turbo continuous: https://huggingface.co/ACE-Step/acestep-v15-turbo-continuous/tree/main
Turbo-Shift1: https://huggingface.co/ACE-Step/acestep-v15-turbo-shift1/tree/main
Turbo-Shift3: https://huggingface.co/ACE-Step/acestep-v15-turbo-shift3/tree/main
Merges of SFT, Turbo and Base model: https://huggingface.co/Aryanne/acestep-v15-test-merges/tree/main
GGUF Models "normal" and XL: https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/tree/main
Which models to start with ?
If you just want to try it first before downloading all those models, start with 1 model only, recommend the Turbo or Turbo XL model.
My current choice for normal model: Turbo-SFT merge_ta_0.5 & Turbo-Shift1, using these settings:
Turbo-SFT_merge model with sampler: er_sde, scheduler: beta57 (or beta), 22 steps
Turbo-Shift1 model with sampler euler, scheduler: normal, 138 steps
XL Model settings:
XL Turbo-SFT merge model: sampler: er_sde, scheduler: sgm_uniform, 40 steps
alternative: sampler: res_s2, scheduler beta57 (requires RES4LYF custom nodes)
XL SFT model: sampler: euler (or res_2s), scheduler: normal, 46 steps, CFG = 7.3, Adaptive Projected Guidance: eta = 1.05, norm_thresh= 1.3, momentum=0.0. Increase norm_thresh as the main parameter. These settings deliver "stabil" output for XL SFT,Base and their merges. The merges sound way better, pure SFT or Base introduce a lot of noise. I bypassed ModelSamplingAuraflow (see node next to model loader node). I think the base-turbo XL model merge fits well in that slot.
Disable "generate_audio_codes" in "TextEncodeAceStep" node to get different results, it works very well for many genres and reduces process time.
Ollama Model: Llama-3-NeuralDaredevil-8b-abliterated
More infos on models see thread below in discussion.
Save Location:
π ComfyUI/
βββ π models/
β βββ π diffusion_models/
β β βββ acestep_v1.5_turbo.safetensors
β βββ π text_encoders/
β β βββ qwen_0.6b_ace15.safetensors
β β βββ qwen_4b_ace15.safetensors (or 1.7b)
β βββ π vae/
β βββ ace_1.5_vae.safetensors
Custom Nodes used:
https://github.com/mattjohnpowell/comfyui-audio-expo (as of V1.7)
optional (use Beta57 scheduler for a bit more punch, requires RES4LYF): https://github.com/ClownsharkBatwing/RES4LYF
Examples various styles:
With Lora:
https://soundcloud.com/tele-joe/city-of-silent-sirens_gothic
https://soundcloud.com/tele-joe/forgotten-blazes_future-bass
No Lora:
https://soundcloud.com/tele-joe/burning-echoes-of-the-fallen
https://soundcloud.com/tele-joe/firestorms_unleashed_turbo_000
https://soundcloud.com/tele-joe/westsidevibesforever_funkhipho
Ollama help:
Install Ollama from https://ollama.com/
download a model: Go to a model page, chose a model , then hit the copy button, i.e. https://ollama.com/mirage335/Llama-3-NeuralDaredevil-8B-abliterated-virtuoso
open terminal and paste the model name, i.e.: ollama run huihui_ai/qwen3-vl-abliterated
model will be downloaded and can be selected in green comfy node "Ollama Connectivity". Hit "Reconnect" to refresh.
Description
Workflow setup to run AceStep 1.5 and 1.5 XL Models
FAQ
Comments (11)
To make instrumental tracks, the explanation in the note is kind of vage, so here a step by step help:
(edit: V1.6 simplified it)
1. mention "instrumental" both in tags and own lyrics prompt (red).
2. switch off Ollama for Lyrics
3. switch off the "switch to instrumental" (grey node below that says "Enable Lyrics System prompts", so "no" means "off" => instrumental ).
Know it is a bit too complicated...
.. so here is another method, even more complicated:
1. use "instrumental" in both Tags and "Prompt to Ollama".
2. Keep Ollama for Lyrics active
3. Keep "switch to instrumental" active (on)
4. Enable "Lyrics System Prompt V2 (alt)"
That way the LLM will understands your wish to have an instrumental track and will even create instructions like below. Tho this depends on the LLM you chose and how well AceStep interprets those instructions.
Example instructions created be the LLM:
"[Instrumental]
[Intro - ambient pad]
[Calm atmosphere builds]
[Piano arpeggios enter softly]
..."
Does the XL model require higher GPU? i'm getting below error
RuntimeError: Error(s) in loading state_dict for AceStepConditionGenerationModel: size mismatch for decoder.scale_shift_table: copying a param with shape torch.Size([1, 2, 2560]) from checkpoint, the shape in current model is torch.Size([1, 2, 2048]).
You might need to update comfyui to nightly version or run the update-comfy.bat within your update folder. That worked for me.
Thanks @tremolo28,Β i believe i did update before using the Workflow. will try this one more time.
@mr_bhuuu738Β comfyui 0.19 is now official, with acestep XL features included, a "normal" update should work now.
All those models can be confusing, here some details:
Turbo: Highest Soundquality, but lack of variation, some instruments sound cheap.
SFT: Lower Soundquality, but more variation, instruments sound more real, but at lower quality.
Base: Worst Soundquality, but most variation, usable mainly for training/finetuning. It is supposed to have best prompt adherence.
Merges between Turbo, SFT and Base: The Turbo-SFT merge combines soundquality and variation.
Turbo models (in particular their merges) deliver usually the best sound quality. XL model sounds more "dry", while normal model sounds more processed.
SFT models can be harder to get good sounding tracks from.
Usually it is the best to keep CFG in sampler at 1 (must for Turbo models), SFT models might work with CFG> 1.
The "acestep_v1.5_SFT_Shift1.safetensors" you have in your workflow is impossible to find. What's the source of it? I can only find the Turbo shift1, or the normal 1.5_SFT.
@jacklinchopsΒ It is supposed to be this file (model.safetensor): https://huggingface.co/ACE-Step/acestep-v15-turbo-shift1/tree/main
I renamed it by mistake to "acestep_v1.5_SFT_Shift1.safetensors", instead of ...Turbo_Shift1.
Turbo_Shift1 is actually the right naming.
Whatever my notes say about SFT_Shift1, I actually refer to the Turbo_Shift1.
Sorry for confusion.
Hi. I can't find the remix and repaint option with analyzing input song and write lyrics, song genre etc. This options are in the original ace-step gradio interface but not a single common explicit word can be used which are commonly used in RAP. It is possible to ad it?
Hi, I might have a look on the other edit options, AceStep has in store, but canΒ΄t promise.
regarding explicit lyrics, this depends on the LLM you use with Ollama. Abliterated models, like the the one I recommende (Llama3- NeuralDaredevil), have no issues to create explicit lyrics, you just need to prompt for it.
seems like cover and repaint features are comming soon for comfyui. There are custom nodes by Ryanontheinside that might support some of these features, but IΒ΄ll wait for comfy release.
